There are quite a lot of gadgets like that now, older smartphones running windows mobibe can sometimes have a better OS loaded (though the cost of those is going up now because of this), older PDA's nintendo DS, etc. The reason I say to put somehting else on WM devices is the difficulty finding software for WM, the difficulity of getting documents on them without being connected to a windows PC, etc. The Pocket Word, Exel, etc formats you have to convert to (though pocket word is close enough to rtf that those work), pdf viewing is too slow and you can't do much with the calendar/tasks/etc without outlook. Then the OS version fragmentation is pretty bad, My WM Standard 6.1 I have to really hunt for software, you'll find stuff for 6.5 or stuff for pro or stuff that takes .net 2.x. Then the web browser being IE is pretty bed too, since a lot of web sites are going back to true html standard due to the popularity of non IE browsers, the ones without the ie specific code don;t render very well. Some of the older phones you can load another OS such as Linux or Android where you get more use without need to connect to a host PC for file conversions.
Wife has a Nintendo DSi which works pertty well as a document/web browser, you can drop files on the SD card and read them, those are dropping in price since they have announced a new model.
Though not pocketable, netbooks in the 7-9" size are the size of a book, everyone is going to the much larger 10" so the 9" and smaller are getting cheap. Our 9" Asus netbooks are our main systems at home, with 500G drives in each so should we need to leave they are easy to grab and take.