westwindmike has a valid point, and some of the more critical documents/folders should be encrypted. But some, like medical info, you may want to leave in the open, in case you are not conscious to give the appropriate passwords etc.
Like your credit cards/social security number, the physical security of the flash drive at all time is something to be aware of....
There is always a trade off. I used to work in a place where we used to say "we are so secure, we can't get anything done."