Hi,
I also carry the "Eze Lap Keychain Sharpener" that Jeff mentioned above. I bought it to put in a small survival/hunting kit but it is so handy (with key clip removed) that I have started carrying it in my wallet. It does a reasonable job of touching up a small knife in the field but you have to be very careful as it is easy to cut the fingers of your holding hand (I have a bandage on my thumb right now).
One trouble with having a sharpener on you is that if someone sees you using it you end up sharpening everyone elses knives, not an easy task with a sharpener this small.
Mike