I carry a diamond-sharpening card in my wallet behind a business card to protect the wallet and the card. It does double duty keeping an edge on my knives and seems to file down anything that needs it. A piece of wet and dry sand paper folded up could stand in as a compact way to smooth many things up.
Ragweed Forge carries the credit card sized diamond-sharpening card