I have a small USB hard drive that I use for backups.

On my PC, I have an encrypted volume created with TrueCrypt. I use that volume for everything I want to keep.

To back it up, I just plug the hard drive into a USB port, and copy the encrypted file to the portable drive. Then I stuff the drive into my day rucksack.

I also keep a copy of the encrypted file at work so there's another copy hanging around.

(I do a similar thing with an encrypted volume on my Mac. Details are different, but the principle is the same.)