I'm sure many people will have some great advice and there are many possible ways of doing this, but there's one distinction you might want to think about first, and you can prepare for either or both contingency.
Usually a traditional backup is for a situation when something bad happens to your computer, you replace the part or fix the software glitch, and your aim is to get your computer back running like it was before, including all your old files. Having a hard drive go bad is one example.
But another serious catastrophe is when you simply want your data and can't/don't care if your PC is running. E.g. there's a fire that destroys your house or you need to evacuate from New Orleans to Houston and don't know when you'll be back. In this case, a "backup" is really just a second set of data that won't get wiped out if your computer is destroyed, and which you can access the files without much additional hassle. A basic version of this approach, as you alluded to, is simply dragging your My Documents folder onto a thumb drive which you then store offsite or EDC. If you end up in another town, you can still access your files simply by plugging the thumb drive into another fairly modern computer. It'll be much more difficult for you, in such a situation, if you simply did a traditional backup and just had a stack of DAT backup tapes with you at some random friend's house since you won't have the hardware/software to access your old files.
Personally, I prepare for the latter case. If my computer is fried, I don't mind reinstalling the operating system and software first, then copying all my files back from an external hard drive. On the other hand, some people have very complicated setups, with lot's of software installed, so using the traditional backup will save much time and labor to get yourself up and running again.
Oh, the options that are right for you will also depend on how much data you want to protect, so it would help if you mentioned the amount of stuff you're talking about. You mentioned having a CD burner. If you don't have too much data, then that's a fine way to store data away from your computer. But if you have a huge mp3 or video collection, then you'll need to find a different way.