Hard drives are, in my humble opinion, the best option. You can throw an old HD in a USB enclosure and use it on any USB port. Hard drives last for years (I have one running at home right now that has been a main system drive for seven years). They simply age well. They either work great, or they tend to start to fail in spectacular ways which are easy to notice. In order to have extra security, you can get two of them.

Backing up your data is a continuous process, not a one-time event. So you should have no worries about keeping your data for the future. When USB starts to go the way of the dinosaur, you copy the data over to another hard drive connected to whatever the new external interface flavor is.

The real down side of hard drive storage is that the up-front cost can be sizable. If you already have a hard drive lying around, a USB enclosure is only $30 or so. If you are starting from scratch, a portable HD setup can run you around $100 each. So you can only practically have one or two of these. On the other hand, backing up your data requires no special software, and no special procedures. The cost of each backup drops down to a pennys worth of electricity.

DVDs and CDs are better for making a periodic backup that you stash away. Optical discs are easer to mail and store. However, they are less reliable and more trouble to make.

My plan involves a removable hard drive for local backup, and mailing DVDs to a relative in a different part of the country. The hard drive can be used every few days, or every week. DVD backup would be used every month.