I think your best bet is aqua blocks or retorts. You can get them from that Major Surplus place that Chris linked to. Nalgene seems kind of silly. Another option is ordinary retail bottled water. A case of 24 0.5 liter bottles is about 6 bucks at Walgreens near here. They might not keep as long as the sealed aquablox, but I've had some in my car for a couple of years and they seem to have held up ok. There's a kind you can get that has a squirt cap, so that would take care of the contamination thing.

If you're talking about stashing water in fixed locations, though, you don't have to worry so much about weight, and in fact maybe you want to fill some 55 gallon drums or something like that. Then have some kind of purification device available in case the water has gotten funky in storage.