These filled 5 gallon water bottles run about $7.00 here, if you return your old empty one when you buy the new one.  If you don't return the empty, a new one costs about $14.  You can find them in the grocery stores, office supply stores, home improvement stores, etc.
Sooo ... you can "buy" one of these nice 5 gallon water bottles to keep (just don't return it) for $7 if you want.  Not collapsible, but built sturdily.  And they have a handle to carry with and a nice neck for pouring (although there really didn't intend them to be poured I don't thing).  Once you've removed the cap, you can smack it back in place later, and it seals up nicely upon re-use (the lower part of the cap is torn off and thrown away however).  I don't know how many times you could re-use the cap, since we throw ours away 95% of the time.  You could fashion a new "cap" with Saran Wrap and a rubber band though.  You can also get a 3 gallon version for slightly less money, if you can't manage the larger bottle.
We have these delivered (for a small additional cost), and we always keep around 6 bottles in queue in addition to the one that sits in the water chiller/dispenser.  We cycle through the queue, first in first out, so the water we have on hand is "fresh" (no more than "6 bottles old" - the size of our queue).
