"But how hard does it make the water?"
I don't know, but I don't think it would be very much. Wherever I read it, it just showed a bag holding about a double handful of 1/2"-1" limestone chunks. If it were powdered or in the form of granules, it would probably raise the pH quite a bit, but I doubt that small rocks would give off much due to the limited surface area. Besides, it wouldn't be adding lime to the same water indefinitely, as "treated" water would be moving out, and a fresh supply flowing in at semi-regular intervals. And it would be extremely simple (and cheap) to test the pH of the water frequently and adjust the amount of limestone in the bag if it was an issue. Who knows, you might have to add MORE!
According to Wikipedia, rainwater tends to have a pH of a bit under 6 (7 being neutral, higher being alkaline); rain with a pH of under 5.6 is considered acid rain.
Here is some info on rainwater harvesting in Portland, OR, that is approved by the city:
http://www.rwh.in/Googling 'rainwater harvesting' will bring up a lot of information.
Sue