I think the filter in the cooler is probably not much different from the tap. Filters can be breeding grounds, but usually not at room temperatures do they get nasty. Crank the heat up to 100+ F, and legionella may breed.

The tap on the water filter is probably of more danger than the filter. You might want to have something to keep the mouth of your bottle wiped.

If you are worried, why not get something along the lines of a Brita pitcher, fill it up for the day and let the filter clean it for you. See: http://staging.sftribalddb.com/brita/index_ca.html