Plasti-dip
http://www.plastidip.com/consumer/index.html

Home Depot, Lowes.
Just be sure the straw ends are reasonably clean.