Here is what I currently have in my bike tool kit…my riding ranges from errands around town to day-trip rides to overnight/weekend tours, all on-road:
-frame pump (spring-loaded, I attach it behind the seat tube)
-spare tube
-tire levers
-patch kit
-small adjustable wrench
-Topeak multi-tool with allen wrenches/screwdrivers
-mini chain breaker
-spoke wrench
-Gerber multi-tool with pliers (I believe the model is called Diesel)
-pair of work gloves (drop your chain on a ride and you will appreciate why these are a good idea)
-a couple of tie-down straps for attaching things to rear rack
-spare rack/fender bolts (these can vibrate loose--it helps to apply some blue Loctite to prevent this problem)

All except the pump is in an Ortlieb handlebar bag that has a quick release so it can be detached when I am off the bike…there is also a shoulder strap for carrying. It's nice for running errands etc. because I can put my wallet/keys/phone in it as well. I also carry water/snacks/extra clothing as appropriate for the ride I am doing…anything that doesn't fit in the handlebar bag goes in a trunk bag on the rear rack; my bike lock is generally in that bag also. Usually I have detachable front and rear lights in the handlebar bag as well…they pretty much stay in there unless charging or being used.

The above is what I carry on all rides…overnight trips involve more gear and panniers are definitely needed (unless you use a trailer). Incidentally, you can make decent, inexpensive panniers out of square plastic buckets with a few simple pieces of hardware. The first aid kit somebody mentioned is a good idea…I do carry one on longer rides but probably should just keep it in the bag all the time. And ditto to taking a bike maintenance/repair class! I actually volunteer at a community bike shop in my town and have learned a lot there (how to basically overhaul a bike)…as a result, I am able to take care of most of my own maintenance and repairs provided I have the right tools.