As with most things, the simplest solution is usually the best. If all you have are hooks, line, weights and a float, that'll do. I've fished all manner of rivers, streams and lakes in PA, VA and NC and this setup has never failed to catch fish. They're not big fish, mind you, but they make a good meal.
So, for starters you need

BAIT HOOKS, these hooks have little barbs on the back to hold the bait on more securely. Also, remember that a small hook will catch a big fish, but a big hook will not catch a small fish. They're small and cheap, have lots of 'em
LINE 15-20 lb test so it can be used for other things.
SPLIT-SHOT Lead or steel weights that can be pinched on and off the line.
BOBBER I really like the idea of the foam earplug that Boatman gave me, but whatever fits in your kit will do. It'll take a little trial and error, but be sure to match up your float to your weight, if the float's too small or the weight too big, the whole thing will sink; if the weight's too small or the bobber too big, you'll never see/feel the fish take the hook.

A lot of folks here have a gaff hook in thier kits, too. I have never found one to be necessary, but many people do, so I'll include it. I think it would be more useful for other things, and it doesn't take up hardly any room, so I have one.

That's it, until you're comfortable with this setup, there's no need to get more complicated.

Hope this helps,

Chris.