If you want to fish as a sport, then it can get pretty complicated with all the different kinds of fish and artificial bates/lures.

If you want to buy a fishing pole, then the simplest setup is a bamboo or fiberglass pole without a reel. This works surprisingly well while shoreline fishing, especially for kids. That is just about all my dad used during most of his life. The next steup up would be a fiberglass pole with a spincast reel. My advice is to go to your local sporting goods store (or Kmart/Target/Walmart) and ask the salesperson for help finding a basic setup that comes with the rod/reel/line already together.

If you want to catch fish, then the simplest method is to use a monofiliment line, tie a hook to one end (Google "fishing knots" for a how-to), put a small lead sinker about 8 inches above the hook (keeps the bait from floating), and then put a float (usually a plastic bobber) on the line above the sinker. The float should be attached so the hook lies a foot or more above the bottom of the lake/stream ... so fish can find the bait.

In general the best all-around bait is worms or any kind of grubs you find under logs and such. Most insects will do too, though you don't want them too big or spiny.

Spear the bait on the hook and throw it out into the water. Then wait for the float to move - indicating a fish is grabbing the bait. At that point you need to jerk the line with a fair amount of force to "set" the hook before the fish trys to swallow the bait. Then pull in your fine catch.

Here is a nice basic web site:
http://www.in.gov/dnr/fishwild/fish/fishing/bobber.htm