I have only gotten into fishing in the last couple years (barring childhood trips where I just held the rod until there was something for my grandfather to reel in). As such, I do not have as much experience, but perhaps I have some of the newbie stuff still fresher in my head. A couple things from my perspective:

-You don't HAVE to find grubs/worms/whatever for bait. Often a broken rubber band or a piece of shiny metal can attract a fish. There is no exact science to it, but fish will swallow most things that get their attention in the right way.

-A pole just gives you leverage to maximize the strength of your line. If you don't need this, you just need something to hold the line so it won't cut your fingers when the fish is fighting you. I have seen neat kits using a Nalgene bottle for storage and leverage. You can even set that up to unreel the line when you cast it.

-In small creeks, Native Americans would often dam up a portion of the flow and make a narrow channel. When the fish were restricted to this smaller channel they were easier to spot and spear. A stick split twice, with twine and a pebble to separate the sharpened tines, gives you 4 chances to spear the fish instead of one. A net in this situation would work very well too, of course.

Echoing the other replies, the gear is lightweight, takes up little space, and is good for many chores. Add a needle and you can repair gear with the line, for instance.

Finally, sometimes sitting still and keeping calm is the best way to get yourself rescued. Fishing is good for this.