Start with TeacherRO's first question "What should I plan for". This is the basis of a risk assessment and will get some different answers depending where you live.

As an example I like to use house fire since that is something that can affect everyone.
Imagine the scenario: its winter, your asleep and are woken up by the smoke alarm and smell of smoke. What do you do? You find you can't get out the main entrance of the house. Do you have a flashlight within reach so you can see? Do you have a backpack with a change of clothes, shoes, etc in the room? Can you get clothes on you and the wife and baby, smash a window and get out? Can you shelter somewhere after and call the fire department?

I've known two families that went through this scenario. First I was a teen living in my parents house in the country. Middle of the night the teen aged son of the neighbor a mile up the road was knocking on the door. Let him in and he used out phone to call the fire department. He was wearing only his pajamas, no coat, hat, gloves, socks or shoes and had ran through the snow to get to out house. Their shoes, coats and car keys were all in the living room where the old wood stove caught fire so they had to go out the windows with little protecting from the elements.
Second was just a couple years ago in a suburb. Neighbor dropped a blanket behind the dryer and it caught fire. He ran upstairs, got his wife and kids and cat and went out the door. They did have cell phones in their pockets to call 911. neighbors gave them a car carrier, raincoats, etc. After the fire was out they were able to get keys and a wallet and go to a hotel.
So my lessons learned from this scenario and preps include: Go bag has enough clothes to be comfortable and it kept in the bedroom so I can reach it if I can't get out of the bedroom. Kids have theirs in their rooms and are instructed to swing the backpacks at the window if they can't get them open. Phones and vehicle keys are kept in the bedrooms with us once we retire to the bedroom for the night. I've seen many people with keys by the front door and cell phone charging stations there as well.
Next was a good fireproof safe and expensive keepsakes/heirlooms/pictures/important documents/computer backups there. My MIL for example leaves old pictures in the picture frame in her living room behind new pictures and says if her house is ever on fire all she has to do is grab all the pictures off the walls. If my house is on fire I'm not going to run around gathering stuff. Pictures on my walls are copies made on my own photo printer/copier and originals are scanned and kept in the safe.

You mention baby, so thats another layer of planning.
I took the freebie diaper bags the hospital gave us and kept them stocked and in each vehicle. Run out of diaper bags in the one you normally use you have a backup in the vehicle. Have to run out of town for a relatives funeral, You have some items pre-packed.

So think of all the scenarios. Is flood a risk in your area, tornados, etc.
I planned for fire since it is a risk for everyone and I've personally known two others to go through it. I plan for the trip out of town as my relatives are in another state. So thats sort of a "bug" out scenario where someone gets hurt and you have to go quick. I keep a travel bag pre-packed for each. I keep a "bob" for each which is your typical bob, food, water, gear that we take on those trips (what if you break down/get stuck along the way). I then keep a larger backpack that goes in the back which supplements the smaller packs we take. After listing all our "what to plan for's" I built a modular tiered system.