A couple of strategies.
One small kerosene space heater and/or two kerosene (oil)* lamps.
This is enough to heat a small room.
Some extra blankets and sweaters, you can even hang blankets over doorways that don't have doors to close them off a bit.

A baby can be quite happy carried against your body under your bulky sweater, or shawl.
My sister-in-law used a nursing shawl and a baby carrier with all her kids when walking, same idea.

*The mineral spirits sold as paint thinner are better grade kerosene than kerosene fuel is, more like odorless lamp oil but at a lower price*

Also the oil table lamps can be stored out in the open, like on a bookshelf as antiques.
You can get very nice looking glass ones in antique stores as well as new ones.
They put out quite a bit of heat when you fill them and light them.
Most of them burn about 12 hours on a filling.

You can also get the "Hurricane" style oil lamps for about $25.00 each.
The Deitz #80 Blizzard burns for at least 24 hours.
Lehman's sells a version of the Deitz that can function as a small campstove too, the metalwork above the chimney on the lamp lets you set a small pot on it.