OK, I know this is going to sound silly. . .
when you were in the sleeping bags, did you have a wool or other warm cap on in addition to the mummy bag top?

The head radiates a lot of heat. If you are losing too much heat through your head, the body lets other parts cool down (e.g. the feet) in order to keep the brain working.

So, maybe warmer headgear would help, or, as Grandma said:
"put a lid on the teakettle."