+1
Every time I've dug a snow cave I have gotten soaking wet - with sweat, head to toe, even if I try to pace myself. If I didn't have a change of long underwear and a dry sweater I would be shivering from the get-go. And yes, you have to have at least some really good ground insulation and a bivvy sack to crawl into, otherwise you are sleeping in the proverbial refrigerator.
The one part I've been able to improve on from my first snow caving experiences has been keeping my gloves dry - I wear rubber gloves under my insulated gloves, the kind you use to scrub dishes (the gloves they make for spreading grout over floors seem to do the best, though a friend uses an extra heavy pair used for chemical spill cleanup). Hands can get cold, but keep a dry pair always in reserve, you will have a dry pair of insulated gloves to put them in eventually. Its usually after snow camping that younger Scouts learn to carry 2-3 pair of gloves into the snow, whether they're building snow caves or not.
If you always carry a sleeping bag and insulation above the snow line, I much prefer a tent for shelter. The most I have ever had to do is build up a wind block to stop prevailing winds hitting the tent wall. And it also seems much easier to me to hollow out an existing tree well for enough space to sleep or pitch a tent, rather than to start from scratch building a snow cave. Snow caves seem to go better with more builders too.