N95's are not overkill for wildfire smoke. The tiniest soot particles from a hot wildfire are so small that they can evade your body's normal airway defenses and get down deep into your lungs, which is bad for you. A dust mask or bandana won't filter out these tiniest particles, but an N95 can. This is a separate problem from any sort of fumes a wildfire can generate from combustion, like carbon monoxide.

Actually, even when the skies clear after the fire, the air can still be loaded with these tiniest of soot particles. The air looks clear because the particles are so small that they don't block the light like larger soot/ash particles do. The only way to know if these smallest particles are around is to check an official air quality report and look at the PM2.5 and PM10 levels (well, or you may be able to "feel it" in your lungs). Since they are so small, they can stay suspended in the air for a while unless there's a breeze to blow them away.

I have a couple boxes of N95's at home (purchased from fellow ETSer Red Flare, too!), not for pandemics, but for wildfires. A big wildfire was upwind of me and poured smoke down on us for almost a week, and I only had two N95's to get me through that week. By day 6, I had to go see a doctor because I was having trouble breathing from all the irritation from the particulates even though I had tried to stay indoors pretty much continuously during that week. I even wore the N95 indoors during the day, but not when I slept, which is probably when my lungs would get irritated. The PM2.5 and PM10 levels stayed sky high for maybe 10-14 days even though the skies had been clear after day 6 or so.

Actually, if you haven't already read it, Wildfire Smoke: A Guide for Public Health Officials, is a very readable source of information.