Please provide a link to the magnet case you use. I have had a much different experience than you. The first time I used a magnet case hidden on the frame, it fell off. At the time, I was doing a lot of freeway driving at about about 75 mph. Perhaps freeway driving is more likely to cause the magnet to fall (?).
I am not very good with adding photos to this forum, for some reason. So I compared the picture I just took with images online. The magnetic case I have is an identical match to a
Hy-Ko magnetic key holder I think I mentioned I've had mine for around a decade, so I have between 220,000-250,000 miles of it attached to the frame of one of my vehicles.
I didn't buy it from this link and I don't recall what I paid for it. Almost certainly just found it in a supply store.