I honestly never though about it as a fire hazard, but i have always use the caps of new batteries to cap the older ones they were replacing. The bulk package of 9V's i'm using have one of the poles capped with a little plastic/rubber cap.