A trick that helps with candles, oil lamps, and other sources of flame, is to put a little Museum Wax under the base. It sticks them , in a removable way, to the surface they sit on and goes a long way toward keeping candle stick and oil lamps from sliding off or being knocked over.

http://www.aftosa.com/museumwax.html

Similar products go under various names: quake wax, quake putty, museum putty or similar. The putties seem to be more tolerant of higher temperatures.

This is also handy for other jobs. A P-38 can be stuck to the top of one can in a case with this wax to make sure you always have an opener. This keeps the opener unobscured by tape and obvious to anyone who opens the case.

This stuff can be used to keep things handy and close to where they might be needed in an emergency.