ireckon, since it's not clear if you're referring specifically to IMR-type 16340 cells, or just generically to rechargeable CR123 sized cells, I'll keep my answer fairly generic.

I tend to use rechargeable RCR123 cells in my EDC lights (which I use regularly, too), but I also make sure to top them off periodically. Takes very little time to do a top off charge. For versatility, my EDC lights are LED lights that have multiple modes so I can tailor the amount of light to the task. More efficient use of precious battery life that way, especially for a light that I'm not necessarily carrying spare batteries for at all times. However, most multi-mode LED lights are going to slowly drain your batteries to keep the electronics in them active. For example, any multi-mode light that remembers what level you used last is definitely drawing power to remember that setting. Most lights are not bad about this drain, so this is not a universal problem, but sometimes you'll come across a light that will leave you with a dead battery in a couple of months or even in weeks.

For lights that I have stashed primarily for emergency use, like the lights in my cars, I tend to use non-rechargeable batteries and pick lights that either do not draw any power when not on, or which you can adjust to not draw power (or just remove the batteries and store them outside the light). For those lights, I change out the non-rechargeable batteries annually.

So, for lights that you use often or at least handle often, it makes a lot of sense in most most cases to use rechargeables, I think. Much eaiser to make sure that you'll have a full or pretty full charge when you need it. At least if you're always starting fairly full, you have some idea of how much runtime you may have left. With non-rechargeables in a light that you use periodically or regularly, you may not have any idea at all unless you carry a spare cell.