To be clear, when I say I hate integrated rechargeables, I mean the ones that are sealed inside the device, so you can't swap it and you have to recharge the device instead. Most phones today are like that, as are a lot of small flashlights and headlamps. It's a pain, imho. If you turn on your device and it doesn't work, you can't use it until you have waited hours recharging it. And the device itself also becomes disposable when the battery finally stops working. Thus the tons of e-waste from dead phones and so on.

Rechargeables that you can swap out are great, especially in standardized formats like AA (Eneloop), 18650 (lithium ion), etc. Then if your device is discharged, you can swap out the battery for a charged one, as you mention. They do make the device slightly bigger, but imho this is only worth giving up in really tiny devices. Nothing stops you from recharging them with a powerbank, and I've done that sometimes. My bike light has an 18650 and a USB charging port, so I can either swap out the 18650 or plug the light into a charger/powerbank. Both ways work great.