I am assuming you're referring mostly to the lightweight gas canisters for backpacking stoves?
Nothing "mostly" about it.

I am referring to the lightweight dome shaped canisters for backpacking stoves.
In which case I would agree with you, I'd never try and refill them.
I believe they can be refilled safely -- just not with 100% propane.
However, I do have a coupler for filling the small 1 lb propane bottles from the larger bulk tanks, but those 1 lb bottles are built a lot heavier than a standard stove canister. Even though I've read refilling those aren't recommended, but I've never had a problem and the risk seems much lower.
The key thing here is to know the max weight before you start and never refill to greater than that weight. Overfilling is a sure way to have problems.
HJ