Yes, if you've "done business" with a company, they or their "authorized representative" can call you. And the definition of "doing business" can be pretty lenient. But if you tell them not to call you, that negates the "permission".

So when you get one of these that you don't want, tell them specifically not to call you again. If they do, you have grounds for a complaint.

Bear in mind that if you do business with the originating company again, you renew the "permission", and will have to withdraw it again.

Be polite, but be firm. "No thank you" isn't good enough. "Do not call me again" is.