You are looking for a RACON, they are used by the USCG on entrance buoys.

They transmit a signal (usually a morse code letter such as "A" .-) everytime it receives a RADAR signal.

This morse code letter will appear on the vessel's RADAR directly opposite of where the contact (kayak) is.

Unfortunately, RACON's are expensive, or were. That may have changed. Contact ACR and see if they carry them and for how much.

On the Western Rivers and canals, we report all small boats that we see, via VHF-FM radio, in general broadcasts to other boats in the area. These small boats scare the hell out of us!!!!