Not something one would carry with them, and not available in all parts of the country, but years ago my wife had poison oak badly, nothing worked, then an old Indian (Native American type) woman took some manzanita leaves, boiled them, drained off the liquid, let it cool (a very important part), bathed her rash in the liquid, and within a couple of hours the rash had dried up, and the itch was gone. In a day or so there was no indication she had ever had it.

But, that not being available to everyone, a couple of tubes of Ivarest, or something similar, could be a good idea. I have some in my first aid kit, even though I have never been bothered by the stuff...