I'll be a bit of a contrarian here. I've posted on BisA in the past and my thinking has evolved some since then. I haven't worried about soy-based compounds, so I'll limit my remarks to Bisphenol-A, but the truth is that we really don't know if BisA does cause the changes that scientists see in animals. There are actually quite a lot of animal studies of very low concentrations that show the possibility of harmful changes for people. But, without the proper human studies, we really can't say one way or the other. The EPA says it is safe, but that is based on very limited data so far.
Honestly, though, it may be very difficult to observe harmful changes in humans just because we live so long and have such varied diets and lifestyles. There's just a lot of other crap in our lives that cause all sorts of similar or competing health problems. Someone just posted a thread about Chernobyl so I'll use that to illustrate this point. I posted about this a while ago, too.
We all "know" that radiation causes cancer, and Chernobyl spewed a ton of radioactivity into the atmosphere, much of it settling on the area near the plant. It's everywhere--in the air, water, soil, etc. Well, even after studying affected populations in Russia, Ukraine, and other countries for the past 20 years, scientists have only seen a small increase in one specific type of cancer (and only in children) above the normal rates of cancer. Does that mean that no one got cancer from radioactivity? Hard to say based on the evidence. These people may also smoke, have crappy diets, bad genes, etc. and it's hard to tease out the contribution of Chernobyl when you don't see a dramatic and obvious spike in cancers after the accident.
So, are you being paranoid? Well, I'd say the jury is still out, and we may never really definitively answer the question. But watch Braveheart and chow down on some haggis in the meantime, laddie.