Right now I'm trying really hard to talk myself out of buying some Level IV rifle plates!
I'd be worried that I'd run into an attacker who thinks like me. "Hey, I hit him dead center chest and he didn't go down. I'll aim for the head next."
Sounds gruesome, but if you're serious about self defense you have to be willing to think through the various scenarios you might encounter and plan for them. Including an armored assailant.
So head shots are one potential minus for rifle plates. However, they are cheap (relatively speaking), and that's a definite plus for them. An additional plus is that even an untrained attacker will probably shoot for the chest - where the plate is - just by instinct. And a second minus is that they'll most likely pull the shot way low due to lack of training and practice, and get you in the abdomen, below the plate.
So the typical idiot attacker will pull their initial shot and hit you low, then under recoil their second uncontrolled shot will hit you high. Your rifle plate will remain unscathed. You can't win!
(sarcasm)