From an engineering standpoint it is pretty hard to make a pointer that won't point accurately. The main weakness is typically not pointing but the bearing binding. Silva has a very well designed bearing and assuming it was assembled correctly at the factory there is little that can go wrong.

Perhaps you got a bad one. Rare, I've never seen one new that was defective, but I suppose it is possible. Or perhaps the compass is a knockoff with a cheap bearing. There are look-alike compasses that are inconsistent. I recently laid out a half dozen of these in a local Wally and there was a few degrees difference that came and went if I shook them. Almost certainly a bearing issue.

I have used a very basic Silva for many years and it points as accurately as the USGI lensatic and surveyor's compasses I have checked it against. More expensive compasses get you a larger dial, fancier case, and features like adjustable declination.

Adjustable declination can be a mixed blessing. I had someone insist that my cheap compass was off because his much more expensive model was pointing differently. We checked it against a known N-S road and his was off by better than ten degrees. Come to find out he set the declination for his trip in the SW and hadn't reset it for the SE. Adjustable declination means you can make it inaccurate. Mine consistently follows the magnetic lines. For true north I need to remember to correct for this and when orienteering, where a variation of five meters over several kilometers can alter your score, I slap on a piece of tape with the declination written on it to remind me and use a pace counter.

Most of my camping and hiking is in Florida and there isn't enough declination or any scoring demanding fine accuracy. I mostly go by landmarks and only need a compass to get my rough orientation.

Discounting issues with ferrous metals causing deviation, a pretty common issue, IMHO the main source of inaccuracy is reading errors. Possibly the biggest detriment to any base-plate model is that they require you to hold it square to you, preferably with both hands, to get an accurate reading. I've seen people who couldn't read my compass accurately because they hold it off to one side or cock it slightly. With a little training they usually got it right.

But there is also parallax error and inability of most people to consistently read and practically use any angle smaller than about three degrees. And variations in eyesight and glasses don't help.

Most compasses intended for hand-held use are graduated in five degree increments. That is about as fine as most people can practically use. USGI lensatic and surveyor's compasses have finer graduations but unless you mount them on a tripod the finer graduations are largely superfluous.