I sharpen my knives based on the stuff I've read from The Razors Edge Book of Sharpening. It was written by John Juranick (check spelling). Basically ANY piece of metal can be sharpened. Not knowing EXACTLY (step by step) what you are doing.....It is tough for me to say what is 'wrong".
I am not getting into this verbatum from the book AND I will say right now it is NOT THE ONLY WAY you can sharpen. That said.....here are a few excerps.
The book states that an angle less than 25 degrees should be used. The shallower the angle the more fine the edge....but the faster it will wear down and more brittle it will be.
If the blade wasn't initially 'scrubbed in' (their term not mine) then you may be working with blade that is too think just behind the edge. This will make it tough to sharpen. It also won't stay sharp for long. I thnk the easiest way to describe what I am trying to say here is think of it as the difference in trying to sharpen a blade that is as thinck as 1 or 2 credit cards vs trying to sharpen one that is a 1/4 inch thinck....the thinner .....the easier.
There are other things to consider but it is easier to get the book and read through the parts you need rather than reading me try to explain it.
Hope this helps some.