You are right . Sharpening removes material from your blade. It´s important how it´s done though.
When you use conventional sharpener the force for grinding is applied on a relatively big surface with many fine grains. Therefore the bending forces on the edge are small. A rough edge will be smoothed. Shapeners like the Sterling bend the knife edge away from a sharp hard edge. In combination with the leverage of the knife you create a lot of bending stress on the edge. A cheap knife is soft enough to just bend. A high quality knife can´t take that much bending because the steel is more brittle. So you break small chunks out of the edge (rip it up). Once that started the motion will become rougher which will make it even worse. When you look at an edge that has been ripped up with such a sharpener you will notice that the edge gets worse towards the tip where you have the longest lever.