It sounds like the spring that keeps the "gate" closed is either broken, incorrectly placed, or missing in yours, but that's just a guess based on your description.

The silver slide button you pull back to open the gate should spring back automatically closing the space where the ball end of the key rings are inserted, keeping them from falling out.

I'm pulling this from memory, so I apologize if I'm way off base here.

I guess another possibility is that the two disc halves could be bent at the pin allowing more space where the balls slide than originally intended, but that's speculation.

Is this the model you have?