We took a shotgun with us, not a bit of a problem. Downloaded the forms in advance, took them into the border crossing office, they went out to inspect the weapon (has to be in a locked container, separate from the ammo), took $25 from us, and we were on our way. They did make us throw a couple of apples we had away...
Thats all I thought that you needed for rifles.
As for handguns, thats a different story.