I'm assuming the original poster is not concerned with off-shore survival situation since he mentioned using the PFD in his canoe.

I agree over stuffing a panel type PFD makes it uncomfortable for lengthy wear. On occasion, I wear a fanny pack as a "bail-out" kit container in my solo canoe.
When I recently began kayak paddling, I found any pocket or belt based kit uncomfortable and makes getting into or out off a kayak cockpit, especially with a cover in place, difficult at best.
In addition to a basic kit in the PFD pocket(s), take a look at the Lotus Design EFT Pack. It rides high enough on the back of a panel PFD to clear kayak seat backs. The problem is, it's out of production, but may be found yet at some kayak shops or on e-bay.
Take a look at www.watertribe.com for some good canoe/kayak based kits.