Technically the template rights should have no impact on the fields - the security is completely independent. However, so many people assumed that removing the ability to see the template should also remove the ability to see the fields in context with that template that we added a client side check that if a document had a template assigned that the user couldn't view, they wouldn't be able to open it up and see the fields of that template. You should still be able to see the fields in other templates or in contexts outside of that specific template though. The fact that you can't add the field to the entry listing columns seems to be a quirk of this behavior. I'm not sure that's intentional.

Update: Actually, thinking about it a bit more, it probably is simply that the fields are grouped by template in the UI and so it doesn't show them (when it probably should show them as fields outside templates). I'm not sure if that's possible to easily address but I'll follow up.