If no security is set, it will default to inherited field rights based on group membership, the highest level being the Everyone group.

At time of creation (of either template or field) default rights will be applied to the field/template. The default initially set by the system is the field/template creator has full rights while the Everyone group has read/create/edit/read security rights. This behavior is completely customizable.

Setting Departmental read rights on a template does not guarantee the user will be able to read all the fields as template/field rights work independently.

Template T contains Field F.

If a user is not given Read rights of Template T, it will not see Field F wherever it is applied within Template T. If Field F is applied independent of a template, or in other templates, the user will be able to see the field.

If a user is not given rights to read Field F, it will not see Field F anywhere in the repository. Regardless of template rights.

My assumption given your statement is the Everyone group can read all fields. To validate this, you can go to View Effective Rights on the Template Security screen. It will show the rights inherited and which group gave those rights to this user.

TL;DR: Leaving Field rights blank, allows inherited field rights to pass through. By default this is set to Read at time of field creation, not any relationship with template read security. Not allowing a user to read a template, will trump field rights, and not allow viewing of any of the templates fields whenever the template is in effect. This will not impare viewing of these fields if they are shared in other templates.

(Manage Templates and Fields overrides all individual template and field security.)