Hi Kelly,
For a contact, the tag needs to be added to each one who you want to receive an invoice email. The default email address used if there are no tags is the organisation’s primary email address, so if you don’t tag any contacts, that’s the one that will get the invoice.
We have an article which goes into more detail: How can I specify which customer email addresses will receive invoice/delivery note emails.