External calendar invites don't create event in nextcloud

I have an email@website.com account that I receive an calendar invite on (which I accept).
On my cloud.website.com instance I have a user created with same email@website.com that will not see the accepted invite in nextcloud calendar.

I know this is my fault for not using builtin nextcloud email, but I don’t want to have my.name@cloud.website.com account. Is there any way I can make this work?

The only way I was able to make the calendar invite show up was if I accepted the invite from the Nextcloud Mail interface. Accepting from the external email client did not render it.

This kinda forces me to use the NextCloud mail which I’d rather not do. Is there a way to make the external email client work with the invites?