How to avoid duplicates from shared address books?

I have my own address book and the shared address book of all users on my Nextcloud.

That shared public address book shows what each user put in their NC profile about themselves and many users are very sparing with their info. For several of those users I already have an entry in my own address book with lots more data - picture, birthday, notes, more contact info etc.

Everywhere contacts are used, in the web interface as well as on my devices, I get duplicate entries for each of these - one richly populated one that I can control and a sparse one that I cannot control or delete. Having them basically treated as two different persons causes problems.

Hiding the shared address book of all users isn’t really an option either, sometimes I do need to look up someone. How to deal with this situation? Is there a way to merge them?

Does anyone have any suggestions?