Is it possible to remove group all_user?

I don’t want users can see all users (even in others group) in autocomplete or sharing list.
I know “share only with users in same group” setting but doesn’t make any difference because there’s a root group called all_user.

You can segregate users into different groups but as every user belong to all_user group, everyone is in the same group at the end.
I’ve tried removing that group but it’s created over and over again.

Is it a way to remove it?

I’ve not played around with it yet but disabling the system addressbook might be a way to go: