Restrict contacts to just the group they are a member of?

Is is possible to restrict contacts to just the group they are a member of?