Why any user can view all the contacts belonging to other groups?

For those that end up here looking for a solution. This Link might be helpful in resolving your issue. You will of also want to create groups and put your users into them of course.