Do you have photos in your contacts? That increases the size of contacts a lot.
You could try to move some contacts of the large groups to new groups to reduce the number of contacts in the larger groups.
Does exporting the entire addressbook work?
Another option would be to export the addressbook with calcardbackup.