The Contacts app is only working with the Nextcloud CardDAV server. If you already have a CardDAV server, you can move its content to Nextcloud or try to sync both servers outside of Nextcloud. For Linux the Cadaver command line tool might be helpful to setup such a job.
The Contacts app only supports groups in the “CATOGORIES:” field yet, but no VCard groups. A feature request to add this function already exists on Github.