In principal you’re right but a personal password for web access is faster changed than light and to prevent any issues in the future you should use app passwords by default (that’s at least my recommendation).
The contacts which you can see on your Android device, have they been synchronized from Nextcloud to the device or did they already exist before you established the connection?
The reason for my question is, that Android devices usually tent to store contact data in a local address book or in the Google address book. You need to make sure that all your contacts are stored in the Nextcloud address book to be able to synchronize it both ways.
A good way to achive this is, to export the contact to a CSV file on your mobile device and to re-import it to the Nextcloud address book afterwards. See e.g.