[Discussion] Why is contact syncing so . . . convoluted?

A quick google search shows the Outlook app has several issues syncing contacts properly, all with separate proposed fixes/workarounds. I have lots of non-google apps that play nicely with DAVx5 such as whatsapp, signal, facebook, K9 email and so on, sync between all of them is instantaneous.

Sorry sounds very much like you’ve hit a bug in the outlook app

I don’t want to hold up Outlook as a “good” example - any integration with Outlook has been pretty terrible, historically. I was just using it as “an” example.

Let’s use K9. Does it reference the Android contact store to pull and manage contacts?

Yep K9 uses the contact store flawlessly. I won’t lie it’s a bit ugly, I’ve been meaning to look at fairmail witch looks to be a fork of k9 that’s had a fair amount of ux work done on it. Might be worth a look

I don’t think I’m explaining this correctly. Here’s a picture.Nextcloud%20Contacts

My understanding is that DAVx5 is a syncadapter and is what “adapts” webdav communications into an android addressbook device side.

Nextcloud strictly serves the addressbook through the webdav communication standard using saberdav. The android address book is a storage implementation not a communication method. As you can see from the diagram below doing it this way keeps the Nextcloud server standard and generic where as individual clients are expected to adapt the data as needed. The same happens with the desktop Outlook plugin, it syncs using webdav and adapts the data to be stored in Outlook.

As far as android and android apps are concerned it’s now just another standard addressbook, separate from your gmail (or google) address book
image

2 Likes

Instead of reinventing the wheel you could also run a Z-Push server that interacts with your Nextcloud server and provides ActiveSync support for all kinds of clients that support this (Android and Outlook have native support for it for example). Here is a short howto on how to set it up:

So, my understanding of SyncAdapters is completely different (and I could be completely wrong). From Create a sync adapter  |  Connectivity  |  Android Developers

The sync adapter component in your app encapsulates the code for the tasks that transfer data between the device and a server. Based on the scheduling and triggers you provide in your app, the sync adapter framework runs the code in the sync adapter component.

From what I gather, SyncAdapters are like cron jobs + some sort of data transfer protocol (I have yet to see an example of that format). They provide asynchronous data transfer and updates between an app and a cloud service, not a means for apps to communicate with the Android system. It is what Nextcloud is currently using to sync and update files between the device and the Nextcloud instance.

The API call that DAVx5 and other apps use to interact with the address book is ContactsContract. That is what allows contacts to be created, read, updated, and deleted by third-party apps.

My suggestion is to eliminate the need to use CardDAV protocol on Android since it is non-native. Instead, use the native SyncAdapter API call (protocol?) to send the data to the server. Then, since Nextcloud uses CardDAV (and since it is FAR easier to implement the CardDAV protocol in PHP), create a “bridge” between what is received by the server-side SyncAdapter listener and Nextcloud. That way, when data is received, it will be converted it to vCard format and sent to Nextcloud over CardDAV (via a localhost connection).

This keeps everything native to the respective platforms, would likely integrate much more seamlessly, and wouldn’t break any existing functionality. Let me know if I’ve misunderstood how these APIs and protocols work - since I only just learned about the ones on Android last week, it’s entirely possible.

From what I gather, SyncAdapters are like cron jobs + some sort of data transfer protocol

Yes that is my understanding. But you still need to translate said data into something that can be used. You need to do this with any API, For example how would android know what to do with a json response? At the very least you need to dump it into a database for later use.

It is what Nextcloud is currently using to sync and update files between the device and the Nextcloud instance.

Yes the Nextcloud app uses Webdav in order to sync files, the same (extended http) protocol DAVx5 uses to get your contacts

The API call that DAVx5 and other apps use to interact with the address book is ContactsContract. That is what allows contacts to be created, read, updated, and deleted by third-party apps.

Ok, but you don’t communicate with a remote API the same way you communicate with a device side API. There are a few considerations like data sanitation, security concerns and general efficiency that prevent doing that.

My suggestion is to eliminate the need to use CardDAV protocol on Android since it is non-native.

The Webdav protocol is just and extended HTTP request, nothing non-native about that. WebDAV - Wikipedia Even if you used a Rest API you would still need to tell Android what to do with it. So why not use a good web standard.

create a “bridge” between what is received by the server-side SyncAdapter listener and Nextcloud.

Saberdav (aka Nextcloud Webdav library) is the “SyncAdapter listener” as you call it. There is no such thing as a server-side “SyncAdapter listener”, SyncAdapters are only an Android thing. Google has it’s own SyncAdapter talking to their proprietary contact back end then they translate that data into Android contacts. That said they may still be using Cardav themselves, I haven’t checked if that’s still available for a few years.

This keeps everything native to the respective platforms, would likely integrate much more seamlessly, and wouldn’t break any existing functionality.

This is actually the whole point of using a Web standard like Webdav. All data translation is done device side natively where things can change often, the Webdav standard is well defined and been around a long time so is a stable target. The DAVx5 project has years of work built in dealing with all the changes Android has had, along with the device specific fixes, hence why the Nexcloud dev’s wouldn’t want to re-do all the work already done. Even if you didn’t use Webdav you would still need to process the data on Android’s end anyway, you would just be re-doing all the existing work for little to no benefit.

As always this is based on my best understanding, I’m only a novice coder

No, I get what CardDAV/WebDAV are (though admittedly, I have no direct experience utilizing them). My comment about CardDAV being non-native to Android was the fact that it provides no interface for it - something like DAVx5 is required. As you said in another message, it’s a complicated app. Since that’s the case, why do the conversion to vCard and use a CardDAV app on the client? just in order to say, “We sent the data via CardDAV - huzzah?” That’s silly and complicated.

What format does a ContactsContract call use to dump contracts/what format does a SyncAdapter use when sending data? Is it just JSON? Something else? Why not send the data to the server in the format that Android provides and then convert/sync it using CardDAV server-side? My whole thought is to avoid having to use a third-party app just for the simple task of syncing contacts whilst maintaining adherence to CardDAV. just use the most default protocols provided by Android and offload all of the difficult stuff to the server.

I think this is the main misunderstanding. Syncing data as varied and extensible as contacts is NOT simple. You also have to deal with edits, deletes and potentially thousands of contacts, that’s a lot of technical problems to deal with.

Because CardDAV is a Standard specifically designed for this task. You would basically just be re-inventing the wheel if you wanted to do something else and that would be even more complicated, mainly because you would have to write your own specification.

Typically how data is presented by internal API’s is not conclusive for transport over the network. When you sync over the network you want to do it as lightly as possible by using UID’s and other things rather than dumping a large blob of data back and forth. Something you deal with by using a Webdav client

There is no default protocol. So I’m not sure what you mean by this? DAVx5 is just doing http requests through the SyncAdapter, this isn’t anything unusual.

1 Like

Well, how do other apps do it without requiring a third-party app like DAVx5? Maybe we should start there. Dozens if not hundreds of apps are able to do this in a way that is seamless and straightforward for typical end-users. You can tell me till you’re blue in the face that the majority of Nextcloud users are technically-competent and prefer F-Droid, and I’ll respond by saying that if you want to appeal to a broader user base that extends beyond such people, things like this really need to be addressed. So, how might it be addressed? I put forth a (potentially half-baked yet well-meaning) suggestion. Got any ideas?

2 Likes

Perhaps a middle way is to build a link between DAVx5 and the Nextcloud client so that it automatically can find the user’s account? This is how some of the other Nextcloud apps work.

2 Likes

Which apps? I’m curious.

I’d think that the preferable solution would be to avoid the need for DAVx5 altogether if possible, but anything is better than nothing . . .

I think that Nextcloud Talk and SMS/phone sync does it.

Here is how it looks like in Talk. See the Import… link?

1 Like

Good thought, but it says that it’s pulling contacts from the Nextcloud app . . . which is using the convoluted DAVx5 setup that I think needs to die. It’s not connecting directly to Android’s contact storage or using its own management system.

I did a search of the code and couldn’t find any calls to ContactsContract or DAVx5, but I did find where it is connecting to the local Nextcloud app to pull contacts. So, that’s unfortunately not too helpful.

On a side note with Phone Sync, I would love to see it be able to send texts much like how https://messages.android.com works. Turns out that the KDE Connect team is almost done with their own standalone SMS syncing app. I tried to revitalize a feature request for Phone Sync over here:
https://github.com/nextcloud/ocsms/issues/4

Maybe I should start a new thread about this? It’s #2 on my Nextcloud wish list.

I’m not sure I follow you. It’s simple. Currently the Nextcloud app does not have a builtin contact client. Instead you should use the stand-alone DAVx5, or another CardDAV client to sync contacts.

As for Android, it has no native support for migrating contacts between different accounts. This is not unique to Nextcloud. Perhaps some apps can do it (Facebook, Skype, etc) but it isn’t an Android feature.

So I am thinking that you want Nextcloud to builds in contact support in the Nextcloud app? It would be good. Another option, perhaps easier, is to adapt DAVx5 so it can import accounts from the Nextcloud App?

Edit: you can support here https://forums.bitfire.at/topic/2121/please-support-nextclouds-singlesignon-sso/8

1 Like

Imagine you’re an employee at Hobby Lobby, you have just enough technical know-how to click “deploy” on a cloud-based VM of Nextcloud, and your primary computing device is your phone. Is it still simple?

It can be improved. If you follow the link to DAVx5 you see they are adding support. It works pretty easy now :smile:
SSO is still not there though.

1 Like

Hello!

I would consider myself somewhat techical competent and I want to join the ship saying that CardDav/Nextcloud/Davx5 stuff is hard to set up as it is today, hands down. The amount of effort required to do a simple thing (sync contacts) seems to be unituitive. Although I cannot come with concrete suggestions as you may have, I want to share my experience doing it. Reason I want to share my experience is because I yet haven’t figured out how to set it up properly and…:

  1. Which way the contacts sync and what is required to move entirely over to it
  2. Which way calendars sync, and why can’t i sync my local calendars on my phone up to nextcloud? (tried this).
  3. Why do we use dav for it? As mentioned above, can an alternative method be used and baked in to the nextcloud app on the mobile devices, and be an alternative you flip in your settings to enable/disable? What is limiting that today? About the google discussion above, I would say a huge amount of us are using google play services as long as we use android. The people that is anti-google 100%, feel free to continue using webdav because i respect that :+1:

Although, I am not directly criticising nextcloud because I am super happy with it, but I am just curious and want to follow this thread forward. @summersab very interesting topic you brought up.
den

1 Like

Thanks, @denNorske. Again, I’m really not trying to be a jerk about this. I just think that the current state of contact syncing stands in stark contrast to the highly-polished product that is Nextcloud, and I believe there is room for discussion for how to make it better.

1 Like