Handling a large number (>1000) of contacts in the web-UI

🍱 Features & apps 📇 Contacts
1

@irgendwie Many thanks Alexander for this great update! That’s a wonderful load of improvements.

I’ve installed 2.0.1. on all 5 instances which I manage, works fine. I myself hardly use the Nextcloud GUI at all, it’s all DAV for me (Thunderbird-Lightning, Thunderbird-Cardbook, Nextcloud Client). Which all works fine.

But one of my clients initially loaded an address book with THOUSANDS of contacts, and we have never been able to use it since in the GUI. When I now read “Introducing lazy loading, faster initial loading #88”, I hoped that his problem would be solved. But not so. There is still a very basic problem in the software architecture - i.e. trying to load all the contacts at once and handle them in JavaScript. That idea just HAS to break on larger amounts of data. Firefox keeps complaining that a script is running for too long, after several times of “keep on running it” every user will give up. I did note that the contacts app now seems to have a display limit of 999 set in it, but this does not solve the basic problem.

I do remember starting or adding to a different thread on this topic a year ago or so, in which I detailed some ideas to have real lazy loading. The main problem is that you want to select / sort by categories in the GUI, which means that the GUI needs ALL the contacts. IMHO this should be done by server requests - which indeed may be a bit slower on small amounts of contacts - but will work for LARGE amounts of contacts.

My own personal contacts - grouped in three different books - total to 969 contacts, and are sluggish in loading, but it works. What I really miss is the option to select WHICH of those three books are to be displayed in the GUI: I would only really need one in everyday use, the other two are my archive and my girlfriends book, both of which I only seldom need. When I click on “Einstellungen” a list of address books slides up, at the end of each address book line are three dots for a context menu. That menu offers: Copy link / Download / Rename / Delete. It would be GREAT if it could also offer a checkbox to include/exclude the book from being loaded.

Those two features I have noted above would make the contacts app - finally - into something which I could wholly recommend for professional use - it’s the only reason I don’t (yet). Obviously I would love to!

The bees knees would be an option to use either client-side or server-side category selection :star_struck:

Finally: Let’s not loose perspective! You guys have done a really great job this far - THANKS!

Tim

2

Hi @timreeves !
First, thanks for taking your time and writing this constructive feedback!

It needs to be said, that the lazy loading only works with Nextcloud 12+. A back port only for additional functionality wasn’t accepted (see [11] [3rdparty] bump sabre/dav to 3.2.2 by irgendwie · Pull Request #4729 · nextcloud/server · GitHub). Maybe this is the reason why it still isn’t that fast on your side?

Not directly, the sidebar only shows 999+, if the number of contacts in a group is over 999. Still all contacts are loaded if I’m not mistaken - would need to check the code to be 100% sure.

We already have an open issue for this (ref. Possibility to filter by account · Issue #36 · nextcloud/contacts · GitHub) but it wasn’t implemented until now. Maybe we are able to add this for 2.1.0!

Regards,
Alex

3

Hi Alex,
thanks to you for making this an own issue!

All my NC instances are 12.0.3…

So it would seem, going by the script timeouts in the browser :grinning:

I’m very glad to hear that and have plus-oned it - really hope it does indeed appear in 2.1.0!

Resounding silence on this one, eh? :wink:

I fully understand that re server-side selection by category I’m talking about a major rework or extension of the app - and oh boy I sure do know that it means a lot of hard work! And that no-one is stumping up to pay for it. That said, this problem is the last hurdle that I see to be able to recommend Nextcloud as a total groupware solution for businesses (KMUs) - 'cos they sure are going to have more than 1.000 contacts.

Another idea I had: Fork the contacts app into two versions: The free “normal user” (current) version to handle up to ca. 1.000 contacts in total; and a “commercial user” version using server-side heavy lifting (and “real” lazy loading in the GUI) - as a non-free app. Businesses should pay a reasonable sum for it! I’d be at the head of the queue for at least 2 licenses.

Again, many thanks, really - it’s so easy to forget all the skilled hours donated when we are winging about some problem - but I never do. So THANKS!

Tim

4

It was a design choice which was taken when the new Nextcloud contacts app was initiated. With the DAV back end in nextcloud/server it was decided, that the contacts app should act as a simple Javascript front end to show your contacts and nothing more. :wink:
I must admit I myself never had in mind to have that amount of contacts in the address book - but I don’t come from a business perspective. Additionally I’m coding on this app in my free time, as a hobby and host Nextcloud for my friends and family - as it is for a lot of the developers working on the contacts app.
As it is with open source, everyone can work on the app and implement the features one wants.

But as a small outlook - It could be done “relatively” easily, when storing the groups in separate vcards (as it is done by many services by now). These vcards could be queried in the beginning, such that when selecting a group only the contacts of that group are queried.
Additionally we could limit the number of contacts loaded in the beginning and reload additional contacts when scrolling down the list, etc. :+1:

There are many possibilities as you can see but the main problem is, someone needs to implement this - and it needs time, even more if you can’t or don’t want to work on it in full time. Be it due to money or that working on the contacts app isn’t the main job of many developers and they are simply investing their spare time to work on an interesting project.
If you really are in need for better handling of that issue, maybe think about offering a bounty for it to attract more/new developers: https://www.bountysource.com/teams/nextcloud/issues
Although this also is no guarantee that someone will work on the issue! :slight_smile:

I hope this answer helps you in some way :smiley: And don’t get me wrong - I totally understand your point of view (also business wise) and I don’t want to appear greedy or ungrateful or anything. It’s just my point of view on the open source subject matter.

5

Dear Alexander,

again thank you so much for taking the time to explain the situation. It is in fact pretty much as I imagined it would be - but it always drives me crazy not knowing exactly what a situation is and how its future perspectives are, then you can’t plan or advise clients properly.

In principle I would be glad to help “Big-Contacts” to happen, but my personal situation does not permit - I’m 63, my PHP is getting rusty and my old-age provision is inadequate, so I can’t really contribute skills or money. Under other circumstances I gladly would.

That’s exactly what I’ve been dreaming of for well over a year. :grin: And the group vCard idea sounds interesting but I don’t quite get it: Would it be such a strain on the DB to select categories (groups) from the existing DB table structures, perhaps by adding an index or foreign key or intermediating table?

Thanks again,

Tim

6

It wouldn’t be, but like I said - it was a design choice to go without “server side” (PHP) solutions and do everything in JavaScript. Nonetheless we should be able to improve the performance in such a way, that even those big amounts of contacts shouldn’t be a problem. CardDAV should provide enough functionality for that. It’s just a matter of time and effort to implement it.