Generic domain for app development

[Rello]

Hello,

I want to suggest the survey_client app.
it reports general app data accross NC-, PHP-versions and a lot of server statistics.
it is installed in - i would guess - most of the instances and reports anonymous data to a trusted provider (nextcloud themself).

In a PoC last year, I created an Analytics connector for this.
Using this, flexible reporting on this huge dataset can be performed e.g. like the adoption rate of an app (% of installs compared to all server installs).
This could be used to flexibly share app-data to app owners via the existing report-sharing-feature.

The only issue is, that the survey_server would need some love to be updated to the lated NC version (my status from last year).

But regarding to your request, you are also looking for app specific data. Thinking about this, a way for apps to “register” their own metadata to this data feed would be proper concept.

Perhaps @blizzz can join this discussion on the status of the survey_server.

[marcelklehr]

Hey everyone,

Thank you for starting this interesting discussion topic! I understand that a lot of people don’t like data collection. Especially if you don’t know what’s being collected. I myself often disable data collection in the apps I use out of principle. Yet, I can’t deny that I have also considered to add it to the apps I develop, because of the same or similar reasons that @christianlupus outlined. So far, none of my software has data collection, except for the data that app store vendors provide.
I think I’d support an overhaul of the survey_server, possibly along with a statistics tab in the app store like other app stores have, for basic statistics.
Regarding custom collected values, I’m skeptical. I think data can be really useful. But not everyone will want to “leak” their server’s supported ciphers for example, even if to a trusted service. And there are lots of other examples that seem harmless on the face of it but a lot of people would rather not share when asked.
Why don’t we ask the wider community / the users in the next community survey?

Cheers,
Marcel

[deblan]

Hi,

I totally agree with this message and it a real need. I also understand considerations about privacy and data collect.

For now, I only analyse downloads but I miss PHP and Nexctloud versions.

About the domains used and how it works, I have no opinion right now!

The dev of Custom Menu

[Rello]

@deblan

For now, I only analyse downloads but I miss PHP and Nexctloud versions.

I recognize this yes. thats one very common usecase for Analytics.

some impressions based on pretty old survey data (example the contacts app)

[mdw]

About the domains: I have several services in my app that would have benefited from domains like this, e.g. a Demo server, or servers where i host third party services used by the app. But i would never integrate a domain that is owned by someone else than me or the Nextcloud Project. That just has to go wrong. Also there would have to be some kind of control of what’s happening on these domains. This is the internet after all.

For telemetry: Some of my usage statistics have already been used here as a example.
I didn’t want telemetry in my app, but Nextcloud doesn’t share the data they collect like Mozilla does on AMO or Chrome in the Webstore. Knowing what environment the app runs in or when users update helps me make decisions about the development of the app. e.g.:

@Daphne

oh I can guarantee you there is still Nextcloud 15 out there and running

Yeah, 3 people do (in my case). Many users with outdated NC versions also don’t update the app to the final compatible release. I don’t need to care about these. If i can see the data, i can decide that the that the 200+ users on NC 25 are more important to me than the 3 on NC 15. Without it, one user complaint would leave me wondering if i’m dropping support for four-year-out-of-date-software too quickly and i would waste development time on that instead of supporting new versions when/before they come out.
Even just basic data helps me make my app better, e.g. which PHP versions are used, which apps are commonly installed with my app, when do users update Nextcloud or my app etc.

As admin i would rather configure telemetry in one central place than the current wild west of data collection that we have now.

[blizzz]

@Rello

Perhaps @blizzz can join this discussion on the status of the survey_server.

sad status, and no focus on this atm.

[Rello]

so we habe 2 options:

• App devs think about their own way to implement this and might create a privacy debate
• enough community developers speak up and ask Nextcloud for some community love

[christianlupus]

That is well formulated. If it is done centrally, I think that the chances are better that no dev makes a small bug that might cause trouble with leaked data. One could add a third option: some devs in the community stand up and create something working together. So, less work for NC GmbH but the official verification/certification/validation by them.

[christianlupus]

One thing, I want to add to the discussion here: 95% of the comment are related to the data privacy aspect in case of the telemetry of apps. This is not the only topic in this thread. Originally, it was just about a domain to be registered (and kept by e.g. the NC GmbH or some other official entity) to help the devs. Not more!

The idea of telemetry was one aspect of it.
Other aspects range from documentation, user interaction, community building (links to supporting chat rooms) to real-world and app-specific data providers (allowing the app to download the holidays in some countries). This list is for sure not complete and I am sure I do not have the creativity for all possible cases here.

I have to admit that there needs to be rulea on how this might work best to prevent abuse. This thread was started to get a general idea if such a generic domain would be feasible and acceptable. It would allow the app devs to share things/API endpoints without running costs for a domain. I personally would consider it a benefit and service for app devs.

[dartcafe]

Crosspost from Cooperation to get anonymized user data as app developer · Issue #2657 · nextcloud/polls · GitHub

Hey @christianlupus

Although I am interested in telemetrics data, it is difficult to make sure keeping the responsability for the privacy. At least when it comes to GDPR.

Who watches about applying privacy rules?
Where is the data stored?
Who is responsible for data protection and security?
Who guarantees transparancy about the collected data?
How can a user/admin be sure, there is no illegal storage of private data?

I see two options to realize that:

• Use the Nextcloud org to be able to collect data in a centralized way or
• foundig a new organization for Nc developers and collect the data an provide it in a privacy compatible way to the members.

And at last, where does the needed money come from? I.e. for infrastructure, security, organisation, …

[Daphne]

Hi everyone, just a small update. I talked with Frank about if we can get the survey server working again. He agreed it makes sense to get it back up, however until the next release all resources are booked so we cannot get this done any time soon.

Perhaps in the meantime we could think of what the requirements would be and how we can do it in the most responsible and privacy friendly manner.

[MichaIng]

My 3 cents to telemetry:

• While I’m opting out for very most (intransparent) usage data collection, maintaining an own project, I know now how valuable it can be for developers. So e.g. in case of Nextcloud survey app, which is fully opt-in and one can customise and review the data sent, I’m participating and seeing no GDPR issues, at least not from the client side.
• Generally I think it’s a good idea to extend the existing survey app, instead of adding another telemetry facility, also to avoid redundancy when 10 apps all collect data about your Nextcloud and PHP instance separately. With an additional tag to separate core data and data requested by individual apps, it can be still removed from the whole data set on the collection server, e.g. if an app stops being maintained, or detected to collect improper data or such.
• The question remains how the data is stored, how it is made available to developers, and in case, how it is made available publicly. To rule out that data from the same instance is counted multiple times, as far as I can think of, data from each instance needs to be stored separately. But for publishing, IMO, it must be only in a processed way, showing only overall counts/percentages, to avoid any way of identifying or fingerprinting instances. Even an API (whether available publicly or for approved app developers only) which allows queries only, one must take care that with multiple queries it is not possible to strip off individual instance data.

About a general apps domain for other tasks:

• Instead of adding another platform, isn’t it sufficient to create an app category here on the forum, like it exists in most cases, and devs can pin one or more topics there to provide additional information, news, FAQ/docs, links to chat channels and other 3rd party communication/information platforms, if wanted.
• Additional platforms can have upsides, but can also complicate/duplicate communication/information, as it is already partly the case with GitHub and the forum now. So unless there is a specific need which cannot be achieved via forum (Discourse also has a chat feature in the meantime, isn’t it?), I vote against it.

[rawtaz]

TL;DR: Don’t spend time reading the below unless you’re curious, it’s just some general thoughts on what has been discussed and said already. I’m currently not an app developer (yet, but sure looks like I’ll have to do some work soon) so I shouldn’t interfere too much

---

@Daphne

an opt-in (or out, don’t know)

In any and all cases, a very explicit opt-in. Don’t make Nextcloud or its apps one of the softwares that automatically collect data from users unless they explicitly disable this. That is not in line with what Nextcloud is about.

Personally I would never supply or activate telemetry or survey data sending. And if I were to develop apps I would not expect my users to do so either.

---

In general, I don’t think anyone really need to collect information about what PHP or NC versions people run - just clearly state in your app’s release notes which versions are the minimum required ones. I get that you want to know this data to target development efforts, but at the same time you can’t cater to users all the time. It’s up to the users to make sure that they don’t install apps which don’t have requirements met. The decision as to when you will raise the minimum PHP or NC version required is up to your technical requirements and your own feel for when it’s time that everyone should have upgraded or have to upgrade if they haven’t already. Even if some users don’t upgrade, they are still able to run the older versions of NC and the app, if they don’t want to jump on the train of being current and up to date. For example, supporting users who are still on NC 15, or even any unsupported version, makes no sense. If they wan’t to run old and unsupported software, then let them. But that should not hinder you from continuing to develop your apps with modern version requirements of the underlying technologies. Minimum requirements should be driven to a large extent by what makes sense, rather than how lazy users are to upgrade their stack #unpopularopinion

---

Regarding a common app domain, someone has to take responsibility for the running of that and the services on it. Is this something the NC company wants to do (especially seeing how overloaded the company or project is with other work, issues and PRs)? I’m not so sure. Maybe it’s better that whatever an app developer need themselves is managed by themselves. E.g. if I make an app that needs to support receiving some kind of data from the users of the app (explicitly and knowingly by the user of course, such as some AI training data), then I would set that up myself. Why would I want to make my app depend on additional third parties instead of being in control myself of prividing a stable and solid backend for my app? I’m not sure what the point in creating a dependency between the Nextcloud project and an app’s specific needs on that part is.

---

All that said; Nothing is preventing app developers from fully adding telemetry and what not to their apps, this is nothing NC can control. So perhaps the best way to steer that topic in the right direction is for NC to implement a standard way for that type of thing to be done, of course in such a way that there is never any telemetry or data collection being done without the user’s explicit acting on it (including not being lured into it by some hidden away checkbox or what not, it must always be clear as day).

Also, there are already app information on the NC website and app sections in this forum. To do what @christianlupus talked about besides the privacy thing, can’t these two be improved and more tightly integrated, so that we make use of what we have instead of inventing a third place to have app information and interaction?