Nextcloud Ethical AI Rating

Originally published at: https://nextcloud.com/blog/nextcloud-ethical-ai-rating/

Now that Hub 4 has been released, it’s time to introduce the Nextcloud Ethical AI Rating.

Progressively, there are more and more risks associated with computer intelligence, and as a transparent software company we have the responsibility to intervene and protect our users.

Recently, Microsoft laid off its entire ethics and society team, the team that taught employees how to make AI tools responsibly. Nextcloud on the other hand, embraces the ethics and challenges that make up today’s AI and aim to take them head on.

Challenges

The field of AI is moving fast, and many of the new capabilities face ethical and even legal challenges.

For example, there are problems with:

  • Privacy and security of user data
  • Discrimination and biases
  • Energy usage

In particular with neural networks based AI technologies, the mere availability of open source code is no longer enough to be able to say you are in control over your data or that the software is safe or ethical. The set of data and the software used in the training, as well as the availability of the final model are all factors that determine the amount of freedom and control a user has.

Ethical AI standards

Until Hub 3, we succeeded in offering features like related resources, recommended files, our priority inbox and even face and object recognition without reliance on proprietary blobs or third party servers.

Yet, while there is a large community developing ethical, safe and privacy-respecting technologies, there are many other relevant technologies users might want to use.

With Hub 4, we want to provide users these cutting-edge technologies – but also be transparent. For some use cases, ChatGPT might be a reasonable solution, while for other data, it is paramount to have a local, on-prem, open solution. To differentiate these, we developed an Ethical AI Rating.

Ethical AI rating rules

Our Ethical AI Rating is designed to give a quick insight into the ethical implications of a particular integration of AI in Nextcloud. We of course still encourage users to look more deeply into the specific solution they use, but hope that this way we simplify the choice for the majority of our users and customers.

The rating has four levels:

Red 😡

Orange 😩

Yellow 🙁

Green 😊

And is based on points from these factors:

Is the software (both for inferencing and training) open source?

Is the trained model freely available for self-hosting?

Is the training data available and free to use?

If all of these points are met, we give it a Green 😊 label. If none are met, it is Red 😡. If 1 condition is met, it is Orange 😩 and if 2 conditions are met, Yellow 🙁.

We add one additional note to the rating: bias. As it is impractical to prove there is no bias, we merely point out if, at time of our last check, major biases (like discrimination on race or gender for a face recognition technology for example) were discovered in the data set or in the expression of the model.

There are other ethical considerations for AI, of course. There are legal challenges around the used data sets (in particular, copyright) and the energy usage of especially deep neural networks is of great concern. However, unfortunately, those concerns are extremely hard to quantify in an objective manner and while we intend to try to warn users of any open issues, we can not (yet) include them in our rating.

For that reason, we recommend users to investigate for themselves what the consequences of the use of AI are for their individual case using the Nextcloud Ethical AI Rating.

3 Likes

Now I’m surely not the only one wanting to know which AI services you have rated so far, and what their rating is. :thinking:

OK, I see the OpenAI integration rated as “Neutral”/yellow. But still: is there a rating list somewhere?

Btw., which point is not met by OpenAI?

That’s the wrong rating @jondo , the rating you refer to is only a user rating, and does not have anything to do with the ethical AI rating. I’m sorry for the confusion.

you can watch our release presentation back soon and there you can see the ratings.

In the future we will also publish this in the app store.

Please further clarify what you mean by training data as “free to use” so it is not misinterpreted by definition. Thanks.

That green looks very yellow to me

1 Like

What would you suggest to add for your green?

I guess he refers to the colour of the icon. “Green :blush:” also looks yellow to me.

I wonder if this gimmick has any negative impact on the speed of Nextclouds more important functions of file handling - synch upload download? Can it be turned off? Or removed completely?

All of it is extra apps do you don’t have to enable it. None are on by default.

1 Like

While I agree in principle, just to point out how difficult such a rating would be, e.g. so called “discrimination” in face recognition technology.

There’s the physics of photography, i.e, LIGHT, and less light, less contrast, the more difficult to recognize things. That’s why night vision is more difficult than day time vision.

So, if a face recognition algorithm is less accurate with dark skinned people, it’s not “discrimination”, or at least not a-priori, but simply the limitations of physics: lower contrast pictures lead to lower accuracy.

As a matter of fact, it would be surprising if there were NO difference in accuracy.

So, if activists yell about supposed discrimination, it warrants extreme caution before such a claim is taken up and incorporated into a rating, because one really needs to break down the actual factors of how a particular discrepancy comes to be.

Discrimination requires INTENT, or at least willful neglect, it’s not the side effect of physics.
If there’s actual intentional discrimination, then that, of course, should be flagged, and not just flagged, but condemned. But people are too quick to jump to conclusions when something deviates from their utopia of equal outcomes despite unequal inputs.

They already anticipated this in the sentence you quoted.

As it is impractical to prove there is no bias, we merely point out if, at time of our last check, major biases (like discrimination on race or gender for a face recognition technology for example) were discovered in the data set or in the expression of the model.

A few more general thoughts on the subject:

The biggest problem with all these discussions about social justice, political correctness, privacy etc, in combination with A.I., from conservative latent racist right wing people all the way over to leftist social justice warriors, is that everyone always brings up a subset of a more general issue and then propose measures that only address that specific subset. The bigger picture is usually ignored.

Technology, just like we human beeings, is pretty much what it is, an it will never be perfect. And while I’m not saying it should left completely unregulated, neither ratings nor bans will ever change that fact.

What is important is how we as a society use this technology. The most important thing for me is, that institutions, such as politics, law enforcement, and especially the judiciary, must never make definitive decisions based solely on what the technology says. Technology can be a tool, but it must always be manually cross-checked before drawing final conclusions or making final decisions. A.I. should never be the sole basis of evidence.

But I don’t see how any of this would be an issue on your personal Nextcloud instance, where things are hosted locally, and then maybe a few dark-skinned people are going to be detected wrong on your personal holiday pictures.

I mean don’t get me wrong, this should definitely be improved and it will over time, but I think in this case it is enough if Nextcloud provides general ratings and guidelines so that you can roughly classify the tools. The rest is your responsibility. You can blindly trust the ratings and the A.I., or you can continue to use your eyes and simply see the A.I. as an additional tool. :slight_smile:

2 Likes

As an open source enthusiast, I find that rating to be naive.

First: Stable Diffusion is open source, and brings many concerns with it.
It steals artists work, has bias and can lead to other moral complications.

Second: OpenAI may be mostly proprietary, but their leaders do show more awareness and consideration to their actions, than I know it from most people in the industry - and that counts for the entirety of computing, not just AI.

¹ stable diffusion ethical criticism at DuckDuckGo
² Sam Altman: OpenAI CEO on GPT-4, ChatGPT, and the Future of AI | Lex Fridman Podcast #367 - YouTube

2 Likes