Nextcloud 11 sets new standard for security and scalability

[system]

Originally published at: Nextcloud 11 sets new standard for security and scalability - Nextcloud

We’re proud to release a huge milestone: Nextcloud 11, delivering a wide range of security and scalability improvements with a number of important features on top. This release provides you the most secure solution for keeping your data under control we’ve delivered yet.

A video showing many of the small refinements in this release

Nextcloud 11 main improvements

The Nextcloud project has become the largest and most active in its space and is backed by a healthy, growing and profitable company. Together, the project merged almost 1200 pull requests in core alone, bringing a wide range of enhancements to users.

This release introduces significant security improvements, attested by NCC Group, a global expert in cyber security and risk mitigation to “complement the existing security architecture” and “enhance the general standing of the security working environment.”

New security capabilities include:

• Support for cutting edge browser security features CSP 3.0 and Same-site Cookies
• Support for Kerberos authentication and Two-factor Authentication providers based on Universal 2nd Factor and Time-based One-Time Password
• Expanded brute force protection to all API access points
• More secure Federation through use of SSL/TLS
• Our new app store automatically checks apps and enforces signatures

Scalability is a prime concern among our large enterprise customers. This release decreases database load by up to 80% and improves response time by up to 60% for common server operations. Combined with multi-bucket Object Store support, improved handling of previews and Collabora Online speed improvements Nextcloud 11 enables scaling to greater numbers of users and files, decreases the server load and improves the user experience.

Nextcloud 11 introduces Apache Solr powered Full Text Search, enabling users to find words or phrases in text, pdf and common office documents on internal, external, shared and encrypted storage. The next generation Federation technology introduces a central lookup server, enabling Nextcloud users to find each other irrespective of the server their account resides on. The experimental Spreed app integrates secure, peer to peer audio and video chat in Nextcloud.

Read on for more details on these features.

Security

For Nextcloud 11, security has been a prime concern. Privacy and control over data mean little if security is not in order. Nextcloud 11 introduces a series of new security capabilities and hardening, delivering security following the latest industry standards and enabling administrators to opt in to further measures.

• This release includes Two factor authentication support for Universal 2nd Factor (Yubikey and compatible) and Time-based One-Time Password (Google Authenticator and compatible)
• Support for Kerberos authentication
• Password confirmation for sensitive actions
• HTTPS by default for federation
• Password reset logic auto-invalidates after changing mail address
• Improved brute force protection on more endpoints
• CSP 3.0 protection and Same-site Cookies Support improvements
• App specific tokens can be forbidden file system access
• Our Collabora Online integration app has several security hardenings
• Our new app store automatically checks apps and enforces signatures

NCC Group, a global expert in cyber security and risk mitigation, reviewed the addition of these features and noted they "enrich the security layers with minimum impact on the user" and are developed using industry standard security processes (assessed against ISO27001 clause 14 controls). Read more and download their independent security assurance on our website and learn more details about the new features in our blog on security in Nextcloud 11.

[caption id=“attachment_779” align=“alignright” width=“300”] App store in action[/caption]

New app store

Given some security related incidents with the old app store like this and this, security of Nextcloud systems benefits from a move to our new app store on apps.nextcloud.com. The new app store integrates an automatic check on uploaded apps, ensuring a significant proportion of scam and spam attempts is blocked before even making it to the store. Vetted developers have to sign their app with a private key, signed in turn by the Nextcloud Code Signing Authority which ensures even a successful compromise of an app store account can not result in malicious apps being uploaded to user systems.

Find more info in our earlier blog about appstore improvements and a recent blog about app signing.

Performance and scalability improvements

Scalability and efficient operation are key capabilities sought after by system administrators. Nextcloud 11 introduces changes which result in decreased database load and faster execution of common operations known to cause significant load on large instances.

Typical Nextcloud instances should benefit from up to 80% less database queries and 60% faster execution of checks for changes (“propfind”) as done by the Nextcloud clients. This operation is often accounting for over 95% of the load on big installations. With database usage often a major limiting factor in scaling up this should result in significant decreases in load and a subsequent increase in capacity for Nextcloud servers. Syncing small files also benefits with up to 60% faster transfer of small files.

To make Nextcloud systems with object storage more scalable, this release introduces a multi bucket object store. Not all s3/swift implementations like having multiple million objects in a bucket, so Nextcloud 11 will create a bucket per user, helping with storage performance on large installations.

Thanks to improvements in the integration with Collabora Online, startup time has been improved in Nextcloud 11 and users will now be able to enjoy their online office suite faster than ever!

Last but not least, large folders will load considerably faster while thumbnails are handled far more efficiently as well as now shared between users, saving significant resources on instances where many users share the same set of files.

You can read more details and an analysis of their impact in our blog on the performance improvements in Nextcloud 11.

Full Text Search

Nextcloud 11 introduces full text search with the optional Nextant search application. Using Apache Solr, Nextant indexes the content of:

• User files
• Shared files
• Federated cloud shares
• External storage
• Server-side encrypted storage
• Bookmarks

It recognizes plain text, rtf, pdf, jpeg & tiff, html, LibreOffice and Microsoft Office file formats. System administrators can opt to use a standalone Solr or use Solr as a Service for a more scalable solution. Learn more about full text search in Nextcloud 11 in our blog.

Next level of Federation

Federated Cloud Sharing enables users to share files between Nextcloud servers. Nextcloud 11 introduces an optional, global 'address book' to search for other Nextcloud users to share data with. In the personal settings, users can op to share certain data like country of residence or social media accounts which can be used by friends to identify them for file sharing.

Nextcloud will operate the lookup server and we are looking into distributing this task, creating a fully decentralized network.

You can learn more in Bjoern’s blog on Federation in Nextcloud 11.

Technology Preview of Spreed integration

Nextcloud 11 brings Spreed audio and video chat integrated in the user interface. The Spreed allows you to invite any of your friends and colleagues on your Nextcloud server for a call and also allows sending a public link to people who don't have an account yet. Users will get a notification if they are receiving a call. Up to six users can join a conversation. Spreed for Nextcloud 11 is still under heavy development and considered a technology preview. While the app is largely stable there are a lot of features we'd still like to add and cross-browser and cross-device compatibility improvements are a work in progress.

Read our blogs for more information about Spreed video calls in Nextcloud 11.

User interface and other improvements

Delivering a great user experience has always been a central goal for the Nextcloud team and this release makes significant strides in this area. File handling was improved with move actions for files and folders as well as a easy way to share the current folder and share files and folders by mail. Thanks to a database update, we can handle emoticons in file names without breaking their synchronization on servers using MySQL.

The updater has seen a rewrite with a new approach aiming for reliability and stand alone functionality so a break anywhere in Nextcloud won’t impact the updater.

The Contacts and Calendar apps also have seen a number of improvements, most notably by introducing public sharing links in the Calendar and improved contacts import in the Contacts app.

How to get it

The massive number of improvements are a true testament to the progress the Nextcloud project is making. Dozens of users have helped us test this release (some earning a t-shirt in the process - and hundreds contributed with translations, code and in other ways.

The packages for Nextcloud 11 are building and will be available in the next hour or so.

If you would like to update, you can do so by downloading the zip file on our download page and following the manual upgrade instructions or wait for 2-3 weeks until we make the new release available in the Nextcloud Updater app. If you prefer to update now already using the easy to use Updater app, you can set the release channel to beta this will give you Nextcloud 11 final. Be sure to set it back to stable after the upgrade is done!

If you encounter any issues, we recommend to look for help on our forums and, if it is a bug, report it.

[Akhenaton]

Hi,

@system

If you would like to update, you can do so by downloading the zip file on our download page and following the manual upgrade instructions or wait for 2-3 weeks until we make the new release available in the Nextcloud Updater app. If you prefer to update now already using the easy to use Updater app, you can set the release channel to beta this will give you Nextcloud 11 final. Be sure to set it back to stable after the upgrade is done!

I am sorry, but this statement doesn’t make things clear AMHO…

Just two question : Why wait for 2-3 weeks delay for getting in a “normal” updater way"(stable version) the new 11.0 relese ? and why must we opt for a beta channel to get now this 11.0 version ? Will we get actually through the beta channel a “stable” version or a “beta” version ?

Thank you !

[BernhardPosselt]

Sounds like a controlled roll out. Get the enthusiasts on board first to check if any unfound critical issues are found, then push it to standard users.

[jospoortvliet]

@Akhenaton

Just two question : Why wait for 2-3 weeks delay for getting in a “normal” updater way"(stable version) the new 11.0 relese ? and why must we opt for a beta channel to get now this 11.0 version ? Will we get actually through the beta channel a “stable” version or a “beta” version ?

Hi,

Sorry for the confusion! We usually only offer a new major release some weeks after it is out on the updater. So if you want to use that to upgrade you have to wait until we ‘turn it on’ and then you get notified and can update.

You can of course manually update already.

But some people prefer to use the updater app (like myself) because it is super easy and fast. So - we put Nextcloud 11 on the ‘beta’ channel. If you set your version to beta and reload the page, you get an upgrade to Nextcloud 11 (once it is on the mirrors - that will still take some minutes). After upgrading you set your updater back to ‘stable’ and that’s what you get from then on

[grzegor]

Ok, so if it’s a security update does that mean that filenames on external storages are finally getting encrypted as well as contents? Because that was a major (and by that I mean MAJOR) security concern when 3rd party can guess what kind of data am I storing (anti_government_file.docx or my_gay_sex_tape.mov like countless of others can be used against someone even without exposing their contents). I cannot see any info about that in your changelog note.

[BernhardPosselt]

@grzegor

Ok, so if it’s a security update does that mean that filenames on external storages are finally getting encrypted as well as contents? Because that was a major (and by that I mean MAJOR) security concern when 3rd party can guess what kind of data am I storing (anti_government_file.docx or my_gay_sex_tape.mov like countless of others can be used against someone even without exposing their contents). I cannot see any info about that in your changelog note.

Where did you read that? I don’t see this anywhere in the release notices

[pixolin]

Just as a feedback:
I currently run Nextcloud 10.0.2 on Nextcloud Box with Ubuntu 16.04.1 LTS.
If I switch to Beta channel and run the updater, I receive dozens of file permission error messages for /snap/nextcloud/389/htdocs/updater and it’s files and subfolders. I also can’t simply chmod /snap/nextcloud/current as the folder is in a Read only file system.

I guess I’ll have to wait, until a new snap is provided? iirc, snaps weren’t mentioned in above update message.

Good things take time.

[grzegor]

Me neither! That’s why I’m asking if they did it, because that seemed like a no brainer to finally fix in 2016, but I don’t see any info about that!

[BernhardPosselt]

It’s certainly not a no-brainer. If it’s fixed it will be announced.

[grzegor]

For a security related update I most certainly think it really is!

[BernhardPosselt]

@grzegor Lots of people have different understandings and priosations for various topics. Also you cant implement everything that is nice to have in a timeframe of a few months. If this topic is super important for you either get a support contract or create a pull request.

[grzegor]

No, it’s not super important for me per se (I’ve been mounting external storage with encfs on system level for years so I’m not even using your encryption modules) - I just think it should be important for everyone for sake of their privacy. It’s a headscratcher that you would post a special major security related update (with a whole version bump for that!) yet didn’t address something that was discussed numerous times on github since 2014 if I recall correctly (thus why I considered this a little obvious security enhancement and not just nice to have as you said). I don’t think I can think of any modern software with file encryption capabilities that leaves the filenames unchanged. Actually, in today’s world even simply encrypting filenames doesn’t mean you’re on the safe side of getting criticized if you’re not doing them according to current standards (with init vectors chaining for example, so same files would have different encrypted names). And the best thing is that with your current security model in which you focus on encrypting external storage providers leaving local security on system level that could be actually achieved by a way easier workaround, by just generating UUIDs for each file and mapping them in a database. And this isn’t even my idea - I read it on oC’s github months ago! Besides that I gladly welcome the rest of enchancements in 11! Very nice to see that you’re giving a little more love to your outstanding product than it was getting while on oC branch. Hope now I managed to clarify my point a little better. Cheers!

[BernhardPosselt]

It’s a headscratcher that you would post a special major security related update yet didn’t address something that was discussed numerous times on github since 2014 if I recall correctly

It works like this: Developer picks a few topics he’s interested in or paying customers want, finishes and stablizes it, then whoever is responsible for PR goes over the features that were implemented. If a lot of security features went in, then he releases a post about that a focus of this release was security.

Again: Encrypting file names on external storages is not trivial nor important for many users (I for instance couldnt care less). It is important for a few users that actually use encrypted external storage (most dont because they use cheap webhosting) which want to upload their super secret or very damaging stuff (advice: dont upload anything like that at all or encrypt it before uploading). Adding the feature is also not as trivial as just adding uuids: the hardest part is migrating and encrypting existing data.

;D

[Akhenaton]

@BernhardPosselt
I like very much your “controlled roll out” expression !

@jospoortvliet
Thank you for your kind response. So i made the jump and did the upgrade with the updater and the beta channel (i’m now back on the stable one). All went well and smoothly, not a single glitch !
I didn’t have time to test everything thoroughly but as far as i went that 11.0 version seems a very good one and is slightly more responsive than the 10.0.2.
I will, as usual, report if i find something wrong.
Thank you to all for your great work !
Regards,

[guddl]

… file browsing is incredibly fast

Thanks for the new release!

[masteroman]

Great release! Upgrade to a new version went smoothly. Well, except it had some issues with .well-known directory which I use for verifying LetsEncrypt certificate. Is there a way to place that on some kind of “Update ignore list”?

Once I’ve moved that directory out of webroot everything went smoothly…

[JasonBayton]

@system

Typical Nextcloud instances should benefit from up to 80% less database queries and 60% faster execution of checks for changes (“propfind”) as done by the Nextcloud clients.

Sweet Godzilla.

@system

Nextcloud users to share data with. In the personal settings, users can op to share certain data like country of residence or social media accounts which can be used by friends to identify them for file sharing.

@system

Nextcloud will operate the lookup server and we are looking into distributing this task, creating a fully decentralized network.

If I can help, count me in.

@system

File handling was improved with move actions for files and folders

You guys are phenomenal.

[jospoortvliet]

@pixolin

I guess I’ll have to wait, until a new snap is provided? iirc, snaps weren’t mentioned in above update message.

Hi Pixolin,

The snaps don’t update via the updater app, they have their own mechanism where you don’t have to do anything for the snap to update. You can expect Nextcloud 11 in the snap some time early next year. It takes extra time to built and test that and we first want to release the image for the Raspberry Pi 3.

@grzegor in reality, very few people use the encryption app and none of our customers do. So, as @BernhardPosselt pointed out - this is in the end a community driven project. As in, if somebody does the work, it happens. I do believe there are discussions about doing something around encryption for Nextcloud 12, though, so perhaps this will get tackled. But no promises.

By the way, you can use the Encryption app with an object storage, in that case no file names are leaked. There is a work-around, thus. With regards to the UUID, that is what the object storage does and I suppose it would be possible to adapt it to the encryption app somehow. But it would require a major re-architecture of how external storage works, from what I can tell. In any case, perhaps for 12 or 13. We have plenty releases ahead of us

Thanks for caring!

[jospoortvliet]

@JasonBayton

You guys are phenomenal.

I’m happy to take that compliment on behalf of the people who ACTUALLY do the work

/me hides now for the bananas thrown at him

[JasonBayton]

Here’s another for you: I just upped to 11 manually and damn it is fast.