TU Berlin halves database load by migrating 22K users to Nextcloud

Originally published at: TU Berlin halves database load by migrating 22K users to Nextcloud - Nextcloud



Last week, we announced that 9 German education and research organizations moved from ownCloud to Nextcloud together with the TU Berlin. Today we’ve published a case study on our website which details the migration of the TU Berlin and the results achieved, including 50% lower base and 38% lower peak database loads. The end result was faster service to users at lower cost.

TU Berlin’s ownCloud-Nextcloud infrastructure

The Tu Berlin was one of the first universities to provide its own file sync and share solution, already evaluating ownCloud back in 2011. They provided cloud storage within the DFN cloud research program, servicing 16 member research and higher education institutions today. Their service has over 22.000 active users at the TU Berlin, with more at the other organizations and storage for the TU Berlin itself is well over 70 TB, with students allowed a maximum of 20 GB data and staff members up to 100 GB.

The TU Berlin uses load balancers spreading user requests between 4 application servers which run a NGINX/MySQL/PHP 5.6 stack on CentOS 7.3. Data can be found on a GPFS-based storage and a Galera MySQL cluster while LDAP and Kerberos handle authentication and all of it runs in LXD containers managed with OpenStack.

Migration

After Nextcloud won a tender for providing a new file sync and share solution, TU Berlin decided to migrate to Nextcloud 11 which promised significant scalability, security and feature improvements. The team is also interested in exploring Nextcloud Global Scale in the future, for scaling up their DFN cloud service and decreasing the total cost of ownership.

The migration was planned during the course of three weeks and executed in one week, fitting the update window set between the result of the tender and end of the old contract. A test environment built from a copy of production was used to prepare. As part of the migration to Nextcloud 11, the TU Berlin decided to move to NGINX from Apache, which was made possible thanks to the native SAML support in Nextcloud.

The migration process itself was ready in time and worked exactly as planned. The TU Berlin did not need any stand-by support from Nextcloud.

Overall, the migration was like a typical upgrade.

Results

Before Nextcloud 11 was deployed, the TU Berlin faced scaling issues with heavy peak load on the database cluster during working hours. The migration resulted in peak load decreasing by over 38% while off-peak times even show a 49% lower stress on the servers. Thanks to this improvement users experience a snappier experience and faster syncing while the TU Berlin can now grow further without significant investment.

Server load before and after migration from ownCloud to Nextcloud

You can find more details in the case study on our website and Dr.-Ing. Thomas Hildmann from the TU Berlin will talk about the migration and its results at the Focus Friday on the Nextcloud Conference later in August.

Hey,
thats great news. I have three questions:

  1. Has the migration to Nginx part at the performance gains?
  2. Why did they move to Ngingx?
  3. Why do they not use PHP7?

kind regards

1 Like

Showing my ignorance here, but why was this a blocker before? Also really disappointed to be missing this talk on the Friday

1 Like

They moved to NGINX to lower memory usage on the application servers, to avoid having to deploy more. This has no effect on the database servers, where the performance gains were the biggest (this was also the bottleneck to scaling to more users).

As written in the case study, the application servers load actually went up by a few percent because the database was so much more responsive.

The TU uses SAML and ownCloud has no native SAML support. That means they had to use Apache to use its SAML support module, NGINX doesn’t have one.

The native SAML app in Nextcloud also improves session management (no forced relogin after X seconds like with the Apache module) and supports app specific passwords and group memberships, neither of which work with the Apache module.

1 Like

It would be interesting to see by how much efficiency would be (further) improved by moving from mysql to postgres.