Hello Community,
this is the first time, that I have a problem with nextcloud, that I can’t address myself.
I want to mirror two instances of nextcloud on two different hardware servers in two different locations. Is this possible and in which way should I do this. Is federated cloud the right way or is there a different approach, that makes this easier to achieve?
Thanking you in anticipation.
You can solve it by using an sql cluster and some kind of mirrored storage. If you like two identical redundant systems federated is not the way to go.
That will prevent vital failures but your forgetting one thing…
The web server itself which I believe can only be done with a load balancer which doesn’t make sense to me. Since it’s a single point failure once again if the balancer itself goes.
Pardon the copy pasta, but since I did just give a very thorough response on mirroring Nextcloud servers last week…
I agree that this is super important! It does not exist for us basic users at this time. 
Your current best bet is to use Rclone to mirror your webdav data directory to another machine via rsync. Borg Backup has been scripted by some to help automate this process.
Request to optionally mirror federated data between instances as a remotely federated share will disappear from your instance if it goes offline. There needs to be an option for these shares to be mirrored in the webui since they already will be downloaded on any clients federating them…
IPFS, or InterPlanetaryFileSystem for Javascript, as external storage would allow Nextcloud users to mirror their data across multiple users, similar to seeding torrent files. See the external storage request here; there is a javascript implementation that could be adapted. It has been mentioned previously on GSOC 2017 and here discussing P2P file systems.
Request to support the Zot protocol, which is developed in PHP for Hubzilla and allows users to literally mirror and migrate their accounts across infinite trusted servers. This process is called Nomadic Identity and would completely reshape our use of Nextcloud in the best way possible! A dev from that project has even discussed the possibility on this forum before!
HubZilla, an interesting selfhosted social network companion to your Nextcloud
Worth noting also that OpenCloudMesh (OCM) was announced by devs of the Nextcloud project in 2016 and is documented here. It does not specifically address mirroring data as discussed in the previous post, but it does relate to finding common federation protocols for all the various, enterprise file sharing projects like Nextcloud, Seafile, Filerun, etc.
Project github is located here, but has not been updated since mid-2017. However, Major federation improvements were added to the server in late 2018 via this Pull Request.
Back in Nextcloud 12 it was announced that Global Scale architecture would be deployed to help scale up for large user case scenarios, but should be deployable on a minimum of two machines. This is designed for enterprise use, but it would address the kinds of questions you have for balancing between multiple servers. Not all of it has been implemented or documented so it is still more of a conceptual white paper.
The Global Site Selector app is available in the app store for setting a Master/Slave relationship between multiple servers.
Lookup Server is being developed as well, but I think it needs additional documentation.
Balancer part of Global Scale does not yet exist, or is not yet being developed as part of Nextcloud’s overall Github.
Taken from this old forum post, these three things accomplish the following:
Global Site Selector
The Global Site Selector (GSS) acts as a central instance that is accessed by the user during the first login, accessing it via the Web, WebDAV or REST. The GSS authenticates the user via the central user management like for example LDAP. It then looks up the node where the user is located in the lookup server and redirects the user to the right hostname. The following calls during the same session are done directly from client to the node.
Lookup Server
The lookup server stores the physical location of a user. It can be queried using a valid user id to fetch the federated sharing id of a user. In some situations, it is important to limit queries to a certain IP space to avoid data leaks. It also keeps track of old federated sharing IDs. The lookup server stores additional data of the users like for example the required QoS metrics like storage/quota settings, speed class, reliability class and so on.
Balancer
The Balancer runs on a dedicated machine, monitoring the various nodes and their storage, CPU, RAM and network utilization. It can mark nodes as online or offline and initiate the migration of user accounts to different nodes based on data in the Lookup Server like business or legal requirements, QoS settings or user location. If for example, a user would move from the US to Europe, the Balancer would initiate a migration from their data to an EU data center to improve the quality of service.
You can learn more details on our webpage about Global Scale.
Hello abz.
May I ask for your primary purpose?
I’m still of the mind that a truly useful mirroring solution for NC will be at an API level, with the mirroring servers understanding the idea of a file being (in process of being uploaded/locked/edited/…) that is not yet available on the local copy of the NC install, or one that has been deleted and therefore should be moved locally to the deleted items folder, etc… There would need to be some checks around space per server etc., but an approach like this allows branch office installs for fastest local performance, as well as hybrid cloud/local hardware solution to keep bandwidth loads separated.
This approach also provides for an ideal DR/BC solution; an office goes down, the NC resource is still available elsewhere.
Thank you all for the interessing ideas.
I want to mirror my nextcloud in a second location, because of a really small internetconnection there. It has only 2 mib/s, so it is really slow, for normal operation. Therefore the second nextcloud-server should be in the local net just to have all data in faster access. The time for mirroring is not so relevant, because this can happen over night.
A second argument is hardware problems. A second instance helps me, to have access even in case of a hardware outage. I have a backup of all cloud data, but the time to reinstall the cloud is also relevant and this time could be eliminated, when there is a second instance of the cloud, that can be used as fallover server.
There is a command line Nextcloud client that can be used to have the local server (slow connection) act as a client also. It’s far from ideal of course but I think it could be setup with a script to launch it during night hours only.
But that is just a work around. A more robust way of mirroring files would be good to have for sure.
A bounty been opened for mirroring federated data across instances via the webui. Please donate to it.
Please consider increasing the bounties if you desire these features!
Nearly a year ago, I asked, if there is a possibility for mirroring one nextcloud server in one location to a second one in another location.
I follow this list, but I haven’d seen any news to this question. Is there somebody out there who has the same problem?
I do not believe any of the issues we’ve discussed have changed at all in the last year. There are a lot of good suggestions above.
The sacrilege model would be to switch over to Filestash, which does not include built-in syncing on purpose and is super light weight. You would then mirror all of your storage between multiple locations via Syncthing.
Filed a request here tp see if the OCM-API has any future plans for development of Nextcloud Federation since it hasn’t been updated since 2017.
hmm maybe the same could be done with Nextcloud if the SQL database was also syncronized? Something like CockroachDB might work for that?
I’ve read about people using DRDB for replicating iSCSI targets. Seems to me you could replicate the block device(s) storing the web root and data folders, and cluster the SQL DB on non-replicated storage… Purely theoretical. I have no idea if that would work in practice.
Hi
I have fully functioned Nextcloud Server on VPS, we also makes backup of data and files directories. Now we are looking for such solution that if prodcution nextcloud server fails or currupted, so we up the second nextcloud server (secondary/mirrored).
please propose a solution
edit: moving this question back into the main discussion thread so others can share thoughts.
Unfortunately, no. Try using the forum search and github search or internet search for many previous discussions on this. The answer is:
No, there is no simple available mirror between two Nextcloud instances.
Use a different service such as Syncthing or rsync to migrate your data directories between two servers.
Anyone else have thoughts or suggestions?
Hey i have 2 nextcloud servers , I want to external mount files from one server to another , and on the second server , to save the file locally in case the first server goes down.
is that possible using external storage?
Split the mirror question back into this discussion thread, to keep that talk here.
Your access question will stay in support. Cheers.
Hey , i have 2 linux machines running nextcloud servers.
the first nextcloud url is : https://nextcloud01.domain/remote.php/dav/uploads/user/
the secound nextcloud url is https://nextcloud02.domain/remote.php/dav/uploads/user/
currently i am copying (mirroring) the two nextcloud servers using rclone.
one application running script every 10 min with
rclone copy nextcloud1: nextcloud2:
and the other application running script every 10 min with
rclone copy nextcloud2: nextcloud1:
in that way, i was able to “sync” the data between the application and save it to DB and data directory.
unfortunately federating the two application does not gives me a valid option in case one of the servers goes down.
only using webdav(rclone) i was able to make it work as expected.
is there is a way in nextcloud to share(copy and paste it in the other application) using the webdav like rclone does ?
No, only as you’ve already found. Connecting two Nextclouds by webdav is also known as federation.
Moving this conversation to one of the existing threads on mirroring two servers. 