I have no support/technical question and have seen the support category. (Be aware that direct support questions will be deleted.)
on
Which general topic do you have
Hi All,
I’m thinking about running a NC instance on a rather nice piece of hardware we purchased for another self hosted file solution that isn’t working out. I have a question about federated sharing.
We have three office locations, if we put an NC instance at each location (A - 50 Users) (B - 50 users) and (C - 95 users) can we set up a federated sharing “zone” / “folder” etc… where the files in that folder will be syncronized to the local instance at each location? That way if a user at location C uploads a large video file and a user at location A wants to download it, assuming the sync is up to date, the user at location A will get a fast download speed from their local server?
Really appreciate the help with this question. Just looking for a moderately detailed answer about achieving this, not a detailed guide. We will test at one office first but this would be a massive feature for us.
I agree. File locking / merging is a huge challenge when trying to design something like this. But the files that we need to synchronize don’t change frequently and as long as the sync happens within a day or so, there shouldn’t be an issue. Something we certainly would need to be cognizant of.
Yes, I use rclone to sync between 2 locations - but not as a hot standby, because I use the rclone only to sync from one location to the other and not back. There is a 2-way-sync option in rclone, but experimental (at least last time I checked).
Prior to use rclone I tested the federated sharing which for me has a major problem: If you have several federated servers linked together via federated shares, as soon as one location has problems and cannot be reached, the other Nextcloud instance automatically also gets problems. So then immediately have timeout waitings in the Nextcloud ui. So the more federated servers linked together the more rises the risk of none of them works any more. This was a no-go for me. In contrary I need a redundant decentral solution. Especially because in my case one location is unattended (hosted on different physical location, but no capable administrator available there), so if hardware or essential software fail, it cannot be repaired in short-term.
Limits of the rclone: Only replicates files, so apps that deal with data in separate tables will not have there data replicated. - If you want that, you would need additional tools for that. - Thas also affects user, groups and shares - shares are not replicated either. - For my use case I mostly have the shares implemented on top level for groups and I have a limited number of users and groups. So I manually create the users and groups on both sides (and need to create an app key for the rclone for each user). And I set the shares manually on both sides. Then I have a nightly (or could be done more often in theory) cron-job that replicate to the other location.
Single file or folder ad-hoc sharing I use sometimes, but not critical in case of main site down.
I tested the 2-way-sync once and apart from the fact that the duration of one sync run is then longer, there were problems in case of conflicts. - I checked now and I think in the meantime they added more options here. See Bisync
The rclone setup I explained here, is running for approximately a year now. Since then I did not get back to test the 2-way-sync any more.
A complete 2-way-sync that even allows other location to be used as hot standby would be awesome. My wild guess is, that decentral replicating repositories where local users could use their local repository for faster access (or just because other location is a partner company that wants to have their data even locally) would have a bright future.
I was thinking like “US Files” “EU Files” and configuring where rclone (or whatever) could try to sync between the two, where when processing the EU files folder, if there is any conflict the file from EU would be the source of truth.
But this is certainly a challenging idea. I’ve reached out for some commercial solution’s. Nothing has been quite right or has been really really really really prohibitively expensive. I don’t mind if there is some price to pay, but… still needs to be reasonable.
In fact, rclone can be configured to sync specific folders only, not necessarily the whole account. And you could run different rclone jobs against different folders/shares. And it should be possible to run these sync jobs with different conflict resolve strategies. - See the Bisync-Link I posted:
--conflict-resolve string Automatically resolve conflicts by preferring the version that is: none, path1, path2, newer, older, larger, smaller (default: none) (default "none")