LAN sync please

Take a look at UnionFS. You can mount your Home-ownCloud with dav2fs on the Cloud-Server (e.g. /mnt/webdav). Then you create a “proxy-folder” (e.g. /mnt/proxy) which has to have the same folder structure (in parts) as /mnt/webdav. You could use rsync or something like that to put files that should be cached into /mnt/proxy. Then you combine those two dirs into one dir (/mnt/union) with UnionFS. Finally you mount /mnt/union into ownCloud using externalStorage.

UnionFS automatically checks if a file is available in /mnt/proxy and uses this file. If it isn’t there, it uses the file in /mnt/webdav (from your home-owncloud). UnionFS will always prefer /mnt/proxy over /mnt/webdav. If your home-OwnCloud ist offline, the files in /mnt/proxy are still available (in the same ownCloud folder, because only /mnt/union is mounted inside owncloud).

You should definitely read the docs of UnionFS before doing this, but I think this solution could fit your needs quite well

2 Likes

Is there any chance we see local network sync in the near future as a regular NC feature?

1 Like

Not sure about Lan Sync, but feeling like I am the only one who has noticed or taken interest in GSOC 2017.

Maybe post some ideas there?

I wouldn’t call it Lan Sync, for me that is two Nextcloud servers and the call should be for server replication.
Its a standard split DNS where the lan subnet is a different IP & Server to the replicated cloud public IP & Server.

It would work transparently depending on where you connect.

I can not understand why anyone with a busy lan would chose to work on a remote repository when most of us are on asynchronous broadband connections when Gig+ synchronous ethernet is easily achievable on the lan.
You would have a local server and that would replicate out.

Personally unless I had multiple remote lans I wouldn’t bother, but if you did have multiple remote lans it could be worthwhile.
In my scenario I new work would be on the lan and have installed the server on the lan and access is through a static connection on the wrong end of a asynchronous broadband connection.
It doesn’t matter really though as remote work is the exception and the client app syncs local files, yeah its slower, but its the exception.

Unless you have multiple remote lan requirements I would be more likely to say you have installed Nextcloud in the wrong place.

If you want an example of lan sync, without even needing a central server, see syncthing. I wonder if the nextcloud people could combine forces with them in some way.

There is nobody at Nextcloud who is doing active client development (judging from github). The Nextcloud client is a rebrand of the ownCloud client, not more, not less. Recommend that you discuss those things with the people who actually work on the code. As far as I understand from github activities virtual file system (now called smart sync by many vendors in the space) is right now most in demand.

you can use syncthing on your server and use the folder in NC as external storage.

That’s right.

This LanSync-thing will be that one missing feature for beeing essential in every software-enviroment.

Is there anyway to use syncthing external storage in Nextcloud while still allowing files to be picked up by the Activity app?

Good question. In the settings of external storage you can set if the storage should be checked for changed files on each access. I’m not sure if this triggers the activity app (you could easily trigger an access via cronjob).

1 Like

Thanks the follow up. It would be great to have a better sync integration between both products, and I would like to break my existing Nextcloud apps as little as possible. Here is a recent thread asking about improving integration and here is a recent github issue as well.

Here is my feature wishlist:

  • Hopefully adopt Syncthing as a fully supported method for Lan Sync between Nextcloud instances
  • Allow Nextcloud apps to communicate with Syncthing apps as p2p, such as between mobile devices. No server available, no problem!
  • Activity app picks up changes
  • Nextcloud is able to track changes and additions of files without lulls.
  • Total disk space in Nextcloud includes syncthing directories.

Integration is probably not that easy. Is it possible that the Nextcloud server can directly talk this syncthing protocol? It would have to be implemented in php, opening ports and such stuff is probably difficult


Could Syncthing implement a webdav api and could be used via federated sharing?

Or you run a special client software of syncthing on the Nextcloud server that can directly talk to the Nextcloud server (push notifications, update database, 
).

Says who? Please give some arguments.

It will reduce traffic on our site and speedup the sync.

Our team uses three shared folders. There are several “heavy” files (graphics) in them.
If we are all on site - and this happens often - the sync is done by our shared internet-connection. So the line is used for syncing and there is no space for doing mails and using internet.

NC is so much useful software, but this missing LAN-Sync-Thing is a handbrake for quick access.

Says who? Please give some arguments.

you got slow dsl in rural area.
you run your own nextcloud server and have 15-230gb data in it.
one pc already synced
try adding other pc to that and see how much you get done during the day.
lan sync reduces internet bandwidth and is faster

1 Like

Haven’t got yet such problem. But maybe this can be a quicksolution:

For me, a simple change which would do most of what I would want LAN sync for is to allow multiple urls to be given to the client for the same nextcloud account, and when trying to sync have the nextcloud client try each of them in turn until one works. I could put the local IP address on my LAN as the top option, and the remote url as the second.

1 Like

thing is the dropbox method just plain works well. it would ( IMO ) behoove the devs to emulate that method.
it is SO simple to use and just plain works well.

Is this possible? I want this for localhost and hosted urls to help with uptime.

That sounds pretty non-trivial to me
 how would one go about doing that?

For those curious on how this might be implemented, here’s a suggested workflow which would probably be most expected by users familiar with how dropbox lan sync works:

  1. Server remains connected to clients, to ensure permission based attacks are not possible
  2. Server and client agree on which files need to be copied in the normal way
  3. If in local sync mode,* instead of copying the file from the server, the server somehow identifies the file needing to be synced, perhaps with a hash. The client broadcasts this hash to all available peers, and then downloads it from them if available. The peer client would also need to check with the server to see if the user requesting had permission to access that file hash. If all that succeeds, transfer from client instead of server.

You’d also want to probably gate this and specify that files under a certain size are never lan synced, because the protocol overhead would be more than the file transfer time savings.

*Re: ‘If in local sync mode’
This could be determined in a number of ways - one such simplistic algo could be:

if [ping time to server] > [5x ping time to a peer client] {
try to sync from client
}

In other words, if the server is substantially further than another client (keeping in mind, a really fast WAN ping would be ~10-20ms, and LAN pings are typically ~1ms).

Similar tests could also be done with bandwidth instead of ping.

1 Like