Local sync with NextCloud

I installed NextCloud 11 on my server. In that server, I have a 300GB folder which I would like to sync. Since it is already on the server, can we skip online sync and do an offline sync, improving speed?

I’m not talking about connecting to localhost and syncing data like that, because it still relies on a connection, I would like to manually tell nextcloud “take this folder and import it into user X directory”. Is it possible?

Can you provide a little more context?

  • What and where is user x directory?
  • What protocol are you looking to use in place of http?

Right now I’m using the DAV way to connect with cURL under debian. I’m connecting to localhost, providing cURL the full path of the file I want to upload, along with the full path of where it should be located in the server.

curl --insecure -T /path/to/local/file.zip --user user1:password https://localhost/remote.php/dav/files/user1/path/to/remote/file.zip

I was looking for a way which didn’t involve cURL. I don’t know, maybe a binary or an executable inside the nextcloud directory which allows to batch import an entire folder of your choice into an existing user’s account

If I’m following correctly, the directory you wish to import is on the server, and you’d like to move/copy it directly into a user account?

If that’s the case as a one-off;

  • rsync, cp or mv the data directory into /path/to/nextcloud/data/username/files/ (symlinks aren’t supported)
  • from the /path/to/nextcloud directory, run sudo -u www-data php occ files:scan --user (or --all if you do this multiple times to different users)

Where user in both instances is the user in question. This will import the folder to the root of the NC data dir for the user (the folder will sit alongside Documents, etc) and force Nextcloud to scan and import it into the database. This should not be done frequently. Return to webdav or the webUI going forward as changing the filesystem regularly unbeknownst to Nextcloud can lead to database mismatches.

Alternatively, you can add the directory on the server in place as external storage within the NC UI. It doesn’t matter how often you change the underlying filesystem in that case as external storage is regularly scanned for changes.

Yes, that’s what I meant! Is there a way to tell nextcloud a custom last-modified date instead of “today”? I’d like to upload all files as if they were uploaded 10 days ago, if this is possible

Not that I’m aware of. If you rsync the data you can keep last-modified dates intact on the filesystem, and Nextcloud should pick that up (I think, certainly works for media).

It worked! It just take ages to rescan, but I’ve searched a bit and it seems to be due to various reasons (cache and other things).

I did a cp -Rp and nextcloud retained file dates, but this didn’t work for directories: while they have the correct timestamp on the filesystem (checked with ls -la), nextcloud imported them with “today” as the creation date.

Thank you!

1 Like

I’ll test with rsync later. See if that happens with the archive and attributes flags.

Happy it worked :slight_smile:

Not sure if it was a nextcloud or cp issue, though. I checked with both ls -la, ls -lc, ls -lu and none of the timestamps displayed was actually picked up by nextcloud, it seems it picked the time it added the folder. Anyway, that’s not an issue, as I needed files to retain dates :slight_smile:

1 Like

with rsync it was a separate sync command to make sure directories have the same date after they are copied.

I’m using NextcloudPi on a Raspberry Pi (Nextcloud 13).

Here’s a few more tips which tidy up the files you’ve copied in:

After doing a giant copy of files into my Nextcloud user’s data folder (as above), I corrected the file and directory ownerships with the following (running this from the Nextcloud user’s data folder which I had just copied into):

chown -R www-data.www-data *

…then I cleaned up the permissions on the newly-copied files and folders like this:

find . -type f -print0 | xargs -0 -iHeRe chmod 644 HeRe
find . -type d -print0 | xargs -0 -iHeRe chmod 755 HeRe

This gives all files “-rw-r–r–” permissions, and all folders “drwxr-xr-x” permissions.

Finally I deleted any and all junk “desktop.ini” files (which Windows had annoyingly and automatically added earlier):

find . -name "desktop.ini" -print0 | xargs -0 -iHeRe rm HeRe

The “scanning” command I had to do in NextcloudPi looked like this instead (note absolute pathname to “occ”, also the absense of two dashes before the Nextcloud username, which was “ncp” in my case):

sudo -u www-data php /var/www/nextcloud/occ files:scan ncp