Using syncthing to backup data folder

ℹ️ Support
,
1

I have no support/technical question and have seen the support category. (Be aware that direct support questions will be deleted.)

on

Which general topic do you have

Hello all,

Not sure if this fits a support question. I would like to use syncthing to backup the data folder to another server (also running syncthing). I have setup syncthing on the mathing running nextcloud and run it with the www-data folder. Is there something I should be aware of doing it this way?

I started the process but the machine froze and when accessing the nextcloud interface it complained it couldn’t find or access the .ncdata file.

Is anyone using syncthing to backup the data folder?

2

Hey @luciandf welcome back :waving_hand:

that would be expected, since you’re trying to “clone” a running instance including data on os level which is bound to fail, either on the database side or as you’ve experienced with the .ncdata file which is instance relevant. prefer copying/cloning/syncing the data only.

yes, consider this;

you’re using syncthing to copy raw data on os level… the nextcloud database is unaware of anything taking place on os level. thus your database will not be aware of changes and will need to reread those files with occ files:scan --all.

it would be more flexible if you were to use something like rclone for syncing/copying files/directories between instances using webdav, as that would be database aware from one instance to another incrementally.

IMPORTANT

Synchronisation is NOT Backup!!

in addition rclone is easily scripted and can be added to cronjobs at will. Here’s an example of how to synchronise notes from one instance to another via script, the syntax is similar to rsync.

Example sync script:

  #!/bin/bash

## Sync notes NEXTCLOUD-00 <--> NEXTCLOUD-01

    /usr/bin/rclone sync NEXTCLOUD-00:/Notes NEXTCLOUD-01:/Notes -P ; 

exit

Example copy script:

  #!/bin/bash

## Copy notes NEXTCLOUD-00 --> NEXTCLOUD-01

    /usr/bin/rclone copy NEXTCLOUD-00:/Notes NEXTCLOUD-01:/Notes -P ; 

exit
2 Likes
3

Thank you for the detailed response! When you say that the database is not aware of changes what does that mean in the context of syncthing doing its thing (i.e. sending the files one way to a remote location?). I most definitely do not understand how syncthing is operating but I was under the impression that it doesn’t change the files is sending one way only.

If that’s the case then what makes rclone any different? or rsync? Or any type of sync must be done via webdav to not break nextcloud?

I am so confused.

4

you’re going to have to do some reading about syncing/copying/cloning Nextcloud data then…

5

OK so from what I read, it appears that syncing is only possible between a nextcloud client and the nextcloud server or (not sure I go this) with federation between nextcloud servers.

So for a backup I need to copy the data to the backup server which inevitably require maintenance mode while the data is sent over.

If this is correct then that means that with something like borg backup I will have to wait until the intial copy is made then setup something with cron and send the incremental changes sometime when there is no activity on the nextcloud server.

Is this correct?

6

Hey @luciandf heads up for doing your homework :+1:

couldn’t have said it better myself :100:

sounds plausible

bar having any experience with syncthing you could try to exclude the active parts of the server like .ncdata and the database, thus only copying raw user-data… not sure if that helps though?

7

I have one more question: which are the active dirs in the data folder?

drwxr-sr-x  4 www-data www-data         4 Feb 24  2020 admin
drwxr-xr-x 18 www-data www-data        18 Jan  5  2024 appdata_ock7m1cgtkba
-rw-r-----  1 www-data www-data    183604 Sep 15 09:49 audit.log
-rw-r-----  1 www-data www-data 104884259 Sep 14 07:36 audit.log.1
drwxr-xr-x  2 www-data www-data         3 Jul 16  2024 files_external
-rw-r--r--  1 www-data www-data       542 Sep 12 17:25 .htaccess
-rw-r--r--  1 www-data www-data         0 Feb 24  2020 index.html
drwxrwxrwx  7 www-data www-data         7 Oct  9  2020 user1
drwxrwxrwx  7 www-data www-data         7 Mar 26  2020 user2
drwxr-xr-x  6 www-data www-data         6 Apr  8  2024 user3
-rw-rw-r--  1 www-data www-data        52 Sep 12 17:25 .ncdata
drwxr-xr-x  3 www-data www-data         3 Jan  4  2024 user4
drwxrwx---  3 www-data www-data         3 Sep 11 17:08 nextcloud
drwxrwxrwx  7 www-data www-data         7 Oct 10  2020 user5
drwxrwxrwx  4 nobody   nogroup          7 Oct 20  2023 security
drwxr-xr-x  2 www-data www-data         3 Sep 13 03:03 .stfolder
-rw-------  1 www-data www-data       315 Sep 13 03:03 .stignore
-rw-rw-r--  1 www-data www-data    342804 Sep 12 17:26 updater.log
drwxrwxr-x  4 www-data www-data         5 Sep 12 17:26 updater-ock7m1cgtkba
drwxrwxrwx  7 www-data www-data         7 May 20  2022 user6
drwxrwxrwx  7 www-data www-data         7 May  8  2020 user7

The updated folder I understand is not that important (is that wrong?) but is the appdata folder active? Can I include it in the syncthing?

Regarding the database I will create a scrypt to dump the database at midnight and sync it over.

Thank you so much for your patience!

8

not sure about that… try it out :pinching_hand: