Nextcloud Backup and Restore

Hi,
I’m looking for a way to backup and restore my NextCloud Server.
My linux is basic - I’m learning

OS and dependencies

  • Ubuntu Server 18.04.1 LTS (64-bit)
  • Apache 2.4.39
  • PostgreSQL 10.6
  • PHP-FPM 7.2
  • Linux Kernel: 4.15
  • [Redis Memcache]

I’ve managed to backup and restore the database using

Backup PostgrSQL Database

To backup and restore Nextcloud’s database

backups will be created in /var/lib/postgresql (these could be moved to a new server or restored from here)

Login

su - postgres

Create a backup

pg_dump nextcloud_db > nextcloud_db.bak

To restore

dropdb nextcloud_db

createdb nextcloud_db

psql nextcloud_db < nextcloud_db.bak

I’m now trying to figure out what I need to do to backup in order restore the user and nextcloud data

My data is /var/www/nextcloud and /mnt/nc-data

The end goal is to create a daily cron to back up the required files and save to a folder on a NAS (the NAS is freenas and has 2 disk resiliency)

Any help is appreciated !

Thanks.

1 Like

https://www.c-rieger.de/nextcloud-backup-and-restore/

and

should give you some ideas.

Thanks for that :grinning:
I have tried C Reiger’s Guide before - It’s failed and broke my server every time ?
His build is slightly different than mine and I’ve adapted his guide to suit my build however it doesn’t work ?

It’s very frustrating that this is such a flawed process on Nextcloud?

https://docs.nextcloud.com/server/stable/admin_manual/maintenance/backup.html

1 Like

I’ve written a bash-script which I use to upgrade my Nextcloud-instances. It can also be used to only create backups (option -ob) or restore backups (option -rb). With option -bd you can specify a directory where your backups will be stored. Maybe this is useful for you:

2 Likes

install restic as root

sudo -i
wget https://github.com/restic/restic/releases/download/v0.9.4/restic_0.9.4_linux_amd64.bz2
bunzip2 restic_0.9.4_linux_amd64.bz2
mv restic_0.9.4_linux_amd64 /usr/local/bin/restic
chmod 0755 /usr/local/bin/restic

mount the backup directory to /mnt/nc-backup
set two environment variables

export RESTIC_REPOSITORY="/var/nc-backup"
export RESTIC_PASSWORD="oJSesetLzRgphy9UwmWxi50oE0dkYa16"

initialize restic

restic init

create a backup script /usr/local/bin/backup_nextcloud.sh

#!/bin/bash

export RESTIC_REPOSITORY="/mnt/nc-backup"
export RESTIC_PASSWORD="oJSesetLzRgphy9UwmWxi50oE0dkYa16"

# abort entire script if any command fails
set -e

# Make sure nextcloud is enabled when we are done
trap "sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --off" EXIT

# set nextcloud to maintenance mode
sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --on

# unlock restic
/usr/local/bin/restic unlock

# backup the database
sudo -u postgres pg_dump -c -U postgres nextcloud_db | /usr/local/bin/restic backup --stdin --stdin-filename db_postgres_nextcloud.sql

# backup the data dir
/usr/local/bin/restic backup /mnt/nc-data /var/www/nextcloud

# turn maintenance mode off
sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --off

# delete trap
trap "" EXIT

# clean up backup dir
/usr/local/bin/restic forget --keep-daily 7 --keep-weekly 5 --keep-monthly 12 --keep-yearly 75

/usr/local/bin/restic prune

you may want to add other files and directories to the line

# backup the data dir
/usr/local/bin/restic backup /mnt/nc-data /var/www/nextcloud

e.g. the apache2 config

for more infos about restic look here: https://restic.readthedocs.io/en/latest

restore (single file): you β€œmount” the backups to /mnt/restore with

mkdir /mnt/restore
export RESTIC_REPOSITORY="/var/nc-backup"
export RESTIC_PASSWORD="oJSesetLzRgphy9UwmWxi50oE0dkYa16"
/usr/local/bin/restic mount /mnt/restore

now you can navigate through the backups and look for the file you want to restore and copy it back to /mnt/nc-data/<username>/files

/mnt
β”œβ”€β”€ hosts
β”‚   └── ip-172-31-68-249
β”‚       β”œβ”€β”€ 2019-04-14T07:29:31Z
β”‚       β”‚   └── db_postgres_nextcloud.sql
β”‚       β”œβ”€β”€ 2019-04-14T07:29:32Z
β”‚       β”‚   └── var
β”‚       β”‚       └── nc-data
β”‚       β”œβ”€β”€ 2019-04-14T07:29:33Z
β”‚       β”‚   └── var
β”‚       β”‚       └── www
β”‚       └── latest -> 2019-04-14T07:29:33Z
β”œβ”€β”€ ids
β”‚   β”œβ”€β”€ 0d0ba44a
β”‚   β”‚   └── var
β”‚   β”‚       └── nc-data
β”‚   β”‚           β”œβ”€β”€ admin
β”‚   β”‚           β”œβ”€β”€ appdata_ocjb4w2lzfyj
β”‚   β”‚           β”œβ”€β”€ audit.log
β”‚   β”‚           β”œβ”€β”€ files_external
β”‚   β”‚           β”œβ”€β”€ index.html
β”‚   β”‚           └── nextcloud.log
β”‚   β”œβ”€β”€ 401c5e8e
β”‚   β”‚   └── var
β”‚   β”‚       └── www
β”‚   β”‚           └── nextcloud
β”‚   └── 827fbdde
β”‚       └── db_postgres_nextcloud.sql
β”œβ”€β”€ snapshots
β”‚   β”œβ”€β”€ 2019-04-14T07:29:31Z
β”‚   β”‚   └── db_postgres_nextcloud.sql
β”‚   β”œβ”€β”€ 2019-04-14T07:29:32Z
β”‚   β”‚   └── var
β”‚   β”‚       └── nc-data
β”‚   β”‚           β”œβ”€β”€ admin
β”‚   β”‚           β”œβ”€β”€ appdata_ocjb4w2lzfyj
β”‚   β”‚           β”œβ”€β”€ audit.log
β”‚   β”‚           β”œβ”€β”€ files_external
β”‚   β”‚           β”œβ”€β”€ index.html
β”‚   β”‚           └── nextcloud.log
β”‚   β”œβ”€β”€ 2019-04-14T07:29:33Z
β”‚   β”‚   └── var
β”‚   β”‚       └── www
β”‚   β”‚           └── nextcloud
β”‚   └── latest -> 2019-04-14T07:29:33Z
└── tags

don’t forget to run

sudo -u www-data php /var/www/nextcloud/occ file:scan --all

full restore:
install nextcloud and restic
stop postgres, redis, php, apache2
delete /mnt/nc-data and /var/www/nextcloud
find the snapshot id of the latest backup restic snapshots
restore the directories restic restore <snapshot-id>
or restic restore latest --target /
restore the database psql nextcloud_db < db_postgres_nextcloud.sql

restart everything or reboot

more or less like this. you should try and exercise this on a new maschine a couple of times. :wink:

this has nothing to do with nextcloud. to find a good backup and recovery strategy is a general task in IT.

p.s.: I wouldn’t backup to a mounted file system. because it wouldn’t cover sudo rm -rf / you should look if can use sftp or the restic REST server to connect to your freenas.

further reading: http://plone.4aero.com/Members/lmarzke/talks/restic/

and to setup a test env for restore: β†’ GitHub - ReinerNippes/nextcloud: Ansible playbook to install nextcloud, php, nginx or apache, mariadb or postgres, redis-server, onlyoffice or collabora office

7 Likes

Thanks very much Reiner ! I will try this on a test nextcloud instance and then try and restore to test2 nexcloud instance :+1:

I will let you know how this goes.

Regarding the flawed process, I meant complex ? I work with Veeam backup and restore at work and we backup hundreds of VMs nightly and can restore a windows vm in minutes but even a full vm back of Nextcloud comes with issues, mainly the ncdata folder contains no data during a restore or during a file level restore?

The only time I find data is when I adopt a files and folder backup in Veeam rather than a full vm backup? As I admit, my linux is at a basic level but I’ve a feeling its a folder permissions thing or the fact it’s located in /mnt ?

I’ve also tried the Veeam for linux client locally on my nextcloud instance and when I do a full back up its a success but again no data in ncdata.

So okay not flawed but complex :wink:

maybe the full vm backup β€œcopies” the machines root(?) disc.
the files in a mounted file system are somewhere else. (nfs? another disc?)

You’ve just discovered the issue I think ? It’s a prebuilt vm with 2 disks, one for the OS and the other for storage. So Veeam is possibly backing up the OS and not the Data.

Anyway, I’ve rebuilt a test nextcloud using c-reiger’s guide and its pretty good ! and the ncdata is now in /var/ncdata and its a single 100GB disk. Lets see how this fairs and I’ll test your earlier suggestion as well as Veeam.

edit - Using a veeam backup on the newly built c-reiger vm I can see the ncdata in /var/ncdata so success !! One issue solved

Thanks again.

Thanks to everybody who posted, every little bit has helped me learn a little more!

1 Like

I have a script which is backingUP my server. It is start by udev rules when external USB disk is connected.

  1. at begging is it backup all system, so nextcloud app and config is included …

  2. then it is backup different things including nextcloud DB and data

here is part of my script doing backup of NC data which are in /srv/nextcloud/data

echo " " >> /tmp/zaloha
echo " " >> /tmp/zaloha
echo "----------------------------------------------" >> /tmp/zaloha
echo "-------  rdiff-backup  ->  nextcloud  --------" >> /tmp/zaloha
echo "----------------------------------------------" >> /tmp/zaloha
rdiff-backup /srv/nextcloud/data /path_do_mounted_backup_hdd/SKRIATOK_HOME/srv/nextcloud/data >> /tmp/zaloha 2>&1
rdiff-backup --list-increment-sizes /path_do_mounted_backup_hdd/SKRIATOK_HOME/srv/nextcloud/data >> /tmp/zaloha 2>&1
rdiff-backup --remove-older-than 1M --force /path_do_mounted_backup_hdd/SKRIATOK_HOME/srv/nextcloud/data >> /tmp/zaloha 2>&1

this is doing backup DB, which is dump by another script into /home/psql_backup

echo " " >> /tmp/zaloha
echo " -->          PSQL " >> /tmp/zaloha
echo " " >> /tmp/zaloha
source /usr/local/bin/psql_zalohuj.sh >> /tmp/zaloha 2>&1
duplicity --no-encryption --full-if-older-than 14D /home/psql_backup file:///path_do_mounted_backup_hdd/SKRIATOK_HOME/psql_backup >> /tmp/zaloha 2>&1
duplicity --no-encryption remove-all-but-n-full 2 --force file:///path_do_mounted_backup_hdd/SKRIATOK_HOME/psql_backup >> /tmp/zaloha 2>&1

cat of /usr/local/bin/psql_zalohuj.sh

#!/bin/sh

BASE_DIR='/home/psql_backup'
YMD=$(date "+%Y-%m-%d")
FILE="$BASE_DIR/backup_$YMD.sql"

sudo -iu postgres pg_dumpall -U postgres -f $FILE
  1. script will send email with file /tmp/zaloha at the end so I see if everything is OK and I can unplug hdd.
1 Like

Nice script, thanks!
I am using nginx.
Which commands do I have to modify?

for nguinx you don’t need any modification.
if you are using mysql/mariadb you have to change the dump command.

to something like:

mysqldump --single-transaction -h localhost -u <nc_db_user> -p< nc_db_passwd > nextcloud_db | /usr/local/bin/restic backup --stdin --stdin-filename db_mysql_nextcloud.sql

and maybe add /etc/nginx and some other /etc folder (e.g. mysql)

# backup the data dir 
/usr/local/bin/restic backup /mnt/nc-data /var/www/nextcloud /etc/nginx /etc/mysql
1 Like

Hi;

My backup / restore results in a non-functional docker-nextcloud, when following the recommended procedure. It seems to be related to nextcloud’s inability to install with partial (config only) data as discussed here: fpm-alpine fails with `Command "maintenance:install" is not defined.` after upgrade Β· Issue #489 Β· nextcloud/docker Β· GitHub

My backup/restore script (Variables are in $ENV):

#!/bin/sh

is_installed()
{
        pkg=$(opkg list-installed | grep "^$1 " | cut -d " " -f1 | cat )
        if [[ "$pkg" == "$1" ]]; then return 1; else return 0; fi
}

if [ "$1" == "save" ]; then
        is_installed docker-nextcloud
        if [ $? != 0 ]; then
                nextcloud=$(docker ps | grep nextcloud | cut -d " " -f1)
                if [ ! -z "$nextcloud" ]; then
                        docker exec -u www-data "$nextcloud" php /var/www/html/occ maintenance:mode --on >> $LOGFILE 2>&1
                fi
                find /home/data/docker_data/nextcloud/config >> $CONFIG_FILES
                find /home/data/docker_data/nextcloud/data >> $CONFIG_FILES
                find /home/data/docker_data/nextcloud/themes >> $CONFIG_FILES
                find /home/data/docker_data/nextcloud/custom_apps >> $CONFIG_FILES
                # Save nextcloud database
                su - -c "/usr/bin/pg_dump -U postgres -f /tmp/nextcloud_backup.sql nextcloud" postgres >> $LOGFILE 2>&1
                echo "/tmp/nextcloud_backup.sql" >> $CONFIG_FILES
                if [ ! -z "$nextcloud" ]; then
                        docker exec -u www-data "$nextcloud" php /var/www/html/occ maintenance:mode --off >> $LOGFILE 2>&1
                fi
        fi
        is_installed coturn
        if [ $? != 0 ]; then
                echo "/etc/turnserver.conf" >> $CONFIG_FILES
        fi
elif [ -d $RESTORE_DIR/home/data/docker_data/nextcloud ]; then
        opkg install docker-nextcloud 2>&1 >> $LOGFILE
        # Restore config overwritten by install
        cp $RESTORE_DIR/etc/config/docker/nextcloud /etc/config/docker/nextcloud
        # Restore nextcloud database
        su - -c "/usr/bin/psql -c \"DROP DATABASE nextcloud\"" postgres
        su - -c "/usr/bin/createdb nextcloud" postgres
        su - -c "/usr/bin/psql -U postgres -d nextcloud -f $RESTORE_DIR/tmp/nextcloud_backup.sql" postgres
        # Nuke old data
        rm -rf /home/data/docker_data/nextcloud
        mkdir -p /home/data/docker_data/nextcloud
        cp -rf $RESTORE_DIR/home/data/docker_data/nextcloud/* /home/data/docker_data/nextcloud
        chown -R www-data:root /home/data/docker_data/nextcloud
        if [ -f $RESTORE_DIR/etc/turnserver.conf ]; then
                opkg install coturn
                /etc/init.d/coturn enable
        fi
fi

I am running under OpenWrt and have the following constraints:

  • Cannot backup entire nextcloud mapped directory - too large
  • Want to be able to restore to bare metal, which means install the docker image with only directories config, data, themes, custom_apps available (as in documentation).
  • the above script is called by another script which tars / untars $CONFIG_FILES

Any suggested workarounds?

FYI: part of this project: www.rossco.org

Thanks;
Bill