How to backup nextcloud data and apps

I have read many variation of backing up nextcloud but wonder if there is a standard built in method of any kind ?

I have things working the way I like with Talk app, and other apps like bookmarks, passwords etc.

I am confused about what is the easiest backup method vs other secure methods etc.

I’ve read how raid isn’t backup and this makes sense but what do you do ?

Do you image everything, or backup data only ?

Please advise of this from the easiest to the more secure please.
I would be content with an external storage or extra storage for complete backup and restore if this is possible.
Thanks

1 Like

I personally run my NC on a Pi4b with a 1TB SSD, and the only user so i can quite easily take it offline from time to time, usually before an update or change I will shut down, plug the drive into my PC and using Macrium reflect take an Image, or add to an image with an incremental backup.

during normal daily use (while the nc lis live), i Use rpi-clone GitHub - billw2/rpi-clone: A shell script to clone a booted disk. which clones the 1TB SSD to a spare 1TB HDD
it works really well, and is fast… once you have the first clone done, it then only needs to copy changes… and seems to work (if i shutdow & unplug the SSD, and boot from the HDD, everything works perfect!)
**but I still prefer to have an occasional full image to restore from if needed.

Raid is not a backup at all, it is redundancy for your drives (e.g if a drive fails)

I think “the standard” way you mention, would be to use the inbuilt Backup app? (ive not tried that)

@Captain86
You can read Backup and Restore. But there are a lot of ways for a backup. You can use e.g. the app Backup for some backup senarios for only the user and not the system. Also you can combine backup strategies. And most importantly: also perform a restore once in a while, for example on a test system.

Perhaps the files are more important for you than the shares and other data. Then you can backup the data somewhere additionally. Keep in mind that a Nextcloud sync is not a backup.

1 Like

Thanks.

I only have a few users but I guess I want to backup the entire nextcloud, data and database etc.

Your second links shows a “backup app” and looks like it is a full backup. I’m not exactly sure how the restore process goes for this and if others are using it.

I’ll have to read some more on these backup ideas. I’m not really sure what might be best.

What about nextcloud on VM ? Same ? Or would you just only image the VM or both ?

if it was a vm such as hyper-v you could “export” and archive these
or “checkpoint” to create quick rollbacks

1 Like

Any backup software you want to use is fine. If you don’t mind something proprietary, the free Veeam agent for Linux is pretty good.

You are correct that RAID is not a backup. You need to make sure that you store the backup on different storage, otherwise the backup is useless. Use a USB hard drive or something.

Reading the link provided by devnull.
https://docs.nextcloud.com/server/latest/admin_manual/maintenance/backup.html

Please confirm the subject of rsync method ?

If you rsync entire nextcloud, is backup of database still needed ? Or is this just something you do if you only rsync config and folders then do backup of database separately ?

Please clarify this some more
Thanks

there are many ways to do backup and there not all the same.

for example if you make a disk image also known as a disk clone you create a exact copy of your hdd which you can replace incase tou current one fails. This exact copy would be sufficient in case of harddisk failure but requires 2 identical disks.

then the backup of nextcloud is different this only makes a backup of what is needed for nextcloud this does NOT include your system as apache settings or the database used. there for you need to back them up separately aswell as the user data.

so creating a backup of the nextcloud + nextcloud user data + database is needed to be able to restore only nextcloud. this is not enought for a working nextcloud as there is no backup made from system packages like the needed php, apache … and so on.

but it has all the data needed to restore as systempackages can be installed at will.

with a virtual machine you have the ability to export which is basicly the same as a disk clone. when needed you can import it on any vm and everything should work expecting your not using external storage on your vm cloud because that is not exported with the vm.

hope this was helpfull,

If you rsync entire nextcloud, is backup of database still needed ?

yes

1 Like

Perfect thanks,

No VM is not using any external storage.

So with all the methods it seems for me the 2 main methods are still to image/clone vs nextcloud data,database backups etc.

I assume I can export the clone to another disk or external drive ? Then import the VM as needed.

Image/cloning seems easier to a degree but I’m not familiar enough with backing up to know which is best, preferred among server admins etc.

Do server admins typically do VM clone exports as a backup source or do they prefer a data/database backup or both ?

My inexperience doesn’t really give me enough info to choose. I would just be guessing mostly without a reference to what is a preferred server admin method for this. Is there a preferred method for server admins or is it subjective to a degree ?

Thanks all.

I am just a ict fanatic with years of reading knowledge. I do not administer outside my own home.

A True backup consist of several data copies and can differ from size.

Approach one :

Create a full system backup once ( this optionally includes nextcloud installation and optionally the user data ) this may take some time and is best done with minimal services running.

once this backup is complete, you create a copy of that backup, which you will store in a secure fire retardant box atleast 5km from the other backup which should also be placed in a fire retardant box.

then run a backup every X ( day / week / month depending on your activity ) that only copies / delete files that have been changed / added or removed and again repeat this process for the 2nd backup disk ( back in the days we used tape streamers ).

this backup type is called incremental and is in my opinion the best method.

besides these backups I would still recommend to make a SQL DATABASE backup aside ( all databases not only nextcloud ) .

ps. Very few backups follow these rules but it’s the only way

Backup by cloning :

This backup type is widely used by desktop suppliers where you buy a Desktop with windows pre installed.
Offices-es use them to setup workstations.
They setup a workstation once. Then multiply the disk by cloning and put a copy in each workstation where the employees login all have the same software pre-installed.
This can ofcourse also be applied to pre-installed servers.
Where as you now see Docker which is basicly a clone of the software running in a vm. ( roughly seen through my eyes ) .

This is by far the easiest method and has been my favorite in the beginning of my ict learning so 30 years ago using the software Ghost by symantec before I knew dd the easy way to do it :slight_smile:

VM Export :

Personaly I only have experience with virtualbox. Where you assume correct a VM can be exported to anything large enough to keep your data. and when imported on a different machine it should still work. Besides the networking you have set on any HOST machine. ex. if you set iptables to forward a port to a vm you will have to redo that when using a different HOST.

this is all what I can say and is all minimalistic information of what there is of information.
for all backup types it is required/preferred to umount your disk so that there is no activity current.

my advice as I read your post is :
get DSL ( Damn small linux ) or tinyCore or any other
create a VM
install above linux
Personalize or mark the system
export VM
backup exported VM
delete VM ( not the backup ofcourse :stuck_out_tongue: )
restore VM ( from the backup )

see do and learn

here are some quick links about incremental backups using rsync read them then search for some more information.

https://linuxconfig.org/how-to-create-incremental-backups-using-rsync-on-linux

https://www.perfacilis.com/blog/systeembeheer/linux/incremental-back-up-script-using-rsync.html

finally to close it off this is what I use to backup nextcloud installation and nextcloud database.
I store my NC user data outside the installation folder so to run this back-up takes about 2 min although it depends on how many apps are installed and files are stored.

If you like to use this work in progress notice it’s made for ME so adjust to your needs.

you need to create a file db_auth.cfg that will store your db password for your root user ( I should fix this insecurity )

 echo SQL-root-password | base64 > db_auth.cfg && sed -i '1s/^/DB_NC=/' db_auth.cfg; sleep 2 && chmod 400
#!/bin/bash

#
# Nextcloud Settings Script
# for backingup Nextcloud
#
source db_auth.cfg
ncpath='/var/www/nextcloud'  # ADJUST NC Installation location 
bckuppath='/home/USERNAME/nextcloud-backup/'  # ADJUST Where to store the backup ( trailing slash / )
htuser='www-data'
htgroup='www-data'
rootuser='root'  # User to connect to the mysql database
database='nextcloud'  # ADJUST Database name
version=$(cat $ncpath/version.php | head -n 2 | tail -n1 | awk '{ print $3 }' | cut -c 7-14 | sed 's/,/./g')
sbt=$(date +"%x-%H:%M")
dbp=$(eval echo ${DB_NC} | base64 --decode)
function version {
printf "What version Nextcloud are you running\n"
read -p "Version: " version
if [[ $version == "" ]]; then
version
else
startup
fi
}

function startup {
echo
echo "Nextcloud Version: ${version}"
echo
echo "Set Maintenance to BACKUP or for PRODUCTION"
echo "1: PRODUCTION"
echo "2: BACKUP"
echo "3: RESTORE"
echo "0: EXIT"
read -p "Choose 1,2 or 3: " ans
if [[ $ans == "1" ]]
then
production
startup
elif [[ $ans == "2" ]]
then
full
startup
elif [[ $ans == "3" ]]
then
restore

else
quit
fi
}

function production {

printf "Setting Maintenance mode off\n"
echo " Choose y/n "
read -p " Setting Maintenance mode off: [y/n] " prod
if [[ $prod == "y" ]]; then
sudo -u ${htuser} php ${ncpath}/occ maintenance:mode --off
else
echo " Maintenance Mode still ON "
fi
echo " Finished "
}

function full {
read -p " Answer all Yes: [y/n] " fbckup
if [[ $fbckup == "n" ]]; then
backup
else
echo " Turning on maintenance:mode "
sudo -u ${htuser} php ${ncpath}/occ maintenance:mode --on
echo " Back-up ${ncpath} "
# Remove  --exclude 'data'  in the next line to include the data directory to your backup
sudo rsync -Aavx --exclude 'data' ${ncpath}/ ${bckuppath}nextcloud-${version}-`date +"%x-%H:%M"`/
echo " Back-up SQL Database to nextcloud-${version}-${sbt}.sql "
mysqldump --single-transaction --default-character-set=utf8mb4 -u ${rootuser} -p${dbp} ${nextcloud} > ${bckuppath}nextcloud-${version}-`date +"%x-%H:%M"`.sql
echo " Turning off maintenance:mode "
sudo -u ${htuser} php ${ncpath}/occ maintenance:mode --off
fi
echo " Back-up Complete "
quit
}

function backup {
# Sets permissions of the Nextcloud instance for backing up
echo " Choose y/n "
read -p " Setting Maintenance mode on: " yn
if [[ $yn == "y" ]]; then
sudo -u ${htuser} php ${ncpath}/occ maintenance:mode --on
else
echo " Maintenance mode still OFF "
fi
echo " Choose y/n "
read -p " Create backup of '${ncpath}' : " yn1
echo
if [[ $yn1 == "y" ]]
then
# Remove  --exclude 'data'  in the next line to include the data directory to your backup
sudo rsync -Aavx --exclude 'data' ${ncpath}/ ${bckuppath}nextcloud-${version}-`date +"%x-%H:%M"`/
else
echo " Not Backing up '${ncpath}' "
fi
echo " Choose y/n "
read -p " Create backup of MYSQL database 'nextcloud' : " yn2
echo
if [[ $yn2 == "y" ]]
then
mysqldump --single-transaction --default-character-set=utf8mb4 -u ${rootuser} -p${dbp} ${nextcloud} > ${bckuppath}nextcloud-${version}-`date +"%x-%X"`.sql
else
echo " Not backing up 'nextcloud' database "
fi
echo
echo " Finished "
echo " Turning off maintenance:mode "
sudo -u ${htuser} php ${ncpath}/occ maintenance:mode --off
}

function restore {

dirs=($bckuppath*/)

read -p "$(
        f=0
        for dirname in "${dirs[@]}" ; do
                echo "$((++f)): $dirname"
        done

        echo -ne 'Please select a directory > '
)" selection

selected_dir="${dirs[$((selection-1))]}"

echo "You selected '$selected_dir'"

}

function quit {
echo
echo "Peace out."
exit
}

startup
echo " foo "

this will create a backup of your installation like so

510M	nextcloud-backup/nextcloud-20.0.7.1-22-04-21-13:17
533M	nextcloud-backup/nextcloud-20.0.9.1-28-04-21-01:53
580M	nextcloud-backup/nextcloud-21.0.2.1-23-05-21-20:32
580M	nextcloud-backup/nextcloud-21.0.2.1-23-05-21-21:39
580M	nextcloud-backup/nextcloud-21.0.2.1-23-05-21-21:51
580M	nextcloud-backup/nextcloud-21.0.2.1-23-05-21-21:54
580M	nextcloud-backup/nextcloud-21.0.2.1-23-05-21-22:00
580M	nextcloud-backup/nextcloud-21.0.2.1-25-09-21-03:13
595M	nextcloud-backup/nextcloud-21.0.5.1-18-11-21-00:09
608M	nextcloud-backup/nextcloud-22.2.3.0-18-11-21-00:31
640M	nextcloud-backup/nextcloud-22.2.5.1-07-03-22-14:23
711M	nextcloud-backup/nextcloud-23.0.3.2-22-04-22-20:04
714M	nextcloud-backup/nextcloud-23.0.4.1-02-05-22-14:30
698M	nextcloud-backup/nextcloud-23.0.4.1-06-05-22-22:05

and for the database

-rw-rw-r-- 1 vincent vincent 200M apr 22  2021 nextcloud-backup/nextcloud-20.0.7.1-22-04-21-13:18.sql
-rw-rw-r-- 1 vincent vincent 194M apr 28  2021 nextcloud-backup/nextcloud-20.0.9.1-28-04-21-01:55:14.sql
-rw-rw-r-- 1 vincent vincent 183M mei 23  2021 nextcloud-backup/nextcloud-21.0.2.1-23-05-21-20:34:45.sql
-rw-rw-r-- 1 vincent vincent 183M mei 23  2021 nextcloud-backup/nextcloud-21.0.2.1-23-05-21-21:57:18.sql
-rw-rw-r-- 1 vincent vincent 183M mei 23  2021 nextcloud-backup/nextcloud-21.0.2.1-23-05-21-22:01.sql
-rw-rw-r-- 1 vincent vincent 249M sep 25  2021 nextcloud-backup/nextcloud-21.0.2.1-25-09-21-03:14.sql
-rw-rw-r-- 1 vincent vincent 308M nov 18 00:10 nextcloud-backup/nextcloud-21.0.5.1-18-11-21-00:10.sql
-rw-rw-r-- 1 vincent vincent 308M nov 18 00:31 nextcloud-backup/nextcloud-22.2.3.0-18-11-21-00:31.sql
-rw-rw-r-- 1 vincent vincent 288M mrt  7 14:23 nextcloud-backup/nextcloud-22.2.5.1-07-03-22-14:23.sql
-rw-rw-r-- 1 vincent vincent 264M apr 22 20:04 nextcloud-backup/nextcloud-23.0.3.2-22-04-22-20:04.sql
-rw-rw-r-- 1 vincent vincent 265M mei  2 14:30 nextcloud-backup/nextcloud-23.0.4.1-02-05-22-14:30.sql
-rw-rw-r-- 1 vincent vincent 265M mei  6 22:07 nextcloud-backup/nextcloud-23.0.4.1-06-05-22-22:07.sql

This is in no way associated with NC there is no support available what so ever. If admins find this not fit for learning feel free to remove it.

3 Likes

Thanks this seems to ring with others as well. Some seem satisfied with some type of image / full backup while others seem to only backup data/databases etc. Then others do both and also variants of all the above.

I still wonder what server admins actually do for vps’s and VM’s though. If there is such a standard for professional methods. Or is this just also subjective to the user / admin.

Would be nice if there was a standard and if I could form my opinions downward from there to see how much of that particular standard I would want to meet or if I wanted to just do my own thing that satisfies me only.

With so many variants it’s really hard to measure what is what and why ?

Although I understand everything people are saying I still didn’t really see a specific standard for server admins if there is such a thing or if it’s still mostly left up to a personal preference etc.

I liked your reference to the PC sellers that have the backup partition to restore to factory defaults with an OS and it’s pre-installed software etc.

I think I could be satisfied with VM exporting a full backup and rsync that with only new data seems good enough for me, but then that seems to limit to me to only restoring to VM. If this is a reliable method I would be happy with that I think since it simple enough.

your mixing things up. The exported VM contains a complete system copy, This method is more of the image/cloon type . There is nothing to rsync to.
It will also always ( minimal ) be as big in size as all diskspace in the VM.

rsync is more of a single file backup which makes is easier to restore single files incase of a single user file corruption.

backups are made depending in the company policy. The admin may have some input.
overall backups are ran daily.

https://cloudzy.com/knowledge-base/how-to-backup-files-vps/#Backup_Use-Cases

rsync made easy
http://www.debianadmin.com/rsync-backup-web-interfacefrontend-or-gui-tools.html

1 Like

I wrote some backup / restore scripts some time ago which uses borg backup. You can find them here

A good starting point.

Also, Carsten, a nextcloud expert has published similar scripts on his blog to help with backups and restore

Hope this helps

2 Likes

Nextcloud backup has always been a pain in the bu*t.
One would think that for a solution that also strives to be used in business environments, there would be a sophisticated backup solution, but there isnt.

Its a shame that backup is always left to the individual user to create his own (cron)scripts and everyone does it differently.
For someone looking in from the outside, it seems as if Nextcloud devs are just too lazy to create a valid backup solution.

Nextcloud NEEDS to have a built-in solution, where you can

  1. select which data / app you want to backup
  2. choose the backup target (smb, sftp…)
  3. schedule those backups
  4. easily restore backups with one click.

I have not tested the new backup app yet.
However, there was an app in the past that claimed to perform database backup but actually BROKE the database when trying to restore it.

Again:
For a software solution that aims to put data storage / private cloud functionality in users hands, a fully fleshed out backup solution should have been the FIRST feature to be implemented.
I can really not see any scenario where any company would use NC without having a proper built-in way to restore data.
Letting users fill in the gaps of functionality that should have been built in is sad.

5 Likes

Well clearly there are scenarios, because many companies and large organistaions are using Nextcloud. And I’m pretty sure they have some backup soultion implemented :wink: It’s the SMB market and home users that usually struggle with this.

I don’t know any comparable applications running on LAMP/LEMP stacks or literally any business applications that can do full self-contained backups of themselves, and of all the various services they depend on…

Basically there are four reasonable options for smaller companies if they don’t want to thinker and script their own backup solution.

  1. Outsource the planing, installation and maintenance of your IT infrastructure to an IT service provider

  2. use a hypervisor like Proxmox with an integradeted backup solution. This obviously solves the problem of not having a complete backup. And in combination with e.g. Proxmox Backup Server or another professional backup solution it’s even possible to mount VM backups and restore single files from them.

  3. For bare metal use a professional backup software. But you really shouldn’t do bare metal installations in the first place.

  4. A combination of the above points

In a professional IT environment backup is not just a simple app that you can add to anything with one click. Backup is first and foremost a concept and should be an integral part of your infrastructure planning from the beginning.

4 Likes

thanks, I’ll read up on this since I have little experience with it.

I get all these reasons and it makes sense.

However, the subject of self hosting as a generic term for most desktop users implies something that really doesn’t exist. Yes, you can self host and from there stems subjects like raid, server, backups etc.
However, the generic term of “DYI” and “Self Hosting” over simplifies the idea for most people. I realize it’s much more complicated then this.

Without realizing it before hand the generically minded home user gets sucked into this HUGE vaccuum of complicated ideas and solutions after the fact.

There is no easy answer or solution for them once they have installed nextcloud on their PC or on their cheap experimental server they may have purchased used on ebay or some auction site.

There are many more NON-admins then admins that can handle all these variants of options.

So having said that, in the end most will not end up with a backup solution unless someone literally tells them: which is best, which is typical for admins, or which is reasonable and simplest for them.

I see the backup app built in and it’s attractive but I don’t really understand what it actually does. I see incremental backups.

I don’t fully understand if it backups full and then increments from there or if it backups data and databases etc.
Attractive as it seems I still don’t even really understand what that does.

I do understand export of the VM and import that. This seems simple enough but how reliable I do not know.

I think for me I really need to read more and it’s unfortunate because it’s a lengthy time consuming process for those who only want to avoid costly services or addons like ONEDRIVE with expansion space costs. Many only want a network drive solution they can use and access from web as well without paying Microsoft or some other company that sells, tracks and uses any data they collect from you.

The tangled web of complicated solutions for this seems like it could indeed be simplified with at least a built in full backup to external drive and encryption.

For me this seems the easiest least complicated solution, but the other problem is that would be for a generic install of nextcloud with default storage locations and then again this complicates things as storage, databases, and users space may be setup differently for every self hosted nextcloud out there.

WHEW this merry go round doesn’t end does it ?
Thanks for all the posts this give me a lot to think about.

I think for me, I need to learn most about the export and restore features of VM for now at least I can work from there as to what else I might want to do. And those scripts people wrote will be good for me to customize something from there too.

Thanks all

Hi,

maybe it makes sense mentioning that there are Nextcloud appliances that feature backup solutions out of the box. To mention just a few: Nextcloud VM, NextcloudPi and Nextcloud AIO. Maybe it is worth for you to have a look at them.

Hope it helps!

1 Like

I am aware of of multiple enterprise software solutions / appliances and all of them have built-in backup solutions.
In fact, these appliances do not consider 3rd party solutions (VM Snapshot or backup via external tool) a valid backup and only the internal backup solution is covered by support (as it should be).

Even small projects like Tiny Tiny RSS or Pi-Hole have built-in backup solutions that work perfectly fine.
Just dump the database to a “text” file and (optionally) backup the files.
Then create a restore setting in nextcloud where you can upload the database and / or files.
Its really not that hard and everyone does it like that.

In a business environment you can really tell the quality of a software by the quality of the backup feature.
If losing data can cost you money, you dont want to rely on some third party DIY solution that may or may not work.

And in the non-business environment: well, just look at Captain86s response.
Thats the exact thought of the majority of casual NC users (like myself).
If there is no built-in backup solution, then most people will not use any backup at all.

That is the exact reason why I dont use NC for any “critical” (as critical as it can get in a private envrionment) data.