101: backup what and why (not how)

đź“‘ How to
, , , , , ,
1

we frequently see questions related to backup so I decided to write an article how to perform a backup of Nextcloud system. this post in not going to explain any specific command or tool but rather gives you an overview and understanding how to setup a successful backup strategy.

The topic is very versatile and likely every admin has it own perfect backup strategy and I don’t plan cover every single details - like every other 101 topic this is intended to provide a jump start into a topic showing important things to consider and major pitfalls to avoid.

A backup is meant to allow the recovery of your data once it is lost for some reason. This is why it is crucial to perform the backup frequently… and as humans are not very good to repeat the same procedure again and again you must automate the procedure to avoid the situation when your only have many months old backup once you need it. Adopt the frequency to your needs - most private users can easily afford loosing few hours or even days of data.

disclaimer - backup doesn’t protect against data leak - it only protects against data loss - for this reason it less a security technology but rather a “resiliency” goal.

The industry adopted the “assume breach” mindset - meaning - there is no way to fully protect IT systems from breaches. Many many huge organizations telcos and security companies, Microsoft, Deutscher Bundestag where hacked how could SOHO admin prevent it? backup remains the only measure to avoid data_loss in case of disaster. Most private users and small companies don’t really suffer if bad actor steals their data as long they still can recover all/most of their assets. Definitely nothing you want to try out but it much easier to handle a ransomware attack or natural disaster knowing copies of your data still alive and safe in another location.

The backup is intended to recover all of parts of your data in case of hardware or software fault or human error. Often distributed storage like RAID or ZFS replication is considered “backup” bit it doesn’t replace real backup. Such technologies only protect from specific kind of hardware fault - particularly loss of one or more storage devices. other factors like storage controller, mainboard or CPU failure, fire, flood, earthquake are not covered.

This is where the 3-2-1 principle comes into play - keep minimum 3 copies of data on 2 different storages and one copy “off-site” and offline. This simple to remember rule covers almost every possible disaster. different storage devices protect against singe device fault, off-site copy protects against natural disaster like fire and flood, offline copy protects against human error and malware attack when all connected data is destroyed. The hard part is to turn this strategy into reality: it’s trivial to make backups to another drive on the same server but this is not off-site and not offline. For off-site backups there are many storage offerings today at reasonable price tags ~5-6 USD/EUR per TB per month. this allows off-site and online copy and could be easily automated. don’t fear to use cloud storage for backups - as you don’t plan to work with the data it’s easily encrypted - so the provider can’t steal access your data even in case you use the most untrusted provider in the world. offline copy is likely the hardest part - as long you are not an enterprise running an army of admins and huge tape changing robot farms the only reasonable way would be to copy your backup archive to a USB drive from time to time. This copy will allow recovering from encrypt ransomware case. You definitely will run this on longer intervals and suffer from more data loss in case this is the only copy surviving the incident . This procedure could be automated as well by automatically staring copy job when specific device is attached - but still required manual attaching the “offline” device. Another approach would be to have some “offline” server which starts from time to time performs a backup and shutdowns once the copying completes.

and last but not least - periodically test your backups. Think of a very dramatic issue when your server is completely lost and try to recover from your offline storage, cloud backup etc - do you have all keys and passwords, do you know the right commands?

And because users keep asking - no it is really not possible, no way, never to fully backup/restore Nextcloud system using a client (and without access to the server). Client are useful for data access and synchronization, but they cannot provide a complete and consistent backup of a running Nextcloud instance with all additional features like trashbin, versions, metadata, calendar, contacts. Server-side backups using appropriate tools (rsync, borg, mysqldump, etc.) are absolutely essential for real complete backup. You must back up the database, configuration files, and data directory on the server itself.

The most important points:

  • there is no way to perform a backup client-side
  • RAID, file system snapshot and similar are not backups
  • automate backup procedures
  • periodically test restore
  • consider 3-2-1 rule

Here’s a summarized list of what to back up in a Nextcloud system and why:

Nextcloud Data Directory

  • What: Contains all user files, shared files, and other data (Talk recordings, calendar data, etc.).
  • Why: This is the core data. Loss of this directory means loss of all user content. Clients sync some of this, but not all and not consistently.

Database (MariaDB or PostgreSQL)

  • What: Stores metadata about files, users, sharing permissions, app settings, etc. 1
  • Why: Essential for Nextcloud’s functionality. Without the database, the restored files are just raw data without context. Clients do not sync this.

Nextcloud Configuration

  • What: Contains config.php and other configuration files.
  • Why: Without this, you’d have to completely reconfigure Nextcloud, even if you restored the data. Clients do not sync this.

Web Server Configuration (Apache or Nginx)

  • What: Configuration files for your web server.
  • Why: Restores web server settings for Nextcloud. Clients do not sync this.

Operating System or Appliance (Docker, docker-compose, snap, AIO, NCP):

  • What: Configuration files for the “overlay” system you are using
    • in case of docker/docker-compose your command line and arguments or files (volumes, bind mounts, .env files and variables)
  • Why: allows recovery in a disaster by restoring the entire system state

For details review

5 Likes
2

Hey, great guide! Should this also include a link to GitHub - nextcloud/all-in-one: 📦 The official Nextcloud installation method. Provides easy deployment and maintenance with most features included in this one Nextcloud instance.?

3

implementation example

below is a drawing how I do my backup now. This is not means to be a “best practice” or the ideal solution. This my current solution I ended after some time lot of thinking and some testing.

  • all systems running as docker containers
  • automatic backup of running applications using mazzolino/restic container
    • perform required DB backup, maintenance mode etc
    • run pre-/post-backup scripts e.g. notification in a Talk room
  • automatic pull of restic repo to a dedicated NAS
  • automatic restic backup of running applications to cloud object storage (s3 bucket) using another instance of restic
    • remote storage
  • manual rsync of restic repo to offline usb disk (#1 #2 etc…)
    • connect USB drive, sync, disconnect

this far from perfect solution but gives me very good feeling to cover many desaster scenarios:

  • offline backup will survive ransomware attack encrypting online storage
  • online backup (in remote location) will cover hardware outage and natural disaster

I choose restic restic_backup for my backup but there is nothing wrong to use borgbackup solution used by AiO. both are open source backup programs supporting various targets and deduplication - which gives you countless versions of unchanged data at no storage cost. Review the docs for details. Both restic_backup and borg_backup share one disadvantage as well - data is stored in a “proprietary” format so you can not restore using built-in OS tools.

flowchart TB
  subgraph local["local self-hosted site at home"]
	  subgraph srv[docker host]
	  direction TB
		  nc[Nextcloud prod]
		  dev-nc[dev Nextcloud<br>almost same as prod]
		  keycloak["keycloak (OIDC IdP)"]
			restic(restic local<br>container)
			keycloak & nc & dev-nc --> restic
	   restic_s3(restic s3<br>container)
			keycloak & nc & dev-nc --> restic_s3	   
    end

    subgraph online[online backups]
    subgraph nas [QNAP 453]
					resticnas[(restic repo)]
    end
    end
    subgraph offline[offline backups]
  	  direction TB
		  subgraph usbdisk1[usb disk #1]
			 backup_usbdisk1[(restic_repo)]
	    end
		  subgraph usbdisk2[usb disk #2]
		    backup_usbdisk2[(restic repo)]
		   end
    end	
					restic -- "`rsync *pull*<br>over ssh`" --> nas
restic -- "`*manual* rsync<br>on-connect`" --> usbdisk1
restic -- "`*manual* rsync<br>on-connect`" --> usbdisk2
end

 subgraph remote[cloud storage]
 	direction LR
  	s3[s3 bucket]
  end
    restic_s3 -- scheduled push --> remote
1 Like