Backup in reasonable time

gas85 · February 5, 2019, 2:03pm

What is around 17 MB per second that is not so bad for 2 Cores old CPU. Try to not using compression, this will increase your backup speed a bit, but only if you hit CPU limits.

It is already not bad to have 17 MB per second transferred on older CPU.

Try to use rsync, or fpsync. You even do not need to put server in maintenance mode. I’m using this script to simply backup to remote FS.

github.com

GAS85/nextcloud_scripts/blob/master/nextcloud-rsync-to-remote.sh

#!/bin/bash

# By Georgiy Sitnikov.
#
# Will do NC backup and it upload to remote server via SSH with key authentication
#
# AS-IS without any warranty

SSHIdentityFile=/path/to/file/.ssh/id_rsa
SSHUser=user
RemoteAddr=IP_or_host
RemoteBackupFolder=/path/to/backup
NextCloudPath=/var/www/nextcloud

# Folder and files to be excluded from backup.
# - data/updater* exclude updater backups and dowloads
# - *.ocTransferId*.part exclude partly uploaded files
#
# This is reasonable "must have", everything below is just to save place:
#

This file has been truncated. show original

Just run something like:

rsync -a --exclude=data/updater* --exclude=*.ocTransferId*.part --partial --info=progress2 --delete NextCloudPath BackupFolder/nextcloud/

I’m personally really hit my Network bandwidth limits by using rsync, so fpsync will not bring me any benefit.
Investigate what is your problem:

hit CPU Limits - try to disable compression. Encryption will also consumes your CPU.
I’m also doing like this: NC Server has an old 2 core CPU, so I use to move compression and encryption tasks to backup machine:
tar -cvp /NC/FOLDER | nc -q 0 receivingHost 8080
on Backup Server side using compression and encryption with LUKS, because CPU is more efficient.
nc -l 8080 > gzip backup.tar.gz
hit HDD Limits - check you HDD speed, may be it is old and could not handle more MBps of write.
hit USB Limits - try to move HDD to other machine and use Network (if you have more then 1Gbps Net). But it seems you are using USB 3.0 so have a look on a first hint.