Nextcloud AIO backup directory stays empty

ℹ️ Support
, , ,
1

Nextcloud AIO version: 9.0.1
Operating system and version: unRAID 6.12.10

The location set as a target for my AIO backups remains empty, even after initiating a backup. Furthermore, the backup fails, citing missing space as the reason, despite there being multiple terabytes of space available.

There was an issue last night when I attempted to upgrade the containers from the AIO interface and am now asked to reinstall the whole package from a backup. This is when I noticed that, despite having thought to have set it up correctly, no backups were ever created.

The backup location was set via a volume mapping to /mnt/backup which is what I then set in the AIO GUI. It then points to a share in unRAID.

In the Nexcloud AIO GUI it says Backups will be created in the following directory on the host: /mnt/backup/borg. Why is that? That’s not a directory I ever specified, nor one that exists on my host or is ever created by BorgBackup; while the backup runs, I’ve been watching all my drives and there’s no major uptick in write to any of them (array HDDs, cache SSD, boot USB), nor the RAM (I really am puzzled as to where it is writing this data to). In the /mnt/docker-aio-config/data/configuration.json, the path is set to /mnt/backup as I’ve specified it.

configuration.json:

{
    "password": "REDACTED",
    "nextcloud_datadir": "/mnt/user/NextcloudData",
    "secrets": {
        "INSTANCE_ID": "REDACTED",
        "NEXTCLOUD_PASSWORD": "REDACTED",
        "BORGBACKUP_PASSWORD": "REDACTED",
        "TURN_SECRET": "REDACTED",
        "SIGNALING_SECRET": "REDACTED",
        "TALK_INTERNAL_SECRET": "REDACTED",
        "DATABASE_PASSWORD": "REDACTED",
        "REDIS_PASSWORD": "REDACTED",
        "ONLYOFFICE_SECRET": "REDACTED",
        "RECORDING_SECRET": "REDACTED",
        "FULLTEXTSEARCH_PASSWORD": "REDACTED",
        "IMAGINARY_SECRET": "REDACTED"
    },
    "apache_port": "11000",
    "nextcloud_memory_limit": "8192M",
    "nextcloud_enable_dri_device": "true",
    "domain": "REDACTED",
    "borg_restore_password": "",
    "isClamavEnabled": 1,
    "isOnlyofficeEnabled": 0,
    "isCollaboraEnabled": 0,
    "isTalkEnabled": 1,
    "isTalkRecordingEnabled": 1,
    "isImaginaryEnabled": 1,
    "isFulltextsearchEnabled": 1,
    "isDockerSocketProxyEnabled": 1,
    "AIO_URL": "REDACTED:8095",
    "wasStartButtonClicked": 1,
    "install_latest_major": "",
    "AIO_TOKEN": "REDACTED",
    "nextcloud_additional_apks": "imagemagick",
    "nextcloud_additional_php_extensions": "imagick",
    "nextcloud_keep_disabled_apps": "true",
    "borg_backup_host_location": "/mnt/backup",
    "backup-mode": "backup"
}

The backup log:

Initializing repository...
using builtin fallback logging configuration
33 self tests completed in 0.24 seconds
Initializing repository at "/mnt/borgbackup/borg"
Key in "<Repository /mnt/borgbackup/borg>" created.
Keep this key safe. Your data will be inaccessible without it.
check_free_space: few segments, not requiring a full free segment
check_free_space: calculated working space for compact as 0 bytes
check_free_space: required bytes 169138, free bytes 16281665536
security: previous location file /root/.config/borg/security/05d34f25386f1ef8de40aec3bf28149bc55e653f90712a191795de1237ec78e8/location not found
security: manifest timestamp file /root/.config/borg/security/05d34f25386f1ef8de40aec3bf28149bc55e653f90712a191795de1237ec78e8/manifest-timestamp not found
security: determined newest manifest timestamp as 
security: remembering previously unknown repository
security: saving state for 05d34f25386f1ef8de40aec3bf28149bc55e653f90712a191795de1237ec78e8 to /root/.config/borg/security/05d34f25386f1ef8de40aec3bf28149bc55e653f90712a191795de1237ec78e8
security: current location   /mnt/borgbackup/borg
security: key type           5
security: manifest timestamp 2024-06-22T13:27:57.113478
security: repository checks ok, allowing access
Synchronizing chunks cache...
Archives: 0, w/ cached Idx: 0, w/ outdated Idx: 0, w/o cached Idx: 0.
Verified integrity of /mnt/borgbackup/borg/index.1
Done.
security: saving state for 05d34f25386f1ef8de40aec3bf28149bc55e653f90712a191795de1237ec78e8 to /root/.config/borg/security/05d34f25386f1ef8de40aec3bf28149bc55e653f90712a191795de1237ec78e8
security: current location   /mnt/borgbackup/borg
security: key type           5
security: manifest timestamp 2024-06-22T13:27:57.113478

By default repositories initialized with this version will produce security
errors if written to with an older version (up to and including Borg 1.0.8).

If you want to use these older versions, you can disable the check by running:
borg upgrade --disable-tam /mnt/borgbackup/borg

See https://borgbackup.readthedocs.io/en/stable/changes.html#pre-1-0-9-manifest-spoofing-vulnerability for details about the security implications.

IMPORTANT: you will need both KEY AND PASSPHRASE to access this repo!
If you used a repokey mode, the key is stored in the repo, but you should back it up separately.
Use "borg key export" to export the key, optionally in printable format.
Write down the passphrase. Store both at safe place(s).

Repository successfully initialized.
Performing backup...
Starting the backup...
Creating archive at "/mnt/borgbackup/borg::20240622_132758-nextcloud-aio"
0 B O 0 B C 0 B D 0 N nextcloud_aio_volumes                                     
zing cache transaction: Reading config
zing cache transaction: Reading chunks
zing cache transaction: Reading files
O 1.22 MB C 1.22 MB D 37 N nextcloud_aio_volum...gK1zWr-JoM2Xu0z.mjs.map
O 1.96 MB C 1.94 MB D 169 N nextcloud_aio_volum...ps/activity/l10n/it.js
 O 2.44 MB C 2.42 MB D 406 N nextcloud_aio_volu...admin_audit/l10n/hu.js
 O 3.30 MB C 3.27 MB D 703 N nextcloud_aio_volu...ings/package-lock.json
 O 3.98 MB C 3.90 MB D 909 N nextcloud_aio_volu...ommand/CirclesTest.php
 O 4.37 MB C 4.29 MB D 1199 N nextcloud_aio_volu...fication/Notifier.php
 O 4.79 MB C 4.66 MB D 1541 N nextcloud_aio_volu...comments/l10n/gl.json
 O 5.14 MB C 4.99 MB D 1904 N nextcloud_aio_volu...poser/ClassLoader.php
 O 5.74 MB C 5.55 MB D 2112 N nextcloud_aio_volu...tPrincipalBackend.php
...
 O 18.04 GB C 16.25 GB D 110364 N nextcloud_aio_vo...bps NR4 ISO8000.mp4
 O 18.06 GB C 16.26 GB D 110364 N nextcloud_aio_vo...bps NR4 ISO8000.mp4
ud_aio_volumes/nextcloud_aio_nextcloud_data/REDACTED/files/Backups/Projekte/[21-05] Everything about Fujifilm X-T3 ISO Noise Performance for Video and Photo/2 Footage/Video/FLog 4K H265 L-Gop 400Mbps NR4 (640-12800)/FLog 4K H265 L-Gop 400Mbps NR4 ISO8000.mp4: read: [Errno 28] No space left on device
19.90 GB O 18.06 GB C 16.27 GB D 110364 N nextcloud_aio_vo...bps NR0 (640-12800)
...
OOS-errors continue indefinetely

I stopped the run after about 30 minutes.

The backup log from a second run, where I also mounted /mnt/borgbackup/borg to my backup share to see if that would change anything in case the path wasn’t on the host but the container:

Performing backup...
Starting the backup...
Creating archive at "/mnt/borgbackup/borg::20240622_121614-nextcloud-aio"
/nextcloud_aio_volumes/nextcloud_aio_nextcloud_data/REDACTED/files/Backups/Projekte/[21-05] Everything about Fujifilm X-T3 ISO Noise Performance for Video and Photo/2 Footage/Video/FLog 4K H265 L-Gop 400Mbps NR4 (640-12800)/FLog 4K H265 L-Gop 400Mbps NR4 ISO6400.mp4: read: [Errno 28] No space left on device
/nextcloud_aio_volumes/nextcloud_aio_nextcloud_data/REDACTED/files/Backups/Projekte/[21-05] Everything about Fujifilm X-T3 ISO Noise Performance for Video and Photo/2 Footage/Video/FLog 4K H265 L-Gop 400Mbps NR4 (640-12800)/FLog 4K H265 L-Gop 400Mbps NR4 ISO8000.mp4: read: [Errno 28] No space left on device
...

PS: In the AIO GUI, I also get this line about a dozen times: **Warning**: Undefined array key 1 in **/var/www/docker-aio/php/src/Docker/DockerActionManager.php** on line **753**. This comes after switching back from nextcloud/all-in-one:beta to nextcloud/all-in-one; the former wouldn’t allow the container to start, citing nextcloud.com could not be found due to a potential DNS issue; I had originally changed to the beta branch to get NC29 about two months back.

2

Hi, can you post the output of sudo mount | grep /mnt here?

3

Where should I run this? On my host (so via SSH on my unRAID server)?

4

On the host that runs docker and AIO

5 
rootfs on /mnt type rootfs (rw,size=16322288k,nr_inodes=4080572,inode64)
tmpfs on /mnt/disks type tmpfs (rw,relatime,size=1024k,inode64)
tmpfs on /mnt/remotes type tmpfs (rw,relatime,size=1024k,inode64)
tmpfs on /mnt/addons type tmpfs (rw,relatime,size=1024k,inode64)
tmpfs on /mnt/rootshare type tmpfs (rw,relatime,size=1024k,inode64)
/dev/md1p1 on /mnt/disk1 type xfs (rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/md2p1 on /mnt/disk2 type xfs (rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/md3p1 on /mnt/disk3 type xfs (rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/nvme0n1p1 on /mnt/cache type btrfs (rw,noatime,ssd,discard=async,space_cache=v2,subvolid=5,subvol=/)
shfs on /mnt/user0 type fuse.shfs (rw,nosuid,nodev,noatime,user_id=0,group_id=0,default_permissions,allow_other)
shfs on /mnt/user type fuse.shfs (rw,nosuid,nodev,noatime,user_id=0,group_id=0,default_permissions,allow_other)
/dev/sdd1 on /mnt/disks/Ausweichlaufwerk type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096)
/mnt/cache/system/docker/docker.img on /var/lib/docker type btrfs (rw,noatime,ssd,space_cache=v2,subvolid=5,subvol=/)
/mnt/cache/system/docker/docker.img on /var/lib/docker/btrfs type btrfs (rw,noatime,ssd,space_cache=v2,subvolid=5,subvol=/)
/mnt/cache/system/libvirt/libvirt.img on /etc/libvirt type btrfs (rw,noatime,ssd,space_cache=v2,subvolid=5,subvol=/)
6

Thanks!

Based on your command no mountpoint in /mnt/backup was found. That means that the backups get created on this mount:

7

Browsing /mnt on my server, I do not get anything that resembles a backup, though.

Screenshot 2024-06-22 163633

Furthermore, the backups never complete due to out-of-space errors - which is understandable, considering it would need to write into /mnt/user/MyBackupShare, for example, to actually end up on any of my disks. As long as that happens, I am unable to recreate my AIO instance.

The backup fails anwhere from 13 to 20 GB into the backup (sadly, I don’t how how large my DB and such is and don’t know what percentage into the backup this corresponds to).

How would you have me continue?

Thanks for your swift responses so far.

8

You can adjust this to "borg_backup_host_location": "/mnt/user/MyBackupShare", and it should make it work

9

Sorry, made a typo. Should be correct now

10

Thanks, this looks to be working.

I found the error to be myself thinking the paths set via the AIO GUI be within the container since that’s what I’m accustomed to from any other Docker container. I had not for a second assumed this could refer to a path on the host directly. I feel a little stupid now but am incredibly thankful for your help and how easy the AIO team made this, compared to what I’m used to.

1 Like
11

One more issue: BorgBackup seems to back up all data. That would be terrabytes of data that don’t need to get backed up because the Nextcloud data itself is completely separate. I only want to back up configurations, databases, et cetera. From having read past forum posts, I had always assumed that’s what it does, but it seems not.