The Basics
- Nextcloud version:
Nextcloud AIO v9.8.0
- Operating system and version:
Debian 12
- Installation method
AIO
The Storage Model
First, I set up a Nextcloud AIO instance to run on a VPS.
Then, I configured Nextcloud to use IDrive E2, an S3-compatible object storage, as my Nextcloud’s primary storage, which stores the data on a bucket called nextcloud-main
. I followed this documentation page to set it up.
For backups, I created another bucket called nextcloud-backup
, which is mounted with s3fs on /mnt/idrivee2/nextcloud-backup
on system boot. I have configured the Nextcloud AIO instance to make backups on that mountpoint. Nextcloud uses BorgBackup for backups, and after one backup I made, Nextcloud created a directory borg
inside that mountpoint to store that backup.
Finally, I enabled server-side encryption.
It’s all working fine.
The Problem
I am setting up Nextcloud to store data from my family members, who live in different locations than mine. The idea is making each of them have their own Nextcloud account on my self-hosted instance and have their client apps sync their personal data to it.
I would also like to make, from time to time, a local backup of their Nextcloud-stored data onto an external physical drive of mine, such as an external SSD (or a regular SSD with a SATA-to-USB adapter, as I prefer). This is in case something goes wrong with Nextcloud and I can’t for the life of me restore a Nextcloud backup, in which case I will at least have access to the files. As far as security goes, losing this physically-accessible drive is not a concern, as data on it would be encrypted with VeraCrypt.
The problem is that, as the aforementioned documentation page says, when using an object storage as Nextcloud’s primary storage, all file metadata is stored inside of Nextcloud, so there is no way of bypassing Nextcloud by accessing the object storage directly.
My Findings
I did some reading on BorgBackup and I figured out I can access a Nextcloud backup’s contents. The procedure, according to my storage model, is as follows:
1. Create two mountpoints, one for the Nextcloud backup bucket and another for the decrypted Borg repository:
sudo mkdir -p /mnt/idrivee2/{nextcloud-backup,nextcloud-backup-borg}
2. Mount the bucket nextcloud-backup
with s3fs on my local computer. For this, I have added an entry on /etc/fstab
, as there are too many options to use:
# /etc/fstab
# ...
s3fs#nextcloud-backup /mnt/idrivee2/nextcloud-backup fuse rw,_netdev,allow_other,passwd_file=/etc/passwd-s3fs/idrive/nextcloud-backup.rwak,url=<idrive-e2-endpoint>,use_path_request_style,use_cache=/tmp/s3fs,enable_noobj_cache,no_check_certificate,use_xattr,complement_stat 0 0
replacing <idrive-e2-endpoint>
with the bucket’s endpoint URL.
3. Create a passwords file for a read-write access key to my nextcloud-backup
bucket and change its permissions accordingly:
sudo mkdir -p /etc/passwd-s3fs/idrive/
sudo nano /etc/passwd-s3fs/idrive/nextcloud-backup.rwak # <access-key-id>:<access-key-secret>
sudo chmod 600 /etc/passwd-s3fs/idrive/nextcloud-backup.rwak
4. Reload /etc/fstab
and mount it:
sudo systemctl daemon-reload
sudo mount -a
5. Once mounted, there will be borg
directory inside that mountpoint. Mount it with the command:
borg mount /mnt/idrivee2/nextcloud-backup/borg /mnt/idrivee2/nextcloud-backup-borg
I am prompted with the BorgBackup encryption key. I enter the encryption key I was asked to store at a safe place when I enabled the Nextcloud server-side encryption feature.
6. List the contents of the mounted borg repository to see a directory 20241110_160440-nextcloud-aio
:
ls /mnt/idrivee2/nextcloud-backup-borg
Navigating the contents of that directory, I see the following structure:
/mnt/idrivee2/nextcloud-backup-borg
20241110_160440-nextcloud-aio
nextcloud_aio_volumes
nextcloud_aio_apache
nextcloud_aio_database
nextcloud_aio_database_dump
nextcloud_aio_elasticsearch
nextcloud_aio_mastercontainer
nextcloud_aio_nextcloud
nextcloud_aio_nextcloud_data
Looking around for meaningful files, I found out there is a file database-dump.sql
in nextcloud_aio_database_dump
that I thought might have mappings of urn:oid
files (which are stored on the bucket nextcloud-main
) to the actual file paths. However, that would be only one part of the challenge, since the file contents themselves are also encrypted.
So, before I start losing hairs digging deeper into this, I wonder, is there an easier of doing what I want? Or is this the way to go – reading borg backup files to map urn:oid names to actual names and, somehow, decrypt those files manually? If the latter, how can I do that?
Any help is appreciated!
About Object Storage as External Storage
One may ask, “Why not simply use your IDrive bucket as a Nextcloud external storage, instead of primary storage?” I tried, and it’s slow to an unusable degree. It has to be primary storage.