What?
A single Ubuntu server - lets call it CLOUD:
LXD via SNAP.
HAProxy
Main disk: EXT4
Data disks (two Identical connected with eSATA cable): OpenZFS. Pools are created for LXC’s which will have important data, like Nextcloud.
Another Server - lets call it HOME
LXD via SNAP.
LXDMosaic for webbased management of LXD hosts.
Main disk: Small. LXC storage pool only intended for services with small footprints, like OpenLDAP.
Data Disk: EXT4. Used as rsync mirror to ZFS raid for ultimate last resort restore. Compressed and encrypted incremental upload of rsync folder to free US cloud drive (google).
LXD containers spread across the two servers:
OpenLDAP (for nc and for use with other home services like RADIUS for WP3-Enterprise)
PostgreSQL (for other services and testing migration from MariaDB to PostgreSQL with NC)
Jitsi meet
ClamAV
mail (IMAP and SMTP)
Nextcloud (Apache2, php-fpm, MariaDB, Redis)
All is routed with HAProxy with a few exceptions.
Everything runs rock solid and automated. Restore is snapshot restore mostly, and the resilvering of ZFS takes care of corrupted files. Only twice have I had to recover files from the rsync.
Why?
When I installed my first Nextcloud instance, is was on a Raspberry PI 2. The choice of NC distro was an easy choice: NCPi. However I did not find the RPi very stable so after acquiring a Lenovo Tiny Core i7 with 16GB of ram, I decided to look into another approach.
Working with security I have never had any favour of Docker and therefor already had experience with Linux containers. I was impressed with the BTRFS versioning and snapshotting, but was unimpressed with the flaws and bugs. Then I stumbled upon ZFS. I tried it, and that was it. Never looked back.
I then installed the NCPi in a Debian Buster LXC container. Let us just say that NCPi was NOT ready for LXC using the install script at that time. A lot of effort, trial and errors (thank GOD for snapshotting under ZFS!!!), local changes to the NCPi install script, and small tweaks, and it finally came to life. However NCPi exspects BTRFS and not ZFS, and DEFINATELY not when having zero privileges to do ANYTHING CLI with the filesystem controller itself. So half of the features I was enjoying from NCPi, did not work.
I then started over and decided to go with a “clean” install, following NC best practice according to documentation. I have never been more satisfied with the result. I have never been stuck with a bricked system, or lost a single file. And it runs smooth and fast.
I hope this can be an inspiration and a recommendation to look into OpenZFS as a very good alternative to BTRFS, and which plays very nicely with Nextcloud.
It’s a community project, for such implementations that are not used by tons of users, it probably needs feedback and contributions from others.
I also use this approach from the beginning. If you know your way around Linux, it’s not very difficult to setup and you are more flexible. Having it in a virtual machine can be useful for backup/restore.
For some time, I wanted to look into the FreeBSD ecosystem, where you have ZFS and the jail-environment to separate different applications. They have good documentation, however some things are slightly different compared to Linux.
Yes and I gave a few feedbacks back then, but my usecase was a cornercase at the time, and the effort was focused on Rasps and bare metal Debian Buster. In fact the install script for Debian was still “wet” from the print.
Indeed. I was exploring other OS’s as well, but when I decided on using LXC, it did not matter any longer, as I just has to pick the image of choice and download it, and create a container based on it, and my opportunities are much wider than the downsides of Linux distros compared to the benefits of BSD’s.
What I particularly likes about Linux Containers, is the isolation, yet the experience of, working on independant machines. All the threads on the main host can be tracked as they are actually threads. Yes I can see the apache2 thread of the NC container with top on the LXD HOST. But that makes monitoring much easier as well as killing any eventual hanging thread that causes a container to be unresponsive. Logging is also central - if you so choooses. And yet I can monitor each container from inside the container, and only see the relevant from that specific container.
I do have one downside though: Nextcloud recognises all storages on the entire host, so I see some snapfuse drives as well. But I think that is a local issue.
