Hi there,
we have a Nextcloud install with about 1000 users (but many inactive). We are currently running a Debian bare metal server with caddy install on Debian and Nextcloud (and other services) in Docker. We have outgrown our current server (4 cores, 32 GB RAM, no SSDs) and are planning to upgrade. Current plan:
- Proxmox Host with 16 Cores, 128 GB RAM, 2x 2TB NVMe, 2x 16 TB HDD
- ZFS Pools: RAID1 SSD, RAID1 HDD
- VMs: Firewall / SNI Proxy
- LXC Containers with Docker for services (Mattermost, Collabora, Nextcloud…).
Explanation for our considerations:
- Docker within LXC since we have some experience with Docker / docker-compose. We didn’t want to change everything at once, so we are keeping Docker for now. We know container within container is probably not the ideal solution.
- LXC Containers because we want to run backups / snaphots via ZFS/restic from host but still have Proxmox management capabilities (Resource Limitation, restart via GUI, Backup etc.).
- VM for Firewall / SNI Proxy / Reverse Proxy for better security
- HDD ZFS Pool for Nextcloud files, SSD ZFS Pool for everything else.
- ZFS root unencrypted, datasets for containers and VMs encrypted with ZFS encryption. Currently ZFS on top of LUKS needs login via IPMI to unlock.
Pictures / Diagrams (some text in German):
Questions:
- Anybody running a similar setup, experience with PVE/LXC/Docker?
- Especially Nextcloud in Docker in LXC on ZFS?
- We would like to run SNI Proxy (via HAProxy) behind (or in?) Firewall so that we can have TLS Termination within service containers. Any recommendations or experiences to share with SNI Proxies?
- We want a VM/Container with TURN Server for Talk, any experiences with SNI Proxy for that?
- We have decided to not use VMs but LXC containers for our “core” services so that we can run ZFS snaphots / Backup from Host with ZFS. Anybody Nextcloud and other service inside a VM with good Snapshot / Backup Strategy?
Generally: any feedback appreciated! 