The optimal & fastest Nextcloud-FPM docker setup with Caddy as webserver and https-proxy

zilexa · March 10, 2021, 9:01pm

I was on a mission to get Nextcloud-FPM to work with Docker-Caddy-Proxy, this Caddy container allows to use Docker-Compose labels to easily enable HTTPS/reverse proxy access and act as a webserver as well.

I have it working! I get an A+ rating at https://scan.nextcloud.com/.

With this setup, you will get:

The fastest database option (PostgreSQL).

Caching (Redis), Nextcloud without Redis is too slow.

Fastest Nextcloud build (PHP-FPM), faster than the default Apache release.

A single container for https reverse-proxy and webserver (Caddy, replacing nginx).

My docker compose:

version: "2.3"
services:
##________CLOUD________
##_____________________ Caddy [CLOUD/web-proxy]
  caddy:
    container_name: caddy-proxy
    image: lucaslorentz/caddy-docker-proxy:ci-alpine
    restart: always
    networks: 
      - web-proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - $DOCKERDIR/caddy/caddy_data:/data
      - $DOCKERDIR/caddy/config:/config
      - $DOCKERDIR/nextcloud/var/www/html:/nextcloud/var/www/html
      - $DOCKERDIR/nextcloud/var/data:/nextcloud/var/nextdata
    ports:
      - 80:80
      - 443:443
##
##____________________ NextCloud [CLOUD/Files/NextCloud]
  nextcloud:
    image: nextcloud:21-fpm
    container_name: nextcloud
    restart: always
    mem_limit: 2048m
    mem_reservation: 512m
    networks:
      - web-proxy
      - nextcloud
    depends_on:
      - nextcloud-db
      - nextcloud-cache
    environment:
      NEXTCLOUD_DATA_DIR: /var/nextdata
      NEXTCLOUD_TRUSTED_DOMAINS: next.$DOMAIN
      NEXTCLOUD_ADMIN_USER: $USER1
      NEXTCLOUD_ADMIN_PASSWORD: $USER1PW
      POSTGRES_HOST: nextcloud-db
      POSTGRES_DB: nextcloud
      POSTGRES_USER: $USER
      POSTGRES_PASSWORD: $PW_INT
      REDIS_HOST: nextcloud-cache
      REDIS_HOST_PASSWORD: $PW_INT
      SMTP_HOST: $SMTPHOST
      SMTP_SECURE: tls
      SMTP_NAME: $SMTPUSER
      SMTP_PASSWORD: $SMTPPASS
      SMTP_FROM_ADDRESS: $EMAIL
      SMTP_PORT: 587
    volumes:
        # the actual data of the Nextcloud:
      - $DOCKERDIR/nextcloud/var/nextdata:/var/nextdata
        # Main folder needed for updating:
      - $DOCKERDIR/nextcloud/var/www/html:/var/www/html
        # local configuration
      - $DOCKERDIR/nextcloud/var/www/html/config:/var/www/html/config
        # Custom settings for php fpm to make nextcloud work. The default settings resulted in the error:
        # WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
      - $DOCKERDIR/nextcloud/etc/www-custom.ini:/usr/local/etc/php-fpm.d/zz-custom.conf
    labels:
      caddy: next.$DOMAIN
      caddy.tls: $EMAIL
      caddy.file_server: "" 
      caddy.root: "* /nextcloud/var/www/html"
      caddy.php_fastcgi: "{{upstreams 9000}}"
      caddy.php_fastcgi.root: "/var/www/html"
      caddy.php_fastcgi.env: "front_controller_active true"
      caddy.encode: gzip
      caddy.redir_0: "/.well-known/carddav /remote.php/dav 301"
      caddy.redir_1: "/.well-known/caldav /remote.php/dav 301"
      caddy.header.Strict-Transport-Security: '"max-age=15768000;includeSubDomains;preload"' # Required for Nextcloud
      #caddy.header.X-XSS-Protection: '"1; mode=block;"'             # Required for FileRun+OnlyOffice
      #caddy.header.X-Content-Type-Options: "nosniff"                # Required for FileRun+OnlyOffice
      #caddy.header.X-Frame-Options: "SAMEORIGIN"                    # Required for FileRun+OnlyOffice
##____________________ NextCloud [CLOUD/Files/NextCloud/database]
  nextcloud-db:
    container_name: nextcloud-db
    image: postgres:12-alpine
    restart: always
    networks:
      - nextcloud
    environment:
      POSTGRES_USER: $USER
      POSTGRES_PASSWORD: $PW_INT
    volumes:
      - $DOCKERDIR/nextcloud/db:/var/lib/postgresql/data
      - /etc/localtime:/etc/localtime:ro
##____________________ NextCloud [CLOUD/Files/NextCloud/cache]
  nextcloud-cache:
    container_name: nextcloud-cache
    image: redis:alpine
    restart: always
    mem_limit: 2048m
    mem_reservation: 512m
    networks:
      - nextcloud
    command: redis-server --requirepass $PW_INT
#
#
networks:
  web-proxy:
    driver: bridge
  nextcloud:
    driver: bridge

zilexa · March 15, 2021, 7:48pm

I actually decided to stick to filerun https://filerun.com/
Even though I went through all this effort, Filerun is still faster/more smooth. But the difference is smaller now.

What really makes the difference: with Filerun, each user account is linked to a folder on the local disk drive. Local changes are reflected immediately. This fits nicely with your own NAS, your existing data/folder structure and makes it very easy to backup user data or use other tools to sync/copy that data to the users or to a backup location (btrfs send/receive, ssh, rsync, Syncthing).

The downside: limited to 10 user accounts/not open source. But for a homeserver situation, definitely a great choice if you do not need all the extra bells & whistles of Nextcloud, since FileRun is purely a file/drive/dropbox

skoeswanto · April 3, 2021, 10:53am

I like your setup on simple docker-compose but I want to use official caddy image. Can you give me pointers on how to setup the proper Caddyfile? I tried yours:

root * /var/www/html
    file_server
    php_fastcgi app:9000 {
         root /var/www/html
        env front_controller_active true
    }
    encode gzip

    header {
        Strict-Transport-Security "max-age=15768000;includeSubDomains;preload"
    }

    redir /.well-known/carddav /remote.php/dav 301
    redir /.well-known/caldav /remote.php/dav 301

But after few hours, the Nextcloud-app (the fpm image) will crunch at 100% CPU and logs showing:

"user":"- ","app":"PHP","method":"PUT","url":"/index.php/apps/user_status/heartbeat","message":{"Exception":"Error","Message":"ini_set(): Headers already sent. You cannot change the session module's ini settings at this time at /var/www/html/lib/base.php#402","Code":0,"Trace":[{"function":"onError","class":"OC\\Log\\ErrorHandler","type":"::","args":[2,"ini_set(): Headers already sent. You cannot change the session module's ini settings at this time","/var/www/html/lib/base.php",402,[]]},{"file":"/var/www/html/lib/base.php","line":402,"function":"ini_set","args":["session.cookie_secure","1"]},{"file":"/var/www/html/lib/base.php","line":639,"function":"initSession","class":"OC","type":"::","args":[]},{"file":"/var/www/html/lib/base.php","line":1076,"function":"init","class":"OC","type":"::","args":[]},{"file":"/var/www/html/index.php","line":35,"args":["/var/www/html/lib/base.php"],"function":"require_once"}],"File":"/var/www/html/lib/private/Log/ErrorHandler.php","Line":92,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:87.0) Gecko/20100101 Firefox/87.0","version":"21.0.0.18"

and docker log only says something about PROPFIND /remote.php (sorry I already deleted my containers)

Any advice?

Keauval · April 13, 2021, 5:36pm

This setup is working like a charm.
So fast, so reliable on my web server (dedicated 32gb 500mo ssd…), nearly as fast as Google Suite !

I’m wondering if it’s a good idea to run Nextcloud and other services (web hosting) on the same server.

In this idea, how can I use my Traefik set (for web hosting with multiple nginx-phpfpm stack) and this Caddy on the same level ? They both use 80 & 443

Many thanks for this work and for a future answer.

Jim_Daily · April 24, 2021, 2:50pm

This is just what I’m lookin for! And the variable are just Environment Variables right?

Jim_Daily · April 28, 2021, 2:34pm

Coming back to this, I tried the config above and am still having some issues

One thing I noticed is this label

caddy.root: “* /nextcloud/var/www/html”

Is that the correct root? I’m not seeing anywhere the folder structure is defined as /nextcloud/var/www/html

Also noticed that Postgres doesnt actually create a nextcloud database, assuming this is something that has to be done remotely?

JohnMayson · November 21, 2021, 9:20am

I’ve used Nextcloud for a couple of years, but I’m new to Caddy. I launched this docker-compose.yml file. All appears well. I get a “308 permanent redirect” and cannot connect to my new instance. /var/log/Caddy is empty.

I do have Apache running on this same server, but ports 80 & 443 are free. I even stopped Apache but I still have the same problem.

michaelhthomas · November 24, 2021, 10:45pm

I would definitely be careful using this config for Caddy, as it exposes a lot of protected Nextcloud folders to the interwebs (e.g. userdata). I added the following to my config (adapted from the .htaccess file), which seems to have fixed that issue.

# Block certain paths (from .htaccess)
@blocked_locations {
	path_regexp (^/(?:build|tests|config|lib|3rdparty|templates|data)(?:$|/))|(^/(?:\.|autotest|occ|issue|indie|db_|console))
}
route @blocked_locations {
	error * "Not found" 404
}

Personally, I’m planning to take a closer look at the ngnix config for nextcloud here to make sure I cover all the bases.

Samonitari · November 25, 2021, 8:35am

You are right about the issue, although the solution is wrong, and could be a more “caddy-like”!

Don’t get me wrong, I am an utmost beginner in caddyfiles (v2), but:

I had to look up the difference between error and respond, but don’t use error as a go-to solution!. To quote the docs on error: “This handler does not write a response. Instead, it’s meant to be paired with the handle_errors directive to invoke your custom error handling logic.”.
maybe it is a question of taste, but path_regexp is the advanced option when path just doesn’t cut it, at least by my understanding. Your solution is quite idiomatic in nginx - and I presume in apache.
the route directive is completly (and probably distractingly!) unnecessary, as that is for making the block’s inside “sequential as is”: for which there is no reason in a single “command” block.
overriding the body of 404 response is also pointless - unless you are writing this config for somebody who doesn’t know what does 404 mean.

The “official” caddy example uses something like this:

        @forbidden {
          path /.htaccess
          path /.xml
          path /3rdparty/*
          path /README
          path /config/*
          path /console.php
          path /data/*
          path /db_structure
          path /lib/*
          path /occ
          path /templates/*
          path /tests/*
        }

        respond @forbidden 404

Some forbidden locations differ, but I didn’t have the patience to validate. Why? See below:

It is a big pet peeve of mine - and it is also important!:
For the love of anything you people hold dear, please, write your regexes so your multiple choice blocks have their options in alphabetic order!
(?:build|test|config|lib) is the same as (?:test|lib|build|config), but nobody in ther right mind would want to compare these by eye, especially if the choices are getting numerous, like in your example!
However, that “let the chips fall where they may” approach to enumerating choices makes diff tools also saddingly unhelpful as well!.

These - especially my pet peeve - may seem like something irrevelant to you, but these forum threads are high in the search rankings, and providing good and comperable solutions would be the thing you would want too!

michaelhthomas · November 28, 2021, 2:22pm

Thank you, that is indeed a much more readable and “Caddy-like” solution. I was, unfortunately, just copying regex from the Nginx configuration and did not think to see that there might be a better way. Out of curiosity, where did you find that example for Caddy? I can’t seem to find an “official” configuration for Caddy anywhere, which is how I ended up in this discourse in the first place.

Sidenote: I (evidently) am not an expert in Caddy configuration, so thanks for the tips!

Samonitari · November 29, 2021, 3:59pm

Excuse me for that bit impatient answer though!

And here is a Caddy forum topic, with a lot of links to other, related topics.
I tried to get to the bottom of it, but for me - and for others - it seems some rewrites are missing.
Those I couldn’t get right, at least not yet, as I am not expert in any webserver config, so for the time being I am still using nginx, sadly.

zilexa · December 26, 2023, 7:45pm

Just landed here after Googling “Nextcloud Caddy Docker setup” and noticed I created this topic 2 years ago.
With FileRun development stalled i’m ready to switch to Nextcloud… but I’m not up to date on the current state of affairs…

Is this still the way to go, Nextcloud FPM? How to enable the faster backend? My compose plus the modifications of @michaelhthomas and/or @Samonitari ?

LittleWhiteFern · January 2, 2024, 6:23pm

I, too landed here after searching for Nextcloud-Caddy setups. After compiling information from this and several other sources, I landed on a setup that I would describe as ‘pretty fast… for Nextcloud’.

You never really mentioned what kind of performance you were looking for, though. ‘Fast’ is very subjective and what one person finds ‘fast enough’ might be ‘unacceptably slow’ for someone else.

For my setup, navigating from Dashboard to Files in Firefox takes about 3.5s on a local connection. Once loaded, navigating through folders is usually smooth, but with noticeable latency - around 0.4s.

Is that kind of speed acceptable to you?

Now, details - Fedora Server 39, Podman, Nextcloud-FPM, Postgresql, Redis, Caddy, and Cloudflare Tunnels. Very non-standard and unofficial.

All internal connections run over Unix sockets, PHP-FPM is tuned, I made a local CA and SSL certificates to get HTTP/2 locally, and my server is way more than fast enough. I can’t seem to get it any faster than it is without disabling apps, which is frankly a sub-optimal solution.

DevDorrejo · March 3, 2024, 8:57pm

Can you please point me how to create local CA and how to do the SSL http2 part?

LittleWhiteFern · March 3, 2024, 9:26pm

I won’t claim to fully understand the process, but I followed this guide:

You’ll end up with a bunch of certificates and key files and you won’t use most of them on the server.

homelab.crt
homelab.csr
homelab.ext
homelab.key

homelabCA.key
homelabCA.pem
homelabCA.srl

Setting Nextcloud up to use them is going to depend on your specific web server/reverse-proxy (Apache, Nginx, Caddy, etc.) That’s outside the scope of this thread, though.

Don’t feel bad if you mess up a few times.
I certainly did.

DevDorrejo · March 3, 2024, 9:37pm

Thank you, a lot, i am using Caddy, but i saw something in the doc about how to use it.

thank you very much.