Setting up Files (High Performance Backend)

I’d like to clarify how to setup the new Files HPB written in Rust. I’m still unsure, but turning this into a wiki post so anyone can edit it. Continuing the discussion from Nextcloud Hub 21 out with up to 10x better performance, whiteboard and more collaboration features:

The HPB for Files requires a reverse-proxy setup and Redis, among other pieces of infrastructure. You can find instructions here.

docker-compose files and instructions

Please edit this with clearer instructions for setting this up.

3 Likes

And how do you configure the HPB if you have several Nextcloud installations on one server?

Must be set per installation a new service and new port?

[Unit]

Description = Push daemon for Nextcloud clients

[Service]
Environment = PORT=7867 # Change if you already have something running on this port
ExecStart = /path/to/push/binary/notify_push /path/to/nextcloud_1/config/config.php
User=www-data

[Install]
WantedBy = multi-user.target

[Unit]
Description = Push daemon for Nextcloud clients

[Service]
Environment = PORT=7868 # Change if you already have something running on this port
ExecStart = /path/to/push/binary/notify_push /path/to/nextcloud_2/config/config.php
User=www-data

[Install]
WantedBy = multi-user.target

This errors out on mine:

 Error:
   0: Failed to connect to Nextcloud database
   1: error occurred while attempting to establish a TLS connection: InvalidDNSNameError
   2: InvalidDNSNameError

I’m using the following variables:

PORT=7867 DATABASE_URL=postgres://nextcloud:XXXXXXX@127.0.0.1/nextcloud DATABASE_PREFIX=oc_ REDIS_URL=redis://127.0.0.1 NEXTCLOUD_URL=https://nextcloud.example.com

This matches with the configuration file for Nextcloud.

I’ve already tried both setting options via config.php and via environment variable declaration—same error. I’ve tried both running this directly by sudo'ing into my www-data user and via SystemD, no dice. PostgreSQL is on the standard 5432 port and all.

What’s weird about this error is that there should be absolutely no TLS or DNS involved in the connection: database is accessible only through the 127.0.0.1 loopback IP.

I’m using PostgreSQL 13.2 on Ubuntu 20.04, Redis and NGINX with Let’s Encrypt (though I doubt it’ll matter here). I don’t understand what I’m doing wrong, since I followed the README.md to the letter. Any input/advice? Troubleshooting steps? Thanks!

@alex2003super maybe try

DATABASE_URL=postgres://nextcloud:XXXXXXX@127.0.0.1/nextcloud?sslmode=disable

Haven’t tried it with this but did with something else, normally I wouldn’t advise disabling encryption but given your just connecting to the localhost leaving it on doesn’t really give you much benefit

If you connect to localhost then encryption isn’t necessary from a data in transit perspective. It may be different if an unencryted port is exposed. That may be mitigated using a local fw. However, especially for multiple instances on one server, I’d use sockets and avoid network stack at all.

How to set up HPB with NGINX + unix socket (on Arch)

NC21 introduces the Nextcloud Files High Performance Back-End (HPB) aka Client Push. HPB significantly reduces load with many connected clients, but also has advantages for instances with only a few users since notifications arrive instantly, like for Talk.

So, yes, you want to set it up although it requires a reverse proxy and Redis. (At least in NC 21) the Nextcloud app “Client Push” needs to be installed.

It is necessary to at least finalize configuration using OCC. Both, the fact that HPB shall be used instead of classic polling, and the configuration of HPB, appear to be saved to the database since the setup commands do not modify config/config.php but the push service import settings from there.

To allow access, it seems to be necessary to allow the exposed IP of the push service which is used by the proxy to access the Nextcloud. That seems to be necessary even if sockets are used. (?) To start the push service during configuration (as root) its executable needs to be installed first (as web server user).

In my setup, I got a strange error here. It’s yet unclear if that impacts operation: Currently no user complains but I don’t get files modified by other users pushed to my PC.

sudo -i -u http
cd /nextcloud/installation/folder
sed -i "s/);/  'trusted_proxies' =>\n  array (\n    0 => 'MY_IP',\n  ),\n);/" config/config.php
php occ app:install notify_push
exit

Next, the daemon should be configured and started, to access it from the NC web server. Since multiple NC instances would need different ports (and some housekeeping to map them), I prefer to use self-explanatory-named unix sockets. The notify push README.md notes that this is possible, but leaves it to the reader what is needed in case you want to follow this path. This should cover all you need for NGINX: proxy_pass

I learned the hard way that the documentation also lacks some service parameters to make sure all needed services are started prior to attempting to start notify_push – leading to just an error message “cannot access database” and a non-started service after reboot…

sudo -i
# adjust paths to suit your distro's layout
# note that there probably already exist cron .service and .timer files already
mkdir -p /run/notify_push ; chown http:http /run/notify_push
cat >/etc/systemd/system/MY_CLOUD-notify_push.service <<EOF
[Unit]
Description = Push daemon for Nextcloud clients
After=nginx.service php-fpm.service mariadb.service redis.service

[Service]
Environment=SOCKET_PATH=/run/notify_push/MY_CLOUD.sock
ExecStart=/nextcloud/installation/folder/apps/notify_push/bin/x86_64/notify_push /nextcloud/installation/folder/cloud/config/config.php
User=http
Group=http
RuntimeDirectory=notify_push
RuntimeDirectoryMode=755

[Install]
WantedBy = multi-user.target
EOF
systemctl enable --now MY_CLOUD-notify_push
  
# Testen, ob der Socket läuft
ss -xl | grep notify_push

The server { … } section of the NGINX host/site configuration (e.g. /etc/nginx/sites/MY_CLOUD.conf) should look like this if you want to use a socket:

  # Nextcloud files High Performance Back-end 'notify_push'
  location /push/ {
  proxy_pass http://unix:/run/notify_push/MY_CLOUD.sock:/;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "Upgrade";
  proxy_set_header Host $host;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

After modifying it, the NGINX config needs to be reloaded and the app must be activated and configured. You need to provide URL/push – without it, the single server config wizard would be started.

# as root
systemctl reload nginx
# as webserver user
sudo -i -u http
cd /nextcloud/installation/folder
php occ app:enable notify_push
php occ notify_push:setup https://cloud.MY_CLOUD.com/push
exit

After that, you may want to restart any sync clients. Changes to files should be pushed immediately to all connected sync clients. You may want to test that:

I have several PCs connected to the same shares. If I change something, my colleague (on Windows) gets the file immediately. If she changes some file, I (on Linux) only get them when I force sync manually. I have not found a cause yet.

Is there any write-up for Fedora / Centos using Apache?

1 Like

I have looked for Centos 7(plesk with docker) instructions for many months now(I have tried to install it several times without luck) and did not find anything. Very interested as well.

I had little trouble to setup notify_push in my docker-compose environment as I didn’t find clear instructions how the components are tied together so I decided to write all the steps together. This guide shows you how to setup High Performance Back-end for Nextcloud Files aka notify_push in Docker-compose environment with traefik proxy.

prerequisits

as prereqs you need running Nextcloud 21 with Redis (notify_push requires Redis) behind Traefik reverse proxy

environment

overview of the components. I decided to keep my individual container names in this article as it is crucial for the setup to use container names for some configs and I feel unique names easier to follow in opposite to generic hostnames like db or app. I didn’t understand this from the beginning and had hard time to find out the ways which components how depends on another one.

  • dev-nextcloud-app: NC application container
  • dev-nextcloud-redis: redis container
  • dev-nextcloud-notify_push: notify server
  • network traefik-proxy: external network used to expose services to traefik reverse proxy

install push app

docker exec --user www-data dev-nextcloud-app php occ app:install notify_push

Alternatively install the app “Client Push” with your browser.

create container for notify_push server

settings you need to adopt is

  • the image (adjust with you Nextcloud application container)
  • reverse proxy network ( traefik_proxy)
  • NEXTCLOUD_URL - use your Nextcloud container name with http:// prefix. Use the port you expose within application container. I didn’t manage notify_push to talk with public https:// address of Nextcloud (and from my point of view there is no advantage - direct communication through docker network is more direct and therefore faster)
  • ${NEXTCLOUD_FQDN} within the traefik rule - use your public FQDN in this place. Traefik uses this line to forward requests to https://${NEXTCLOUD_FQDN}/push to you notify_push server. Hint: you can expose this value through Docker .env file - so you can reuse it in all places you need this hostname
  • mount same config and apps (or custom_apps if separated) volumes you mount in your Nextcloud application container. config volume is needed to detect redis settings (host and password) custom_apps folder is where the server is installed and will be user to start the process (entrypoint:). No need to mount files

docker-compose.yaml:

services: 
  dev-nextcloud-notify_push:
    image: nextcloud:21.0 # use same image as you NC application
    container_name: dev-nextcloud-notify_push
    restart: unless-stopped
    networks:
      - traefik_proxy
      - default
    depends_on:
      - dev-nextcloud-db
      - dev-nextcloud-redis
      - dev-nextcloud-app
    volumes:
      - ./app:/var/www/html
      - ./config:/var/www/html/config:ro
    environment:
      - PORT=7867
      - NEXTCLOUD_URL=http://dev-nextcloud-app/  # NC app container name
    entrypoint: /var/www/html/custom_apps/notify_push/bin/x86_64/notify_push /var/www/html/config/config.php
    labels:
      - traefik.enable=true
      - traefik.protocol=http
      - traefik.docker.network=traefik_proxy
      - traefik.port=7867
      - traefik.http.services.dev-nextcloud_push.loadbalancer.server.port=7867
      - traefik.http.routers.dev-nextcloud_push.priority=2
      - traefik.http.routers.dev-nextcloud_push.middlewares=dev-nextcloud_strip_push
      - traefik.http.routers.dev-nextcloud_push.tls=true
      - traefik.http.routers.dev-nextcloud_push.entryPoints=web-secure
      - traefik.http.routers.dev-nextcloud_push.tls.certresolver=letsencrypt
      # necessary for the notify_push app to work:
      - traefik.http.routers.dev-nextcloud_push.rule=Host(`${NEXTCLOUD_FQDN}`) && PathPrefix(`/push`)
      - traefik.http.middlewares.dev-nextcloud_strip_push.stripprefix.prefixes=/push

one important hint:
you as both dev-nextcloud-app and dev-nextcloud-notify_push share the same domain it’s nessesary to add priorites to Traefik rules:

services: 
  dev-nextcloud-app:
  - traefik.http.routers.dev-nextcloud-app.priority=1
  dev-nextcloud-notify_push: 
  - traefik.http.routers.dev-nextcloud_push.priority=2

test notify_push container

start the new container with docker-compose up (-d if you prefer to check logs manually). If the dev-nextcloud-notify_push container doesn’t start and complains abont missing notify_push binary make sure you installed the the app first (docker exec or GUI)

verify your notify server is up and reachable from your client through Traefik (TLS works as well)

#display ip address of the user
https://${NEXTCLOUD_FQDN}/index.php/apps/notify_push/test/remote
#JSON with content "null"
https://${NEXTCLOUD_FQDN}/index.php/apps/notify_push/test/version
#some number (cookie id)
https://${NEXTCLOUD_FQDN}/index.php/apps/notify_push/test/cookie

configure push app

here you configure the Nextcloud application side to use the notify_push server component and provide the adress where this component lives. the address is https:// followed by the ${NEXTCLOUD_FQDN} followed by (/push)

docker exec --user www-data dev-nextcloud-app php occ notify_push:setup http://dev-nextcloud-notify_push
#if more logging ist needed
#docker exec --user www-data dev-nextcloud-app php occ notify_push:log 0

the output should be like this:

root@srv:~# docker exec --user www-data dev-nextcloud-app php occ notify_push:setup https://${NEXTCLOUD_FQDN}
✓ redis is configured
✓ push server is receiving redis messages
✓ push server can load mount info from database
✓ push server can connect to the Nextcloud server
✓ push server is a trusted proxy
✓ push server is running the same version as the app
  configuration saved

if there is a problem with trusted proxies

push server is not a trusted proxy, please add '172.19.0.2' to the list of trusted proxies or configure any existing reverse proxy to forward the 'x-forwarded-for' send by the push server.
  See https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/reverse_proxy_configuration.html#defining-trusted-proxies for how to set trusted proxies.
  The following trusted proxies are currently configured: "traefik, 172.16.0.0/12, 192.168.0.0/16, 10.0.0.0/8"
  The following x-forwarded-for header was received by Nextcloud: 1.2.3.4
    from the following remote: 172.19.0.2

make sure the IP address of dev-nextcloud-notify_push is configured within dev-nextcloud-app TRUSTED_PROXIES variable.

# docker name of the reverse proxy container (space separated list of fqdns/ips)
TRUSTED_PROXIES='traefik 172.16.0.0/12 192.168.0.0/16 10.0.0.0/8'

I recommend you to setup the whole 172.16.0.0/12 network (private network used by docker for internal networking) for trusted proxies - depending on your setup you may want to choose more secure configuration. Consult Admin manual > reverse proxy configurations for details

#show effective trusted_proxies from docker command line
docker exec --user www-data dev-nextcloud-app php occ config:system:get trusted_proxies
#should output something like this:
'traefik
172.16.0.0/12
192.168.0.0/16
10.0.0.0/8'

final steps

I found I need to restart my my docker application by

docker-compose down
docker-compose up -d

to make it work. Maybe disable and enable the app or restart browser session would do as well…

verification

notify_push results in changes reflected instant in the client. This is especially noticable in the Talk app - old behavior results in chat messages and incoming call are delayed by up to 15 sec (pull intervall) - once notify_push is running well and client recognize this (logout/login in browser, restart of desktop client) new messages and calls arrive immediately.

you can consult the log of dev-nextcloud-app as well

docker logs dev-nextcloud-app -n 100

without notify_push every client checks the server for updates every 15 sec… (PROPFIND request) after setup of notify_push you should not see such requests anymore (or at least much less frequent)

such PROPFIND request looks like this in client logs (similar in server logs)

[ info nextcloud.sync.accessmanager ]:	6 "PROPFIND" "https://${NEXTCLOUD_FQDN}/remote.php/dav/files/<username>/"

successful notify_push connection is visible in the client log:

2021-03-31 14:43:18:567 [ info nextcloud.sync.networkjob.jsonapi ]:	JsonApiJob of QUrl("https://${NEXTCLOUD_FQDN}/ocs/v1.php/cloud/capabilities?format=json") FINISHED WITH STATUS "OK"
2021-03-31 14:43:18:568 [ info nextcloud.sync.connectionvalidator ]:	Server capabilities QJsonObject( <<<lot of text>>> "notify_push":{"endpoints":{"pre_auth":"https://${NEXTCLOUD_FQDN}/apps/notify_push/pre_auth","websocket":"wss://${NEXTCLOUD_FQDN}/push/ws"} <<lot of text>>)
2021-03-31 14:43:18:569 [ info nextcloud.sync.account ]:	Try to setup push notifications
2021-03-31 14:43:18:569 [ info nextcloud.sync.pushnotifications ]:	Create websocket
2021-03-31 14:43:18:569 [ info nextcloud.sync.pushnotifications ]:	Open connection to websocket on: QUrl("wss://${NEXTCLOUD_FQDN}/push/ws")

if you see some issue verify the client can access the host displayed in this log line.

I hope you enjoy this guide and have much faster results then I had.

2 Likes