High Performance File Backend - Trusted Proxy issues

Hi everybody,

first of all: i’m running my Nextcloud instance behind an nginx reverse proxy.
I’d like to use the push_notify app in order to increase performance on mobile devices. I already set everything up according to the github readme.

At the last step, i tried to activate my config by running the setup with my configured url
occ notify_push:setup https://cloud.*censored*/push
with resulted in the following error message

push server is not a trusted proxy, please add '192.168.178.131' to the list of trusted proxies or configure any existing reverse proxy to forward the 'x-forwarded-for' send by the push server.
  See https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/reverse_proxy_configuration.html#defining-trusted-proxies for how to set trusted proxies.
  The following trusted proxies are currently configured: 
  The following x-forwarded-for header was received by Nextcloud: *censored*
    from the following remote: 192.168.178.131

even though i configured my proxy properly (config/config.php):

'trusted_proxies' =>
  array (
    0 => '192.168.178.131'
  ),

It seems like the HPFB-App or at least the setup doesn’t parse the config correctly, or is there one step that im missing out?

Thank you and kind regards,
Peter

Hey,
can you post your full config.php? (Remove anything sensitive of course)

Sure:

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'instanceid' => '*censored*',
  'passwordsalt' => '*censored*',
  'secret' => '*censored*',
  'trusted_domains' => 
  array (
    0 => 'cloud.*censored*',
    1 => '192.168.178.115',
    2 => 'localhost',
  ),
  'datadirectory' => '/home/data',
  'dbtype' => 'pgsql',
  'version' => '23.0.5.1',
  'overwritehost' => 'cloud.*censored*',
  'overwriteprotocol' => 'https',
  'overwrite.cli.url' => 'http://cloud.*censored*',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost:5432',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'oc_root',
  'dbpassword' => '*censored*',
  'installed' => true,
  'maintenance' => false,
  'loglevel' => 2,
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'ssl',
  'mail_sendmailmode' => 'smtp',
  'mail_from_address' => 'cloud',
  'mail_domain' => '*censored*',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtpauth' => 1,
  'mail_smtphost' => '*censored*',
  'mail_smtpport' => '465',
  'mail_smtpname' => '*censored*',
  'mail_smtppassword' => '*censored*',
  'mail_smtpstreamoptions' => 
  array (
    'ssl' => 
    array (
      'allow_self_signed' => true,
      'verify_peer' => false,
      'verify_peer_name' => false,
    ),
  ),
  'default_phone_region' => 'DE',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => '/run/redis/redis-server.sock',
    'port' => 0,
    'dbindex' => 0,
    'password' => '*censored*',
    'timeout' => 1.5,
  ),
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'preview_max_memory' => 1280,
  'trusted_proxies' => 
  array (
    0 => '192.168.178.131'
  ),
);

Not sure why trusted_proxies is not correctly recognized. The SelfTest is just using the NC functions to retrieve this value. Which NC version are you running btw?

What happens if you access https:///push/test/remote/1.2.3.4 from a browser?

I am running Nextcloud Hub II (23.0.5). The URL returns 192.168.178.131 (which is expected i guess?).

I looked at the code on GitHub and found that the test function even has an optional flag to ignore any proxy errors, but this is accessable via a flag or smth.

If correctly configured, it should return 1.2.3.4. So something is wrong with the trusted_proxies I guess. If it returns 1.2.3.4 the proxy check will succeed.

What happens if you get the config via occ? occ config:list system --private ?
Are you running in docker?

X-Forwarded-For Is correctly setup and working ?

I am running natively without docker in a LxC container within Proxmox.

Here is the Output of occ config:list system --private:

{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "instanceid": "*censored*",
        "passwordsalt": "*censored*",
        "secret": "*censored*",
        "trusted_domains": [
            "cloud.*censored*",
            "192.168.178.115",
            "localhost"
        ],
        "datadirectory": "\/home\/data",
        "dbtype": "pgsql",
        "version": "23.0.5.1",
        "overwritehost": "cloud.*censored*",
        "overwriteprotocol": "https",
        "overwrite.cli.url": "http:\/\/*censored*",
        "dbname": "nextcloud",
        "dbhost": "localhost:5432",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "oc_root",
        "dbpassword": "*censored*",
        "installed": true,
        "maintenance": false,
        "loglevel": 2,
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "ssl",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "cloud",
        "mail_domain": "*censored*",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpauth": 1,
        "mail_smtphost": "*censored*",
        "mail_smtpport": "465",
        "mail_smtpname": "*censored*",
        "mail_smtppassword": "*censored*",
        "mail_smtpstreamoptions": {
            "ssl": {
                "allow_self_signed": true,
                "verify_peer": false,
                "verify_peer_name": false
            }
        },
        "default_phone_region": "DE",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "\/run\/redis\/redis-server.sock",
            "port": 0,
            "dbindex": 0,
            "password": "secret",
            "timeout": 1.5
        },
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "preview_max_memory": 1280
    }
}

So, no trusted proxies… but why?

As for my proxy, i use NginxProxyManager with the following advanced flags:

proxy_hide_header Upgrade;

rewrite ^/\.well-known/carddav https://$server_name/remote.php/dav/ redirect;
rewrite ^/\.well-known/caldav https://$server_name/remote.php/dav/ redirect;

proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;

I tried to log with the Apache Log Forensic mod, an example output is

GET /settings/ajax/checksetup HTTP/1.1|Host:cloud.*censored*|X-Forwarded-Scheme:https|X-Forwarded-Proto:https|X-Forwarded-For:*censored, but my public ip, not 192.168.178.131*|X-Real-IP:*censored*|sec-ch-ua:" Not A;Brand";v="99", "Chromium";v="102", "Google Chrome";v="102"|requesttoken:*censored*=|sec-ch-ua-mobile:?0|user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36|ocs-apirequest:true|accept:*/*|x-requested-with:XMLHttpRequest|sec-ch-ua-platform:"Linux"|sec-fetch-site:same-origin|sec-fetch-mode:cors|sec-fetch-dest:empty|accept-encoding:gzip, deflate, br|accept-language:de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7|cookie:__Host-nc_sameSiteCookielax=true; __Host-nc_sameSiteCookiestrict=true; nc_username=root; oc_sessionPassphrase=*censored*;nc_token=*censored*; nc_session_id=*censored*

Hm. Any caching involved? What happens after restart Apache/Nginx or PHP-FPM and then readoing the occ command?

I restarted everything, sadly no change in the result…
Any idea why the trusted_proxies-Array isn’t recognized?

Side-note: I tried to run the setup-command without specifying the URL (so it used the automatic setup), which also failed.
But it overwrote the trusted_proxies-Array with

'trusted_proxies' => 
  array (
    0 => '127.0.0.1',
    1 => '::1',
  ),

and the Command-Output suggests that this was successfull.

root@ctNextcloud:/var/www/html# sudo -u www-data php occ notify_push:setup           
This setup wizard is intended for use on single server instances
where the nextcloud server, web server/reverse proxy and push daemon all run on the same machine.
If your setup is more complex or involves any kind of load balancing
you should follow the manual setup instruction on the README instead
https://github.com/nextcloud/notify_push
Press enter to continue or ESC to cancel...

Push binary seems to be running already
🗴 failed to run self-test.
test output: ✓ redis is configured
             🗴 using unencrypted http for push server is strongly discouraged
             🗴 push server url is set to localhost, the push server will not be reachable from other machines
             ✓ push server is receiving redis messages
             ✓ push server can load mount info from database
             ✓ push server can connect to the Nextcloud server
             🗴 push server is not a trusted proxy, please add '192.168.178.131' to the list of trusted proxies or configure any existing reverse proxy to forward the 'x-forwarded-for' send by the push server.
               See https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/reverse_proxy_configuration.html#defining-trusted-proxies for how to set trusted proxies.
               The following trusted proxies are currently configured: "127.0.0.1", "::1"
               The following x-forwarded-for header was received by Nextcloud: *censored, my public ip*
                 from the following remote: 192.168.178.131
             
               If you're having issues getting the trusted proxy setup working, you can try bypassing any existing reverse proxy
               in your setup by setting the `NEXTCLOUD_URL` environment variable to point directly to the internal Nextcloud webserver url
               (You will still need the ip address of the push server added as trusted proxy)
             
  See the steps in the README for manual setup instructions: https://github.com/nextcloud/notify_push

But if i run occ config:list system --private the trusted proxies are not recognized either.

Just to make sure that this is not a nextcloud bug i spinned up another instance of the same version (this time in docker) and tested the trusted_proxies-Array. In this instance it worked flawlessly and the proxy was recognized by occ config:list system --private, so something in my instance is broken, i just have no idea what it could be…

Maybe try to manually add the key by typing? To rule out any copy and paste issue?! And move it to the top of the config?