NC20 isn't respecting 'X-Forwarded-For' header from reverse proxy

Nextcloud version 20.0.4
Operating system and version Ubuntu 20.04
Apache or nginx version Apache 2.4.41-4ubuntu3.1
PHP version 7.4

The issue you are facing:

All logins are given a brute-force warning banner, and after checking the table (oc_bruteforce_attempts), Nextcloud is recording the IP of my reverse proxy instead of the client IP in the logs.
For reference, Nextcloud is at 192.168.5.44, the proxy (HAProxy) is 192.168.5.200.
The device I’ve used mostly for testing usually resides at 192.168.5.225.

Is this the first time you’ve seen this error? Y

Steps to replicate it:

  1. Configure Nextcloud behind HTTP-mode HAProxy with TLS termination, with option forwardfor.
  2. Attempt to login
  3. Note incorrect IP address logged when manually querying the oc_bruteforce_attempts table

The output of your Nextcloud log in Admin > Logging: (Way too many deprecation warnings to show raw)

{"reqId":"qMiy2qGh9aiB0a4XTuB0","level":2,"time":"2021-02-09T02:42:54-05:00","remoteAddr":"192.168.5.200","user":"--","app":"no app in context","method":"POST","url":"/index.php","message":"Login failed: me@my.nextcloud.instance (Remote IP: 192.168.5.200)","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.142 Safari/537.36","version":"20.0.4.0"}
{"reqId":"qMiy2qGh9aiB0a4XTuB0","level":1,"time":"2021-02-09T02:42:55-05:00","remoteAddr":"192.168.5.200","user":"--","app":"core","method":"POST","url":"/index.php","message":"Bruteforce attempt from \"192.168.5.200\" detected for action \"login\".","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.142 Safari/537.36","version":"20.0.4.0"}

The output of your config.php file in /path/to/nextcloud:

<?php
$CONFIG = array (
  'trusted_domains' => 
  array (
    0 => 'localhost',
    1 => '192.168.5.200',
    2 => 'border', // Note: this is 192.168.5.200
    3 => 'files', // Note: domain name for self
  ),
  'forwarded_for_headers' => 
  array (
    0 => 'HTTP_X_FORWARDED_FOR',
  ),
  'datadirectory' => '/mnt/ncdata',
  'dbtype' => 'pgsql',
  'version' => '20.0.4.0',
  'overwrite.cli.url' => 'https://files',
  'overwritehost' => 'files',
  'overwriteprotocol' => 'https',
  'overwritewebroot' => '/',
  'dbname' => 'nextcloud_db',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'ncadmin',
  'dbpassword' => '*****',
  'installed' => true,
  'instanceid' => 'ocado5c8khdc',
  'log_type' => 'file',
  'logfile' => '/var/log/nextcloud/nextcloud.log',
  'loglevel' => 0,
  'log.condition' => 
  array (
    'apps' => 
    array (
      0 => 'admin_audit',
    ),
  ),
  'mail_smtpmode' => 'smtp',
  'remember_login_cookie_lifetime' => '1800',
  'log_rotate_size' => '10485760',
  'trashbin_retention_obligation' => 'auto, 180',
  'versions_retention_obligation' => 'auto, 365',
  'simpleSignUpLink.shown' => false,
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'filelocking.enabled' => true,
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => '/var/run/redis/redis-server.sock',
    'port' => 0,
    'timeout' => 0.5,
    'dbindex' => 0,
    'password' => '*****',
  ),
  'logtimezone' => 'America/Indiana/Indianapolis',
  'htaccess.RewriteBase' => '/nextcloud',
  'app_install_overwrite' => 
  array (
    0 => 'joplin',
    1 => 'uploaddetails',
    2 => 'dropit',
    3 => 'breezedark',
  ),
  'ldapIgnoreNamingRules' => false,
  'ldapProviderFactory' => 'OCA\\User_LDAP\\LDAPProviderFactory',
  'updater.release.channel' => 'stable',
);

The output of your Apache/nginx/system log in /var/log/apache2:

files:80 192.168.5.200 - - [09/Feb/2021:02:42:53 -0500] "POST /login HTTP/1.1" 303 804 "https://files/login?redirect_url=/settings/admin/logging" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.142 Safari/537.36"
files:80 192.168.5.200 - - [09/Feb/2021:02:42:55 -0500] "GET /login?redirect_url=/settings/admin/logging&user=me@my.nextcloud.instance HTTP/1.1" 200 7055 "https://files/login?redirect_url=/settings/admin/logging" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.142 Safari/537.36"

Additional notes:

Captured request is showing that it’s sending the header (check the bottom):

GET /login?redirect_url=/settings/admin/logging&user=me@my.nextcloud.instance HTTP/1.1
cache-control: max-age=0
sec-ch-ua: "Chromium";v="87", " Not;A Brand";v="99"
sec-ch-ua-mobile: ?0
dnt: 1
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.142 Safari/537.36
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
sec-fetch-site: same-origin
sec-fetch-mode: navigate
sec-fetch-user: ?1
sec-fetch-dest: document
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
host: files
x-forwarded-for: 192.168.5.225

The only strange thing I see about that request is that HTTP/1 headers are usually capitalized, such as X-Forwarded-For, but since the front-end connection is HTTP/2, in which headers are lowercase, HAProxy isn’t trying to case-convert, but I don’t think case should make a difference?

2 Likes

With NGINX check out these configuration items:
http://nginx.org/en/docs/http/ngx_http_realip_module.html

Given that I… do not use NGINX…

Not a true solution to the problem, but Apache can handle this itself, like NGINX:

Version 2.4.30 and up:

Enable the mod_remoteip module, and if you’re using a header, like X-Forwarded-For, in your VirtualHost, add these two lines:

RemoteIPInternalProxy 192.168.5.200 # Replace with your proxy's IP address
RemoteIPHeader X-Forward-For # Replace with your header, like 'X-Forward-For' or 'X-Client-IP'

If using something that supports the HAProxy PROXY protocol, instead, just add one line:

RemoteIPProxyProtocol On

This will now require the PROXY protocol be present on all connections, except those from addresses listed in RemoteIPProxyProtocolExceptions.

If below version 2.4.30:

Before v2.4.30, the PROXY protocol functionality was in a separate module, mod_proxy_protocol, and contains only one documented directive. After 2.4.30, this was merged into mod_remoteip as shown above.

Therefore, if you’re running Apache that’s below 2.4.30, then enable mod_proxy_protocol and add this to your VirtualHost:

ProxyProtocol On

Once again, this will require the PROXY protocol from upstream, but there is no directive for exemptions. If you need that, well… update Apache.

Given that i don’t use Apache…, I did the best I could: point to relevant info for the system I use.
Which may help to get it straightened out for apache along the same lines.

Also others that do deploy nginx will probably run into the same problem.
may benefit from it as well. Your issue was one of the best hits on the issue.

True, true. And, in fairness, it did, since it made me go on a little Googling trip to see if Apache had something similar (mod_remoteip), I just don’t like using the webserver side as a workaround for what should be a perfectly valid feature in Nextcloud itself. But in either case, your instructions should work for NGINX (even though the linked article seems a little out of date, the NGINX build in my repositories for all my server containers comes with everything there pre-built), and said instructions made me dig out similar instructions for Apache, so in theory that’s the end result fixed…

Hi there,

I got a similiar issue and tried a lot of tweeking. But in the end it helped adding my reverse proxy as trusted proxy in the config.php:

‘trusted_proxies’ => ‘IP.OF.THE.PROXY’,

Greetings

I must have completely skipped over that when reading the documentation, I probably thought it was referring to trusted_domains instead of trusted_proxies. But that seems to be the fix.

this didn’t help me. I’ve the following config where 10.1.3.11 is my reverse proxy:

  'trusted_proxies' => '10.1.3.11',
  'forwarded_for_headers' => array(
	0 => 'HTTP_X_FORWARDED_FOR',
  ),

In the end I get something like this from nextcloud:


And this is the webserver log of my nextcloud webserver:

The webserver get’s the http_x-forwarded-for header but nextcloud doesn’t process it.

my fault, needed to set trusted_proxies as an array. Opened an issue for that proxy header warning disappear when trusted_proxies is set to an invalid value · Issue #26256 · nextcloud/server · GitHub

I am not sure why you think this is an “issue”. It is documented accordingly, including an example at the end of the chapter.

https://docs.nextcloud.com/server/21/admin_manual/configuration_server/reverse_proxy_configuration.html

The Trusted Proxy can be a single proxy without further additional values for forwarded headers etc. Therefore it is correct that the warning disappears, as soon as a trusted proxy value is added to the config.php, and you are accesing your Nextcloud via this proxy. The additional values ​​for x-forwarded headers etc. are optional and can only work in combination with one or multiple trusted proxy values. Therfore they must be in an array with the associated trusted proxy values.

Yeah but it has not to be declared like

 'trusted_proxies' => '10.1.3.11',

Instead it must then be declared like

  'trusted_proxies' => array(
	0 => '10.1.3.11',
  ),

Otherwise only the warning will disappear but NO evaluation of x-forwarded-for is done.
I cannot say “it’s everything okay” on the admin page but ignoring an invalid value at the same time. That’s killing UX

Ah ok. If that’s the case, the warning would indeed be pointless. I assumed that a single “trusted proxy” entry would work without putting it in an array. But you are probably right and it’s not.

1 Like

Is the issue confusing? Should i clearify something?

No. I think it’s fine. As i said i drew wrong conclusions from your post, and should have read the issue more carefully.

Okay. But I would appreciate an upvote of the issue then I am not so alone in the fight against the powerful, evil stale-bot from the dark side of the internet

Can you please share your HAProxy configuration?