Why is my remote IP `127.0.0.1`?

The Basics

  • Nextcloud Server version (e.g., 29.x.x):
    • 31.0.5
  • Operating system and version (e.g., Ubuntu 24.04):
    • Debian 12
  • Web server and version (e.g, Apache 2.4.25):
    • Apache/2.4.63
  • Reverse proxy and version _(e.g. nginx 1.27.2)
    • replace me
  • PHP version (e.g, 8.3):
    • 8.3.21
  • Is this the first time you’ve seen this error? (Yes / No):
    • Yes
  • When did this problem seem to first start?
    • After configuring trusted_proxies with my reverse proxy’s IP
  • Installation method (e.g. AlO, NCP, Bare Metal/Archive, etc.)
    • Nextcloud AIO 11.0
  • Are you using CloudfIare, mod_security, or similar? (Yes / No)
    • No

Summary of the issue you are facing:

/settings/admin/security and that logs are showing my remote IP as 127.0.0.1 leading to some throttling.

Before it showed the Tailnet IP of my reverse proxy, so I configured trusted_proxies 2 to include that IP per ip.address.that.is.shown in the all-in-one issue#2045:

asavage@ifrit:~/docker/nextcloud$ sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ config:system:get trusted_proxies 
127.0.0.1
::1
172.19.0.0/16
100.125.169.125

I have a Caddy reverse proxy on a separate server:

https://nc.redacteddomain.me:443 {
	reverse_proxy 100.113.236.81:11000
}

IPV6 is disabled everywhere.

I don’t think Caddy is misconfigured because my Jellyfin server uses the same reverse proxy and detects the client IP correctly.

Is there something to configure in Apache to debug this further? I can’t edit the config file there because it’s on a read-only filesystem in Nextcloud AIO

Steps to replicate it (hint: details matter!):

Log entries

Nextcloud

Example log entry showing the wrong remote IP

Web server / Reverse Proxy

Reverse proxy detects the remote_ip correctly:

The output of your Apache/nginx/system log in /var/log/____:
docker log nextcloud-aio-apache:

I’m not sure how to get the access logs of the AIO Apache with headers. If anyone knows that would help debug things.

Configuration

Nextcloud

Apps

The output of occ app:list (if possible).

Caddy 2.6.2

Sorry I can’t edit the post. I get an error saying new users can’t post more than 4 links.

I used tcpdump to inspect the traffic on the nextcloud-aio network.

I see requests going from my reverse proxy to the http server with this:

X-Forwarded-For: 98.97.25.71
X-Forwarded-Host: nc.redacteddomain.me
X-Forwarded-Proto: https

So it looks like the headers are correctly being set because 98.97.25.71 was my correct IP at the time.

Just tried a fully clean install and I get the same behavior.

Hello @asavageiv,

welcome to the Nextcloud community! :handshake:

please review 101: reverse proxy
aio has some specialties as well - see AiO reverse proxy

Before spending more time on this, consider upgrading Caddy. This is a very old release with known bugs (See Releases).

Can you post your Compose file?

Also see On the same server in a Docker container (you’ll have to expand it to see the different scenarios).

At least I’m not misunderstanding your topology.

What is the below IP address? Are you not using the Docker IP for some reason?

Thanks @wwe and @jtr.

I think I figured out the problem. Nextcloud AIO has its own Caddy in the nextcloud-aio-apache container that acts as a reverse proxy to all the Nextcloud servers within the docker network. That Caddy has its trusted_proxies configured independently from the Nextcloud configuration in all-in-one/Containers/apache/start.sh

I was able to use the shell in nextcloud-aio-apache to modify the Caddyfile to include my reverse proxy’s internal IP, reload that config, and see that the problem goes away.

I think that when the reverse proxy is running on a different server, we need a way to configure trusted_proxies for both Nextcloud server and the nextcloud-aio-apache Caddy instance. Alternatively, I can insert another reverse proxy into the same docker network that trusts my reverse proxy’s IP into the chain, but I don’t like that solution much.

I also posted about it in a question on the all-in-one github

@jtr thanks for the pointer about the old Caddy. It is the default in the Debian 12 bookworm repository and isn’t even going to be upgraded much in Debian 13! I will look into upgrading it.

To reiterate a bit. My topology is on a different server from the reverse-proxy documentation.

1 Like

@szaimen Any thoughts on adding support for TRUSTED_PROXIES? I can do the leg-work. I see it was explicitly unsupported in add check for unsupported environmental variables by szaimen · Pull Request #3154 · nextcloud/all-in-one · GitHub a couple years ago, but I don’t see the reasoning in the PR. Perhaps things have changed.

Here is my compose.yaml for the record:

In AIO it’s rarely been necessary I think because of this bit of code:

The only reason I guess you encountered it is because your (non-AIO provided) RP is using a public IP address to inter-connect.

1 Like

Yes, I guess this is the reason and is likely a security issue as the traffic is then sent unencrypted over the public internet…

My RP is a VPS proxying from the public internet to my AIO instance on a machine on my Tailnet. No traffic is sent over the public internet in the clear. I did it this way because my ISP uses CGNAT and I wanted to expose my instance to the internet and my VPS is not beefy enough to run AIO.

100.113.236.81 is the Tailnet address of the machine AIO is running on.

I managed a workaround by enabling the caddy container in compose.yaml and doing a chained reverse proxy. My config is now this:

compose.yaml:

services:
  nextcloud-aio-mastercontainer:
    ... snip ...
    ports:
      - 8080:8080
    environment: # Is needed when using any of the options below
      APACHE_PORT: 11000
      APACHE_IP_BINDING: 0.0.0.0
  caddy:
    image: caddy:latest
    restart: always
    container_name: caddy
    volumes:
      - caddy_certs:/certs
      - caddy_config:/config
      - caddy_data:/data
      - caddy_sites:/srv
    network_mode: "host"
    configs:
      - source: Caddyfile
        target: /etc/caddy/Caddyfile
configs:
  Caddyfile:
    content: |
      {
        servers {
          trusted_proxies static private_ranges 100.125.169.125 # Tailnet IP of the RP
        }
      }
      http://nc.redacted.me:11001 {
        log
        reverse_proxy 127.0.0.1:11000
      }

RP:

https://nc.redacted.me:443 {
        reverse_proxy http://100.113.236.81:11001 # Tailnet IP of 
}

The only issues now are:

  • The WOPI requests are identified as coming from the dynamic public IP address of the AIO server, so I can’t whitelist it properly permanently. Not sure what to do about this.
  • HTTP between the RP and AIO caddy means worse performance. I can probably fix this if I use ACME to get a cert for the AIO caddy.

With a bit of hacking I was able to adjust AIO to allow the ADDITIONAL_TRUSTED_PROXY param and pass it to the nextcloud-aio-apache Caddy, which also solves the problem, but I think still has the WOPI issue.

1 Like

Ok, the WOPI issue is solved by configuring Collabora and Nextcloud to use the docker network for internal communication instead of going back to the VPS via the full domain name, so the default allow list works.

To do that I had to make the following changes.

# file: php/containers.json change nextcloud-aio-collabora
  # env
  aliasgroup1=https://%NC_DOMAIN%:443,http://nextcloud-aio-apache:%APACHE_PORT%

  # nextcloud_exec_commands
  php /var/www/html/occ richdocuments:activate-config --wopi-url='http://nextcloud-aio-collabora:9980' --callback-url='http://nextcloud-aio-apache:11000'

# file: Containers/apache/Caddyfile
https://{$ADDITIONAL_TRUSTED_DOMAIN}:443,
http://nextcloud-aio-apache:{$APACHE_PORT},
{$PROTOCOL}://{$NC_DOMAIN}:{$APACHE_PORT} {

To get those to work we also need multi-substitution support for envs in containers.json which I implemented here:

Does this approach of configuring richdocuments so the internal communication stays internal make sense? Would it be accepted as a change to AIO?

This topic was automatically closed 8 days after the last reply. New replies are no longer allowed.