VPN / Proxy Configuration

Iaens · June 23, 2023, 6:34pm

Hi!

I’m currently in the process of installing Nextcloud AIO, but I have a few roadblocks in my journey.
I’m using a Debian 11 server. I’m following the documentation (reverse-proxy docker compose), but I can’t get it to work.

I’m using Apache2 as my reverse proxy, installed on the host.
Here is the conf : (note I’ve replace my domain name with *)

<VirtualHost *:80>
    ServerName ******

    ErrorLog /var/log/apache2/nextcloud-error.log
    CustomLog /var/log/apache2/nextcloud-access.log combined

    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
    RewriteCond %{SERVER_NAME} ^****\.****\.****$
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

<VirtualHost *:443>
    ServerName *****

    # Reverse proxy based on https://httpd.apache.org/docs/current/mod/mod_proxy_wstunnel.html
    RewriteEngine On
    ProxyPreserveHost On
    AllowEncodedSlashes NoDecode

    ProxyPass / http://localhost:11000/ nocanon
    ProxyPassReverse / http://localhost:11000/

    RewriteCond %{HTTP:Upgrade} websocket [NC]
    RewriteCond %{HTTP:Connection} upgrade [NC]
    RewriteCond %{THE_REQUEST} "^[a-zA-Z]+ /(.*) HTTP/\d+(\.\d+)?$"
    RewriteRule .? "ws://localhost:11000/%1" [P,L]

    # Enable h2, h2c and http1.1
    Protocols h2 h2c http/1.1

    # Solves slow upload speeds caused by http2
    H2WindowSize 5242880

    # SSL
    SSLEngine on
    SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
    SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
    SSLHonorCipherOrder off
    SSLSessionTickets off
    SSLCertificateFile /etc/apache2/ssl/****/fullchain.cer
    SSLCertificateKeyFile /etc/apache2/ssl/*****/***.key
    Protocols h2 http/1.1
    Header always set Strict-Transport-Security "max-age=31536000; includeSubDomains;"

    # Disable HTTP TRACE method.
    TraceEnable off
    <Files ".ht*">
        Require all denied
    </Files>

    # Support big file uploads
    LimitRequestBody 0
</VirtualHost>

And here is my docker compose file :

services:
  nextcloud:
    image: nextcloud/all-in-one:latest
    restart: always
    container_name: nextcloud-aio-mastercontainer
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config
      - /var/run/docker.sock:/var/run/docker.sock:ro
    ports:
      - 8080:8080
    environment:
      - APACHE_PORT=11000
      - APACHE_IP_BINDING=127.0.0.1
      - NEXTCLOUD_DATADIR=/data/nextcloud
      - NEXTCLOUD_UPLOAD_LIMIT=10G
      - NEXTCLOUD_MAX_TIME=3600
      - NEXTCLOUD_MEMORY_LIMIT=1024M

volumes:
  nextcloud_aio_mastercontainer:
    name: nextcloud_aio_mastercontainer

When executing the docker compose configuration, the container is setup without errors.
After switching the apache configuration, I get a 503 error (but I believe it’s to be expected as I’ve not set up anything yet).
When I try connecting to the admin panel (@:8080), nextcloud is using a self signed certificate, blocking the connection in all browsers. I can connect directly with the public ip:8080, but then it fails the port checking test, even tho they are open of portchecker.

I assume there must be something wrong with my apache configuration, but I couldn’t figure out what.
I’ve tried the recommended steps, but without luck.

Thanks in advance!
Regards,
Iaens

szaimen · June 23, 2023, 7:22pm

Yes, you need to click on details and then accept the self-signed certifivate in order to actually visit the site and then log in in order to enter the domain.

Iaens · June 23, 2023, 7:37pm

Actually I couldn’t add it when browsing to the domain:8080. But yes I did add the exception for the IP:8080.

szaimen · June 23, 2023, 9:37pm

All right. In that case you might want to skip the domain validation GitHub - nextcloud/all-in-one: Nextcloud AIO stands for Nextcloud All-in-One and provides easy deployment and maintenance with most features included in this one Nextcloud instance.

Iaens · June 23, 2023, 10:19pm

I did finished the install via this method.
However the Nextcloud container is not behaving normally.
I still can’t connect to the instance. There is no errors in the nextcloud logs :

docker compose up
[+] Building 0.0s (0/0)                                                                                                                                                                                                                                                       
[+] Running 1/1
 ✔ Container nextcloud-aio-mastercontainer  Recreated                                                                                                                                                                                                                    0.2s 
Attaching to nextcloud-aio-mastercontainer
nextcloud-aio-mastercontainer  | Trying to fix docker.sock permissions internally...
nextcloud-aio-mastercontainer  | Creating docker group internally with id 997
nextcloud-aio-mastercontainer  | Initial startup of Nextcloud All-in-One complete!
nextcloud-aio-mastercontainer  | You should be able to open the Nextcloud AIO Interface now on port 8080 of this server!
nextcloud-aio-mastercontainer  | E.g. https://internal.ip.of.this.server:8080
nextcloud-aio-mastercontainer  | 
nextcloud-aio-mastercontainer  | If your server has port 80 and 8443 open and you point a domain to your server, you can get a valid certificate automatically by opening the Nextcloud AIO Interface via:
nextcloud-aio-mastercontainer  | https://your-domain-that-points-to-this-server.tld:8443
nextcloud-aio-mastercontainer  | {"level":"info","ts":1687558235.145728,"msg":"using provided configuration","config_file":"/Caddyfile","config_adapter":""}
nextcloud-aio-mastercontainer  | {"level":"info","ts":1687558235.1585228,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Receive-Buffer-Size for details."}
nextcloud-aio-mastercontainer  | [23-Jun-2023 22:10:35] NOTICE: fpm is running, pid 118
nextcloud-aio-mastercontainer  | [23-Jun-2023 22:10:35] NOTICE: ready to handle connections

But all connections timeout in apache and directly from the host :

curl http://localhost:11000 -vvv
*   Trying ::1:11000...
* connect to ::1 port 11000 failed: Connection refused
*   Trying 127.0.0.1:11000...
* Connected to localhost (127.0.0.1) port 11000 (#0)
> GET / HTTP/1.1
> Host: localhost:11000
> User-Agent: curl/7.74.0
> Accept: */*
> 
.... just hangs

I tried removing the APACHE_IP_BINGING but it did nothing.
I copy paste the exact apache config from all-in-one/reverse-proxy.md at main · nextcloud/all-in-one · GitHub, but still get the same result.

szaimen · June 23, 2023, 10:40pm

Hm… Can you post the output of sudo netstat -tulpn ? Also can you post the outout of sudo docker logs nextcloud-aio-apache ?

Iaens · June 24, 2023, 7:26am

The nextcloud-aio-apache logs are almost empty :

...
Waiting for Nextcloud to start...
{"level":"info","ts":1687557379.0413027,"msg":"using provided configuration","config_file":"/Caddyfile","config_adapter":""}
[mpm_event:notice] [pid 408:tid 140712161860424] AH00489: Apache/2.4.57 (Unix) configured -- resuming normal operations
[core:notice] [pid 408:tid 140712161860424] AH00094: Command line: '/usr/local/apache2/bin/httpd -D FOREGROUND'

But the most problematic thing is that there is a port 11000 opened :

tcp        0      0 0.0.0.0:11235           0.0.0.0:*               LISTEN      682/sshd: /usr/sbin 
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1/init              
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      11035/docker-proxy  
tcp        0      0 0.0.0.0:3478            0.0.0.0:*               LISTEN      30505/docker-proxy  
tcp        0      0 127.0.0.1:11000         0.0.0.0:*               LISTEN      32103/docker-proxy  
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1011/exim4          
tcp6       0      0 :::443                  :::*                    LISTEN      868/apache2         
tcp6       0      0 :::11235                :::*                    LISTEN      682/sshd: /usr/sbin 
tcp6       0      0 :::111                  :::*                    LISTEN      1/init              
tcp6       0      0 :::8080                 :::*                    LISTEN      11043/docker-proxy  
tcp6       0      0 :::80                   :::*                    LISTEN      868/apache2         
tcp6       0      0 :::3478                 :::*                    LISTEN      30513/docker-proxy  
tcp6       0      0 ::1:25                  :::*                    LISTEN      1011/exim4          
udp        0      0 0.0.0.0:3478            0.0.0.0:*                           30527/docker-proxy  
udp        0      0 0.0.0.0:65357           0.0.0.0:*                           654/openvpn         
udp        0      0 0.0.0.0:111             0.0.0.0:*                           1/init              
udp        0      0 172.18.0.1:123          0.0.0.0:*                           750/ntpd            
udp        0      0 172.19.0.1:123          0.0.0.0:*                           750/ntpd                       
udp        0      0 127.0.0.1:123           0.0.0.0:*                           750/ntpd            
udp        0      0 0.0.0.0:123             0.0.0.0:*                           750/ntpd

But I still can’t connect. The AIO control panel works.
I get a 502 error :

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request

Reason: Error reading from remote server

The apache logs :

[proxy_http:error] [pid 17038] (104)Connection reset by peer: [client ******] AH01102: error reading status line from remote server localhost:11000
[proxy:error] [pid 17038] [client ******] AH00898: Error reading from remote server returned by /

All containers are up and running (according to the control panel). The curl command still hangs.
My apache configuration is the one from the doc, plus the change to the SSL part and the domain name.
My docker compose file is the same as in the 1st post, minus the IP_BINDING that I changed to 0.0.0.0, and the - SKIP_DOMAIN_VALIDATION=true.

Iaens · June 25, 2023, 12:27pm

I’ve tried with NGINX (1.18) and I get the same result. It seems that there is something bad with the docker configuration.
I really don’t understand …
I’ve reinstalled Nextcloud AIO (prune the last version), I use the NGINX config from the doc : “This config was tested and should normally work”.
Same as before. It just don’t work. I can’t connect on localhost:11000, despite having the port opened : tcp 0 0 127.0.0.1:11000 0.0.0.0:* LISTEN 30332/docker-proxy

If anyone can help I would be really grateful!

szaimen · June 25, 2023, 12:28pm

I have one idea: is there maybe a VPN running on the same host or are you connected to a VPN?

Iaens · June 25, 2023, 12:33pm

Yes I’m running openvpn to remotely connect to the machine. I’ve tried changing the ports (eg 11123) but it didn’t solve it.
Here is the nginx error :

[error] 32318#32318: *28 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.150.163, server: ******, request: "GET / HTTP/2.0", upstream: "http://127.0.0.1:11123/", host: "******"

szaimen · June 25, 2023, 12:34pm

Can you remove the VPN from the chain and check if that helps?

Iaens · June 25, 2023, 12:36pm

As connect to the site without the VPN? I’ve tried with my phone and I couldn’t connect.

szaimen · June 25, 2023, 12:36pm

No, I meant stop the VPN service that runs on the same host.

Iaens · June 25, 2023, 12:37pm

I need it to SSH into the machine. It’s not directly accessible over the internet (the SSH port).

szaimen · June 25, 2023, 12:38pm

Now we get closer to the problem. So you have some forwarding from one machine to another in place to make it accessible publicly?

Iaens · June 25, 2023, 12:47pm

Yes I use a VM as a gateway and VPN access. The VM I’m trying to install nextcloud-aio on isn’t directly connected to the internet, but the ports are forwarded.
I can access the ‘private VM’ via the domain name (forwarding). I just get an error 502 with nginx.
I need to connect to the gateway (VPN) to SSH into the VM.

szaimen · June 25, 2023, 12:49pm

I see. Thanks for the clarification!

I fear something is wrong with the forwarding or VPN then which blocks AIO from working correctly. Unfortunately I am not an expert in forwarding and VPNs so I cannot help you with that. I would probably suggest to ask for help with this in a linux or openvpn forum.

Iaens · June 25, 2023, 12:52pm

Thanks for pointing me in the right direction!
I hadn’t thought of that as I was using my usual VM config, but maybe I need to change something there.
I was fully focused on the reverse proxy/nextcloud configuration.
I’ll let you know as soon as I find out.