Placing Nextcloud behind HAProxy with SSL Passthrough

I was setting up a server for the company I work at that required both a Wordpress website as well as Nextcloud. Now of course, these services require much less thinking if you leave them on their native ports 80 and 443, and you donā€™t have to tell your employees to go to port 8443 to visit the company cloud! :stuck_out_tongue: That meant my solution was to do a reverse proxy, and I chose to do HAProxy. This is a good use case if you want to run separate services on different machines on the local network, VMs or containers. My rig is set up as follows:

OS: Ubuntu Server 16.04
LXD containers inside a ZFS pool
Here are my 3 containers:

  • Ubuntu 16.04 container using LAMP and Wordpress
  • Ubuntu 16.04 container using LAMP and Nextcloud
  • Ubuntu 16.04 container with just HAProxy

Install your SSL certificates on your Nextcloud and other machines (if you have them) to allow HAProxy to pass the SSL traffic to the server. There is an SSL Termination configuration available too, but these configurations only focus on the pass through configuration.

If you donā€™t know much about reverse proxies (like me), be sure to forward ports 80 and 443 on your router to your HAProxy containerā€™s IP address since that machine will be handling the incoming traffic from the web and passing it to the backend. This configuration also utilizes the TLS SNI feature so specifying the ServerName in your apache (or nginx or whatever youā€™re using) configuration files.

In my case I am using Wordpress and Nextcloud, but you can use whatever you want following the template of these configurations.

global
log /dev/log    local0
log /dev/log    local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin
stats timeout 30s
maxconn 4096
user haproxy
group haproxy
daemon


defaults
log     global
mode    tcp
option  tcplog
option  dontlognull
timeout connect 15s
timeout client  15s
timeout server  15s
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http


frontend localhost80  #Front end for port 80, does a redirect to port 443
bind *:80
mode http
redirect scheme https code 301 if !{ ssl_fc }

frontend localhost443   #Listens on port 443
bind *:443
option tcplog
mode tcp

acl tls req.ssl_hello_type 1

tcp-request inspect-delay 5s
tcp-request content accept if tls

acl is_wordpress req.ssl_sni -i domain1.com   # ACL specifying domain1
acl is_nextcloud req.ssl_sni -i www.domain2.com                # ACL specifying domain2
acl is_nextcloud2 req.ssl_sni -i domain2.com   # ACL specifying www.domain2

# I had to make a separate ACL for www.domain2.com because that's my nextcloud's SNI, domain2.com is the alias to it, had to specify both for it to work for me

use_backend nextcloud_cluster if is_nextcloud  #points the ACL to the backend
use_backend nextcloud_cluster if is_nextcloud2
use_backend wordpress_cluster if is_wordpress


backend wordpress_cluster
mode tcp

option ssl-hello-chk

# Provide your server's <ip_addr>:443. You can have many servers in your backend since HAProxy does loadbalancing
server is_wordpress 10.0.0.165:443 check 


backend nextcloud_cluster
mode tcp

option ssl-hello-chk

server is_nextcloud 10.0.0.160:443 check

Since HAProxy can also do load balancing, you can scale Nextcloud across multiple computers for load balancing. The Nextcloud documentation has a page on this which could come in handy if you need this

https://docs.nextcloud.com/server/9/admin_manual/operations/scaling_multiple_machines.html

To start HAProxy, check to see if there are any instances of HAProxy running and kill them:

top -u haproxy

To start HAProxy, I ran this command:

/usr/sbin/haproxy -db -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pi

Hopefully this helps people out! I ended up having to sift through tens of different configurations to get my setup to work so YMMV, but this is what worked for me after about a week of head bashing


UPDATE:

Iā€™m providing an update for the folks who wish to do this with FreeBSD. I recently switched my environment to this Ubuntu setup to FreeBSD, and the differences are mild. First, make sure you have HAProxy installed. The config script will have a full path of /usr/local/etc/haproxy.conf. Also make sure you added the line haproxy_enable="YES" in your /etc/rc.conf file.

The config file will follow all the same configs, but for each backend, make sure you do NOT have option ssl-hello-chk.

Finally, FreeBSD has some really nice HAProxy integration. So, once you have your config file set up the way you like it, itā€™s as simple as running service haproxy start and you should be good!


UPDATE 2:

If you want to pass through all HTTP/HTTPS connections to your webserver and let your webserver do the redirect to port 443 instead of HAProxy, you can do this for the port 80 rules:

frontend localhost80
        bind *:80
        log global
        mode http

        acl is_nextcloud hdr(host) -i domain2.com
        acl is_wordpress hdr(host) -i domain1.com

        use_backend nextcloud_cluster if is_nextcloud
        use_backend wordpress_cluster if is_wordpress

A reason why you might want to do this is to more easily do Letā€™s Encrypt verifications.

4 Likes

this is really useful, thanks.

Could i ask what your proxy configuration in nextcloud config.php was?

Hi thetre97, this is all I have in my config.php file

<?php $CONFIG = array ( 'instanceid' => 'instance', 'passwordsalt' => '++salt', 'secret' => 'secret', 'trusted_domains' => array ( 0 => 'domain.com', 1 => 'www.domain.com', 2 => 'localhost', 3 => 'internal_ip', ), 'datadirectory' => '/var/nc_data', 'overwrite.cli.url' => 'http://domain.com', 'dbtype' => 'mysql', 'version' => '11.0.2.7', 'dbname' => 'nextcloud', 'dbhost' => 'localhost', 'dbport' => '', 'dbtableprefix' => 'oc_', 'dbuser' => 'adminuser', 'dbpassword' => 'passwd', 'logtimezone' => 'UTC', 'installed' => true, 'memcache.local' => '\\OC\\Memcache\\APCu', 'maintenance' => false, 'loglevel' => 1, 'mail_smtpmode' => 'smtp', 'theme' => '', );

okay, that looks similar to mine, however when I use your haproxy config file, it wonā€™t connect to NC - just comes up with ā€˜This site canā€™t be reachedā€™ although i can connect to other site within the config absolutely fineā€¦
I read somewhere in the docs about having to add some proxy configs if you are putting it behind a load balancer etc, but when i added that in it didnā€™t work eitherā€¦ :confused: I am using Docker containers - you wouldnā€™t know if itā€™s something to do with that? Guess i will have to keep fiddling :slight_smile:
Be glad of any help from anyoneā€¦

I honestly donā€™t know anything about Docker containers. I would think Docker might make a difference. Iā€™m using LXD containers for my solution (which is a lot like running on a bare metal server or VM). Perhaps you arenā€™t using the right port for your Docker container? Or (correct me if Iā€™m wrong), doesnā€™t Docker do different ports each time? You may not even have to put your NC container behind a load balancer unless you are doing many instances of your Docker image. I did this originally to have services using ports 80 & 443 be able to sit on those same ports. Iā€™d make certain your port numbers are correct on the same on HAProxy and your containers, and also make sure you have port forwarding enabled.

Thanks, works perfectly :slight_smile:

Iā€™m happy to hear it helped you :slight_smile:

I know I am dredging up a long-dead thread but this seems to be a good place to cast a wider net on my issue and you seem to be a good person to direct my question :slight_smile:

I have almost the identical set-up (using LXD and HAProxy) but cannot get mine to work. Would you mind hopping on over to my issue and seeing if you see the culprit? Thank you!

Thanks a lot. This was a great help.
I downloaded the HAProxy Aloha VM trial version, deployed it on my virtual machine and configured it as you had above (from frontend onwards). Works a charm !

**~**$ /usr/sbin/haproxy -db -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pi

[ALERT] 341/232442 (13460) : Starting proxy http-in: cannot bind socket [0.0.0.0:80]

I am getting above message when I try to start haproxyā€¦
I also tried specific ip address of the node. but same error

Is there another application already binding this interface?

possibly but not at port 80.

@vitachaos you have probably figured it out by now, but non-root users canā€™t bind to 0-1024 ports.
You need to start haproxy as root or you can change the lowest port unprivileged users can bind by
sysctl net.ipv4.ip_unprivileged_port_start=443
or by other means if you must start haproxy as unprivileged user.

Is FreeBSD lighter than Ubuntu 16.04 taking in account your configuration?

Maybe a little bit? If I look at memory on a blank FreeBSD 13 install and a blank Ubuntu 18.04 install, FreeBSD is using ~50MB of RAM at idle and Ubuntu 18.04 is using 120MB. Also note the FreeBSD box is also using ZFS as its filesystem and Ubuntu is using ext4. As for the applications on top of it, theyā€™ll be about the same. I donā€™t have my Ubuntu box anymore, but it seems HAProxy in a jail on my FreeBSD 12.2 server is using about 45MB RAM. Nextcloud is pretty variable in size. Iā€™m not sure if Iā€™m calculating it right or not, but Nextcloud is using about 2.4GB of RAM for me I think. Thatā€™s including redis for file lock cache, postgres, and many different php-fpm and nginx worker processes.

All that said, I canā€™t give a clear answer exactly on ā€œlighterā€ but Iā€™d say in my opinion I like it better and managing it better than I did with Ubuntu.

PS - welcome to the Nextcloud forum!

Hello @stratacast and community.

According to what I see in your post, I see that both the Nextcloud and Wordpress backends listen on 443 (https). My idea was to be able to perform the SSL Offload, in such a way that the encrypted connection was between the clients and HAProxy, and between HAProxy and Nextcloud, all through http (to port 80 directly).

The first configuration (backend listening on HTTPS), I have it without problems (except for the performance a bit). But my goal would be an HTTP communication between the HAProxy reverse proxy and the Nextcloud server (and Wordpress as well, as it was mentioned).

When I try to terminate the SSL connection in HAProxy and try to access the Nextcloud server from the outside, I get this web page:

<!DOCTYPE html>
<html>
<head>
	<script> window.location.href="index.php"; </script>
	<meta http-equiv="refresh" content="0; URL=index.php">
</head>
</html>

Did you try to do it? Is this possible to do? (Does Nextcloud support the configuration I am trying to do?)

any help with this?

Regards!

Hi all, Iā€™ve had my single server nextcloud up and runnng for a few weeks now, the HAProxy SSL passthrough was a bit tricky to get working so here is my experience so far.

My basic network setup, all servers are LXC s on proxmox using ubuntu 20.04.

My plan was to use haproxy to allow me to route multiple incoming URLs to different servers, nextcloud is the first server to go in, but needed haproxy working before I add other servers. Before putting haproxy in I set up portforwarding on the router for port 80 and port 443 directly to the nextcloud server. Once the letsencrypt certificate was set up for ssl the port 80 forward was removed. I then tried various configs on haproxy, many youtubes and blog posts latter found a config that works. More importantly gets A+ on nextcloud when I test security.

Note that Iā€™m not doing healthchecks as Iā€™m only running one server at present. Happy to change this if someone thinks its a good idea.

Here is my haproxy.cfg as I add future servers I will add more backends.

global
maxconn 5000
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
stats timeout 30s
user haproxy
group haproxy
daemon

defaults
log global
mode tcp
option tcplog
option dontlognull
timeout http-request 5s
timeout connect 5000
timeout client 2000000
timeout server 2000000
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http

frontend homeserver443

    bind   *:443
    mode   tcp
    option tcplog
    tcp-request inspect-delay 10s

    tcp-request content accept if { req_ssl_hello_type 1 }
    use_backend nextcloud if { req_ssl_sni -m end nextcloud.domain.site }

backend nextcloud
mode tcp
server nextcloud-server 10.0.1.88:443

Hi @dwrout

The management of the SSL certificates should be done by the reverse proxy (HAProxy in this case) and not by the web server.

In your case, your configuration will work until the expiration date of the certificate (Lets Encrypt certificates are valid for about 60 days if I remember correctly).

In my case, HAProxy resides on pfSense, which also has an Acme Certificates manager, which takes care of the automatic renewal of SSL certificates (I use the DNS Check method for verification, because otherwise it gets complicated).

Then, the SSL certificate that your backend will use will be the SSL certificate issued by the Root CA of your intranet, or some self-signed SSL certificate (in this case, HAProxy must skip the validity check for it to work).

Anyway, the ideal would be for the communication between the Nextcloud server and the reverse proxy to be via HTTP (not encrypted), and not via HTTPS (with encryption) ā€¦ but the truth is that I doubt if this will be possible with Nextcloud (or at least, I have not been successful in achieving it). Some web applications do not support SSL Offloading, and I think Nextcloud is in this category of webapps that do not support SSL Offloading.

Cheers!

Your configuration accomplishes a different goal entirely, which was never the goal of this how to. The intent of this howto is to pass through a TLS/SSL connection with HAProxy, not terminate and then reconnect to a new TLS connection in the backend

Hi Linkstat,

After much trial and error I found it simpler for me with a single home server just to pass through the SSL without terminating it on haproxy. In addition to the post here from stratacast that put me on the right track I used this tutorials I used to get it all working with auto renewal of certificates -

1 Like