Redirect loop on Snap install (Ubuntu 20.04 with Nexcloud Snap on install)

Bernd_Wechner · August 18, 2020, 7:44am

Support intro

Sorry to hear you’re facing problems

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can

Nextcloud version (eg, 18.0.2): 19.0.1
Operating system and version (eg, Ubuntu 20.04): Ubuntu 20.04
Apache or nginx version (eg, Apache 2.4.25): Don’t know it’s default Snap install
PHP version (eg, 7.1): Don’t know it’s default Snap install

The issue you are facing:

I installed it, configured it and it all worked fine under http.

I tested three means of access:

By LAN IP: http://192.168.0.14 - the IP the server is on.
By LAN DNS: http://nextcloud.lan - the name of the box with .lan names resolved locally
By WAN DNS: http://mynextcloud.mydomain.tld

All good.

I wanted to go https, so ran

sudo nextcloud.enable-https lets-encrypt

And tested:

http://192.168.0.14 - Works fine
http://nextcloud.lan - Works fine
http://mynextcloud.mydomain.tld - - Redirect loop.

I tested it with http://www.redirect-checker.org and see:

https://mynextcloud.mydomain.tld/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently
https://mynextcloud.mydomain.tld:443/
301 Moved Permanently

Now the huge proble is diagnosing this! I’m not new to web stuff or Nextcloud but new to Snap and on the edge of tossing the Snap install, but for the fact that well, it seems to be vogue and upskilling here can’t go awry either. So far, pretty painful though.

How: Well I would check the the webserver logs and up webserver logging level to debug and log access requests etc and last I rememember on Apache (been a while to be honest dumped it for lighttpd most everywhere) I could ask it for rule resolution logging too. And if Apache wasn’t doing it I’d start logging PHP in debug.

But alas none of these configs are editable. The directory /snap/nextcloud seems write protected in spite of ext4 permission saying I can write (and logging on as root even). Snap is pulling some trick new to me that the Linux kernel is respecting to keep the whole thing read only. /var/snap/nextcloud is editable and has a famous config file here:

/var/snap/nextcloud/current/nextcloud/config/config.php

And it’s even documented to a good degree:

https://docs.nextcloud.com/server/19/admin_manual/configuration_server/config_sample_php_parameters.html

But the Apache and PHP configs at:

/snap/nextcoud/current/conf/httpd.conf
and
/snap/nextcloud/current/config/php/php.ini

are untouchable.

I can set Nextcloud logging to debug:

sudo snap set nextcloud mode=debug
sudo nextcloud.occ config:system:set loglevel --value=0

But see no clues when I inspect the log:

sudo nextcloud.occ log:watch

Nothing shows when I reload the WAN URL.

Now I am behind a reverse proxy. But it passes everything on that domain name unonditionally through to the Nextcloud box. I don’t expect this to make sense to anyone as it’s a lighttpd conf, as the reverse proxy is an OpenWRT gateway router and that runs lghttpd:

# The NextCloud server
$HTTP["host"] == "mynextcloud.mydomain.tld" {
      proxy.server  = ( "" => ( ( "host" => "192.168.0.14" ) ) )
}

But here’s the deal, because I can’t set Apache logging or PHP (frigging SNAP!) I can’t even see how the request is arriving at the Nextcloud server. If I could see that and know what host it sees, and further how it reacts to that host I’d be laughing.

To make matters worse, that reverse proxy untouched, as it stands, has been feeding a NextCloudPi server for years. NextCloudPi in fact rocks and I love it, but the Pi is a tad sluggish and unreliable (locks up and needs resets all too often so I’m moving to an Ubuntu server and Ubuntu 20.04 asks on install if you want a NextCloud server which is sweet, and it sintalls fine as a snap!)

Which evidences for me, that the proxy is probably, very probably, not the issue. That said, the NextCloudPi instance was still on Nextcloud 18 not 19 and it’s a different build and way more configurable than the Snap!

Even more painful is the sheer number of unrelated redirect loop question is in this forum, suggesting it’d be jolly nice if there was some quick way to diagnose the cause and that clearly demands detailed web request and response logging.

Now there may be a magic combo of settings here:

https://docs.nextcloud.com/server/19/admin_manual/configuration_server/reverse_proxy_configuration.html

that fix this, and as much as I love, respect and cherish the documentation and effort that goes into it (I managed documentation teams for some while), this still does not serve well in clarifying what is going on here and what these settings actually do how and why.

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it: Well covered above.

The output of your Nextcloud log in Admin > Logging:

Well as stated nothing as added to the log in spit fo debug mode and debug log levels when I access through the WAN.

There are some issues in it but unrelated (I’d like to solve those but let’s not digress, they can wait). For the record here’s a tail 1000 of the log immediately after refreshing the WAN URL 3 or 4 times:

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

Snap install and I don’t know what you mean here

The output of your Apache/nginx/system log in /var/log/____:

Snap install and Apache logs only errors and none are logged.

Given how often this crops up on the forum I can’t help but wonder if I missed or if there should be a FAQ page on redirect loops, likely causes, and diagnostic methods (that work for the Snap!)

I did find this:

but do I have to do this? And if so can I download a new snap while I keep my old snap disabled (but configured as I don’t want to lose the configs in /var/snap/nextcloud/current/nextcloud/config/config.php at the very least. How much time and energy is snap going to sap from me in learning to work with it? I wonder.

KarlF12 · August 18, 2020, 6:50pm

To be completely honest, I abandoned snap when I found out I would have to move heaven and earth to add smbclient. I switched to Docker and have never looked back. Seems to me snap is only good for very basic Nextcloud installations due to the necessity to make additions and changes.

I’m thinking this is most likely the issue. When you run certbot, it sets up an automatic redirect for you from HTTP to HTTPS. What may be happening is your reverse proxy is trying to hit HTTP and constantly getting a redirect from the snap Apache.

Normally when using a reverse proxy, you will run certbot in the reverse proxy, not the backend server. And then on that last local hop you can leave it HTTP or use a self-signed certificate and disable validation at the proxy.

Bernd_Wechner · August 18, 2020, 11:09pm

You’re tempting me. But doesn’t Docker have the overhead of a whole OS inside of it? A newb here with snap and docker. I admit I run a few servers behind this proxy and I tend to have certbot run on the server and copy the certs to the proxy with a deploy hook. That’s because my reverse proxy configs tend to be dirt simple forward everything to the LAN server in question. If I ran certbot on the router itself, possible, I’d need to map .well-known onto local router directories and not forward them to the server in question. Also an option. Possibly even a simpler one, just not one I pursued at outset preferring to keep cert management with the respective servers.

I wonder by what means certbot would perform the redirect you mention? It could only do that by adding an Apache config, or? And that’s not possible as the snap is read only! Grrrr - by some obscure mechanism too. But redirectchecker (above) confirms it’s an https to https redirect in any case not http to https.

KarlF12 · August 19, 2020, 12:40am

Oh no, not at all. It isn’t like a virtual machine. Each Docker container (usually) only runs a single process.

At the end of a normal certbot setup, it prompts you whether to add a redirect to your Apache config for you. I walked away from snap before getting to that point, so I don’t know if it does the same thing (potentially without asking).

KarlF12 · August 19, 2020, 12:50am

Can you turn off the snap lets-encrypt the same way you turned it on, and if so, does it work again?

Bernd_Wechner · August 19, 2020, 1:09am

Just did that and yes if I do this:

sudo nextcloud.disable-https

the URL:

https://mynextcloud.mydomain.tld

loads fine!

Now I can live with non encypted LAN access but I do want lan access (and it always worked wit the NextCloudPi server on this IP):

https://nextcloud.lan/

now simply returns ERR_CONNECTION_REFUSED

which is understandable given I disabled https.

But:

http://nextcloud.lan/

presents the login page, but if i log in, it bounces straight back to the login page with no message. Now I have the snap in debug mode and debug logging on so I can watch while trying to login and I see this:

$ sudo nextcloud.occ log:watch
  Debug    core               OC\AppFramework\Middleware\Security\Exceptions\NotLoggedInException: Current user is not logged in at .../Middleware/Security/SecurityMiddleware.php line 142                        2020-08-19T10:54:47+10:00 
                                                                                                                                                                                                                                             
                              0. lib/private/AppFramework/Middleware/MiddlewareDispatcher.php line 98                                                                                                                                        
                                 OC\AppFramework\Middleware\Security\SecurityMiddleware->beforeController(OCA\Files\Controller\ViewController {}, "index")                                                                                   
                              1. lib/private/AppFramework/Http/Dispatcher.php line 98                                                                                                                                                        
                                 OC\AppFramework\Middleware\MiddlewareDispatcher->beforeController(OCA\Files\Controller\ViewController {}, "index")                                                                                          
                              2. lib/private/AppFramework/App.php line 137                                                                                                                                                                   
                                 OC\AppFramework\Http\Dispatcher->dispatch(OCA\Files\Controller\ViewController {}, "index")                                                                                                                  
                              3. lib/private/AppFramework/Routing/RouteActionHandler.php line 47                                                                                                                                             
                                 OC\AppFramework\App::main("OCA\\Files\\Controller\\ViewController", "index", OC\AppFramework\DependencyInjection\DIContainer {}, {_route:"files.view.index"})                                               
                              4. <<closure>>                                                                                                                                                                                                 
                                 OC\AppFramework\Routing\RouteActionHandler->__invoke({_route:"files.view.index"})                                                                                                                           
                              5. lib/private/Route/Router.php line 297                                                                                                                                                                       
                                 call_user_func(OC\AppFramework\Routing\RouteActionHandler {}, {_route:"files.view.index"})                                                                                                                  
                              6. lib/base.php line 1007                                                                                                                                                                                      
                                 OC\Route\Router->match("\/apps\/files\/")                                                                                                                                                                   
                              7. index.php line 37                                                                                                                                                                                           
                                 OC::handleRequest(                                                                                                                                                                                          
                                                                                                                                                                                                                                             
                                 )

which provides a clue, but a poor one. As in doh! Of course I’m not logged in I’m logging in. Useless log!

If I watch the network traffic in Chrome while doing so, it’s fascinating. The login post gets a successful response directing to: http://nextcloud.lan/index.php/apps/files/ but the request for that directs to /index.php/login?redirect_url=/index.php/apps/files/.

But why? This all rather nightmarish, and it looks like the Docker solution may be better. Need to understand Docker better. But sounding more popular, better supported and more functional than snap. Thoughts?

Of particular note if I enable https with letsencrypt, then the URL https://nexcloud.lan logs in fine! But the WAN URL https://mynextcloud.mydomain.tld has an endless redirect loop.

This is proof that the redirect loop is created by the https enable. And I prefer it enabled! Contingent on my being able to drop a deploy hook into the certbot config in the snap! If that’s not possible because of danged snap write protections then snap https management is dead in the water anyhow - as I use on all my other servers a deploy hook that copies the certs to the reverse proxy. Not least as the reverse proxy is particular and wants the key and cert in one .pem so I combine them and copy them over.

KarlF12 · August 19, 2020, 1:18am

Now ideally, this is what you want. I access mine through the reverse proxy even while on my LAN. That way my devices can roam on/off network and the address is always correct and always has a valid certificate. My advice there is just don’t use nextcloud.lan.

That being said, you need a local DNS server if you don’t have one already so you can make mynextcloud.mydomain.tld resolve to the LAN IP while on the LAN. If you don’t have this, you end up with hairpin routing where you access the system locally via the WAN IP which most respectable firewalls don’t even allow.

Your firewall may have a built in DNS server, or you could perhaps run BIND on an existing system you have.

Bernd_Wechner · August 19, 2020, 1:25am

Just checked Docker:

And look sold on that.

I have a DNS running on the one gateway that runs OpenWRT and acts as DNS, firewall and reverse proxy. If I dig my WAN name it returns the WAN IP not the LAN IP. So accessing that would do a hairpin (and I worry cause traffic to flow via the ISP which I’m on the LAN). I use the .lan address only for static lan devices here and have no mobile concerns really.

That said, I’m happy to try that but getting a tad frustrated with how painful snap is proving when consistent recommendations I read are “go Docker”.

I thin I can disable the snap and try a docker and they can live side by side in any case using only disk space really while I experiment a little. What’s miffing me is how much effort this is all consuming when NextCloudPi just worked a charm for years (bar its sluggishness running on a Pi and flakiness - requiring regular resets, which were livable if annoying as Pi’s reboot fast and clean).

Bernd_Wechner · August 19, 2020, 1:54am

Hmmm, can’t find an easy Docker package. Looks complicated. Loads of multicontainer solutions promoted on-line for the nextcloud, webserver and database. There’s are official images:

https://hub.docker.com/_/nextcloud/

Alas confusing. There’s a Apache image that is argued to be quick and easy, but an implied higher performance fastCGI verison which uses SQLite by default … Heck am tempted to go fastcgi and install my own webserver and database. But as I have a dedicated server I wonder why even bother with docker and not run it all native? I guess the Docker fastCGI image will at least containerize the PHP dependencies well.

KarlF12 · August 19, 2020, 1:57am

That, and they can’t share port numbers, so you’ll have to change one or the other away from 80 and 443. But beyond that, they should coexist fine.

I have not used NCP, but if I’m not mistaken, I think there’s a Docker image for it too.

I’ve also done a couple “manual” Nextcloud installations where you install the Apache site locally and run MySQL, and I found that to be fairly easy as well.

What I really find captivating about Docker personally is how easily you can update the software. You just tell it to re-pull the images and hit go, service restarts and everything is updated just like that. And because the containers are sandboxed environments, they always have their own tried-and-proven dependencies install with them. No compatibility issues regardless of what the host system is.

If you really want to try Docker, I wrote a guide for it on a previous version. Maybe it’ll give you some insight.

https://help.nextcloud.com/t/howto-ubuntu-docker-nextcloud-talk-collabora/76430