I agree and definitely it would be great the logs live in only one place. but this is a drawback of running complex system build of different components - and this is your job as sysadmin to educate about involved components and locate all the logs this components write.
regarding the above log snippet I have two observations
I donāt observe any POST request when you submit the password. In my logs there is oneā¦
it looks like you access the system using IPv6. I was under impression IPv6 doesnāt work well with docker - please review you DNS, routing, firewall etc to verify the system everything works well with IPv6 (for quick test you could ābreakā the IPv6 with hosts file ā:: cloud.defariahome.comā or your local DNS server in case you run one)
I donāt do my own DNS. I use 1.1.1.1 and 1.0.0.1.
I have moved and had gotten a new ISP. I have been using Synologyās DDNS (synology.me). This caused my IPv4 address to change so I changed the IPv4 address but I had no idea what to set the new IPv6 address to. Honestly, I didnāt even know if I was using IPv6 at all. It was set to 2603:8000:3602:5720:211:32ff:fed1:7025. However, when I go to places like https://test-ipv6.com/ it tells me āYour IPv6 address on the public Internet appears to be 2603:8000:3602:5720:8d53:448f:91dd:6808ā. I tried changing the IPv6 address in the Synology DDNS configuration and now all of my docker applications return with:
Secure Connection Failed
An error occurred during a connection to cloud.defariahome.com. SSL received a record that exceeded the maximum permissible length.
Error code: SSL_ERROR_RX_RECORD_TOO_LONG
The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.
Please contact the website owners to inform them of this problem.
Suspicious things remain:
My DNS has not really changed except as noted above (having to change my IPv4 address due to my move in Synologyās DDNS)
None of my other docker apps have any problems being accessed (maybe none of them use IPv6 and only NC does?)
Nextcloud connects and presents the login screen so all of that is working. It may be possible that Nextcloud is connecting and presenting the initial login dialog then switches over to IPv6 to continue. Seems dumb. And other users, like my newly created admin user login so if itās the case that NC switches to IPv6 then thatās working for admin.
Perhaps I need to set my IPv6 default gateway? Itās currently stated as fe80::e65e:1bff:fef5:aa97.
Hereās what is returned from nslookup(1) on the Synology:
But then everything fails with the error about not being able to provide a secure connection.
Since I saw there was a way to turn off the External Address(IPv6) on Synology I disabled it. My docker apps work but Nextcloud still fails the same way. I donāt think IPv6 is the probelm though Iām still confused why sites report my IPv6 address is different than what seems to be working for me in Synology.
The line above shows you are accessing your server via IPv6. If this is a problem or not I canāt say now.
2603:8000:3602:5720:8d53:448f:91dd:6808 > seems to be you client IP, which different from your Synology (server) IP. Often there are two different sets of port forwards in the routers, one for each protocol family maybe need to create additional IPv6 port forward for you Synology system additionally to existing IPv4ā¦
What is definitely a problem is the fact you donāt see any POST request, which exactly the piece where you send credentials to your server. in my case I see exactly the same POST request to /login endpoint within browser console and server logs (docker logs) followed by /selectchallenge (2FA provider) - in your case this might be different if you didnāt enable multi-factor authentication
as you are lucky to have both working and non-working user in your system I would recommend you to follow the logs of your browser, server and reverse proxy and identify the differenceā¦
P.S. earlier you posted a POST request - did the log belong to the working request or did something change?
Ok, but what do you say about the oddness of the different IPv6 addresses that I reported?
Doesnāt make sense. First I didnāt have to port forward an IPv6 address before and secondly, if this were the case then no user would be able to log into NextCloud. Also, which IPv6 would I forward and to where?
Oddly, a POST request pops up about a minute or two after I get the failure message in the browser. Thatās why you didnāt see it before. I thought it was done logging stuff. Here it is in this log:
It really seems like Nextcloud is presenting the login screen, taking way too long to authenticate me then switches perhaps to IPv6 or something and gets lost.
BTW why the periodic āGET /remote.php/webdav/ā requests? Some of which succeed (200) and some which fail (401)?
As I said before, I donāt know where these reverse proxy logs might be. Iāve checked the server logs (AFAICT), the docker logs, and the browser inspect things and have posted all my results. I donāt know where else to look or what else to do.
I do see that Synology uses nginx and doesnāt even have Apache installed. Looking into the docker container I see that NextCloud is using Apache and Apache logs are wired to stdout and stderr thus appearing in the docker logs.
Well, thatās extremely dismissive. Where did I say I was not ready to accept your assistance? All I said was it doesnāt make sense that I would need to add a port forward for the IPv6 address because I didnāt have to do that before and the fact that others are able to successfully login without any additional IPv6 port forward. You gotta admit that that last point surely seems to support that this suggestion doesnāt make sense.
But I would be willing to try it however as I pointed out before, I seem to have two different IPv6 addresses that different pieces of software think I have.
The Synology seems to think that the proper IPv6 address is 2603:8000:3602:5720:211:32ff:fed1:7025. In fact, if I change that address in the Synology DDNS settings then all of my other docker apps refuse to connect with āSecure Connection Failedā and SSL_ERROR_RX_RECORD_TOO_LONG.
And when I go to places like https://test-ipv6.com it reports my IPv6 address is 2603:8000:3602:5720:8d53:448f:91dd:6808.
So the question would be which of these two IPv6 addresses should I port forward? And, to which port do I forward from and to?
Iāve tried to add port forwards for 2603:8000:3602:5720:211:32ff:fed1:7025 to both port 8080 and port 80. Even tried forwarding port 443. Same failure.
Note I have a Google Nest Wifi and Iām told to set port forwards using the Google Home app. When doing the port forward I can choose a device. The device is my Synology and the Google Home app presents me with 2603:8000:3602:5720:211:32ff:fed1:7025 as an IPv6 address for that device. Further (I havenāt done IPv6 port forwards before) it only has one edit box for the port. So I added three IPv6 port forwards for that address and ports 80, 8080, and 443. Tried to log into Nextcloud as my user - it failed as before.
Google Nest Wifi has a setting to enable IPv6. I tried disabling that and still had the same problem.
I donāt think anything above indicates that Iām not ready to accept your assistance, indeed, Iām trying to do the things you suggest. They are just not working.
I spend lot of time trying to understand your issue, but you didnāt seriously follow my adviceās to understand your system, compare working and non-working logs.
I definitely agree from what you posted it makes no sense one use can login and another not (assuming the fact everything is the same)⦠but discussing about what makes sense and what not doesnāt head us into right direction.
As you stated you are an engineer I take this statement as kidding - you must be aware the IP address of your client is different from your server. The router must point to the server addressā¦
I donāt really understand what you are looking for - port 80 (often 8080 as well) is for plain http, port 443 is for TLS/SSL secured https. Depending on your setup the one or another can be right but it depends on your setup
Please review your configuration, understand whether your Synology/Nextcloud runs http or https, if there is a reverse proxy or not, collect right log files and come back⦠Iāll be happy to assist you.
The first two lines in the unsuccessful login appeared before it prompted me to log in. There was a long wait and then the 3rd line with POST in it, appeared about a minute after it displayed the error message indicating it failed to log me in. Subsequent lines all seem to be some sort of polling of webdav.
As I see it the log for the successful login has a bunch of GETs after a POST for the login which makes sense, the admin user was able to log in and Nextcloud preceded to do a bunch of GETs getting information and status. Whereas the unsuccessful login does a POST and then seems to time out.
Huh? If something doesnāt make logical sense then it surely is heading us in the right direction to not follow that path right? You can change ādoesnāt make senseā to āI donāt think this is the right path to go downā if it makes you feel better.
You should not take my statement as Iām kidding because Iām not. I realize there are different IP addresses for the client and server. But I wasnāt talking about that. I was talking about that if I go to https://test-ipv5.com it tells me that my IPv6 address is 2603:8000:3602:5720:5ac5:64ec:cd69:42a9.
So my confusion is which IPv6 address represents the serverās address for cloud.defariahome.com? 2603:8000:3602:5720:5ac5:64ec:cd69:42a9 as reported by https://test-ipv6.com or 2603:8000:3602:5720:211:32ff:fed1:7025 as known by DNS with nslookup and who is also configured into the Synology DDNS? My IPv4 address of 75.80.5.95 sure appears to be the IPv4 address of my house as I had to set change that when I moved from the previous IPv4 address before my other docker apps would work.
You said, āOften there are two different sets of port forwards in the routers, one for each protocol family maybe need to create additional IPv6 port forward for you Synology system additionally to existing IPv4ā¦ā. I took this to mean maybe I needed to forward an IPv6 port. But which port? The only IPv4 port I have forwarded is 443 to point to my Synology. As I understand it, the reverse proxy will take requests from that secure port, look at the domain name then route the connection to the appropriate port that the docker container is expecting. As I said, I tried all of 80, 8080, and 443. Why 80 and 8080? Well doesnāt Nextcloud use 8080? Not sure why I tried to forward 80, but for 443 I thought perhaps the 443 port on IPv4 and IPv6 are separate and distinct and maybe NC started using IPv4, put up the login page, then decided to switch to IPv6 for some reason and then back to :443. It was a simple and quick test to do.
To my knowledge I donāt use http, I use only https. Yes, thereās a reverse proxy - that was mentioned in the OP. I collected and reported on all logs that I know about. I mentioned that I donāt know where the reverse proxy logs may be. If you can define what the āright log filesā are and where they are located Iāll go collect them.
My server have the same issue. One of my user unable to login when using reverse proxy. But if directly access the server, this user can finally logined in after about 1 minute waiting. This example why nginx (reverse proxy) reports ātime outā.
Not sure if the login delay is related to the file size of user or not. There are many users under the server.
User A has files size about 900 GB under itās account, it uses about 60 seconds to login in.
User B has file size about 120 GB under itās account, it uses about 10 seconds to login in.
Other users with less files have no login delay.
Still no fixā¦