Cannot log in to web interface. Android sync client works

adm1ns · September 20, 2022, 4:15am

Nextcloud version: 24.0.5.1
Operating system and version: Oracle Linux 8.6
Apache or nginx version: nextcloud:latest docker image
PHP version (eg, 7.4): nextcloud:latest docker image

Hi everyone.

My second serious NC instance failure. Nothing has changed in a 4-month running instance, well, nothing since the last time I was able to log into it. Browser page kicks me back to the login page. Sometimes it logs me in, I am able to navigate pages, but there are errors everywhere, like a phantom logged in state (there is also a popup saying “You are not logged in”), then after I navigate a few pages - kicks me out to the login page again.

I’ve seen a dozen of topics like this one, mostly addressed at NC v16,19, I tried fixes from them - no luck. I have always had since the beginning 'overwriteprotocol' => 'https', , I just changed 'overwrite.cli.url' => 'https://localhost to my actual domain name. Again, no results. Multiple browser changes, restarts, container restarts etc. I do use Nginx Proxy Manager since the beginning, didn’t change its config.

Is this the first time you’ve seen this error? Y:

Steps to replicate it:

Every time
Different browser
New private tab (no cache or cookies)

The output of your Nextcloud log in Admin > Logging:

CANNOT ACCESS

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

On Pastebin

The output of your docker log in nc_webroot/data/nextcloud.log:

nextcloud  | 191.192.193.194 [20/Sep/2022] "POST /login HTTP/1.1" 303 897 "-" "Mozilla/5.0 (Windows NT 10.0; rv:104.0) Gecko/20100101 Firefox/104.0"
nextcloud  | 191.192.193.194 [20/Sep/2022] "GET /apps/dashboard/ HTTP/1.1" 303 899 "-" "Mozilla/5.0 (Windows NT 10.0; rv:104.0) Gecko/20100101 Firefox/104.0"
nextcloud  | 191.192.193.194 [20/Sep/2022] "GET /login?redirect_url=/apps/dashboard/ HTTP/1.1" 200 6821 "-" "Mozilla/5.0 (Windows NT 10.0; rv:104.0) Gecko/20100101 Firefox/104.0"

Output errors in nextcloud.log in /var/www/ or as admin user in top right menu, filtering for errors. Use a pastebin service if necessary.

On Pastebin

EDIT:
When I tried to log in from my Android device (AntennaPod trying to sync podcast lists and progress) it gave me

Access forbidden
State token does not match

or

Access forbidden
CSRF check failed

Another strings found in my log that may indicate of a problem:

192.193.194.195 - - [21/Sep/2022] "PUT /apps/user_status/heartbeat HTTP/1.1" 401 1004 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:104.0) Gecko/20100101 Firefox/104.0"
192.193.194.195 - - [21/Sep/2022] "PUT /apps/user_status/heartbeat HTTP/1.1" 412 1318 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:104.0) Gecko/20100101 Firefox/104.0"

These show up at the same time I do get to log in, but get an error message “You are not logged in”

ligal · September 20, 2022, 6:42pm

Hello,

I’ve read the error output and it seems you have a problem with your reddit integration.

Can you disable it on GUI or maybe on CLI if you can’t on GUI ?

adm1ns · September 20, 2022, 7:13pm

Hi, thanks for response

I cannot log into the GUI, but I did disable both Twitter and Reddit integration apps via
docker exec -it -u www-data nextcloud /var/www/html/occ app:disable integration_reddit
That did not help. That app was working for several months

SDISPO · September 21, 2022, 11:29am

Hello

I’have same issue. Unable to connect to my server from the web with any user or admin accounts

ligal · September 21, 2022, 8:30pm

When you look into the nextcloud.log there is clearly an error from this application.
But maybe I jump on it too quickly.

Can you see if there is the same error again ? It should happen each time you are logout if it’s the issue.

Also does this happen to all your users ? Or just the one you are using ?

Edit : I’ve just seen that :

Citation
EDIT:
When I tried to log in from my Android device (AntennaPod trying to sync podcast lists and progress) it gave me

Access forbidden
State token does not match

or

Access forbidden
CSRF check failed

have you look into your database if there is any problem with the account’s token ?

adm1ns · September 21, 2022, 9:19pm

Since I’ve disabled reddit_integration I haven’t seen this same error anymore. When I log in, I can see the Files widget and the weather. So the app is disabled. There is only one user that’s not using the Reddit integration app.

Now, the database. What do I do with that? I have no idea about any tokens. I looked into the MariaDB docker logs and there is nothing suspicious there, a few warnings, no errors. Thinking about that, I could have pulled a new MariaDB docker image as part of my update process. I wonder if that’s the issue.

Back to the “tokens”. Where do I check for those?

UPDATE: I did occ maintenance:repair - did not help. Still login loop. Looks like the access token error shows only on mobile and only after I try to log in a few times in a row. So, probably brute-forcing or something kicks in…

ligal · September 22, 2022, 7:40pm

It might be the case. But as you said you use private tab so you don’t use old cookies…

I’m a bit lost here because my best guess is : you have changed your database and something in the database is messing with your account.

Citation
Back to the “tokens”. Where do I check for those?

In your database at the user table.

Else maybe you can use the dev tools of your browser to see what is happening when you get log out ?

In a last resort have you tried to install a brand new nextcloud and import all your file and database ?

adm1ns · September 23, 2022, 4:53pm

It’s been a while since I learned SQL, fun to get back into it and remember to finalize everything with a “;”.
So, I queried my users table. This is what I got:

MariaDB [nextcloud]> select * FROM oc_users;
+---------+----------------+-----------------------------------------------------------------------------------------------------+-----------+
| uid     | displayname    | password                                                                                            | uid_lower |
+---------+----------------+-----------------------------------------------------------------------------------------------------+-----------+
| user1   | John Doe       | 3|$argon2id$v=19$m=65536,t=4,p=1$LoTs$/of/2VaRiOu$5y847tdf8734h87th3487het78fh3874ht3784hGiBBeRiShs | user1     |
| user2   | Jane Doe       | 3|$argon2id$v=19$m=65536,t=4,p=1$LoTs$/of/2VaRiOu$5y847tdf8734h87th3487het78fh3874ht3784hGiBBeRiShs | user2     |
+---------+----------------+-----------------------------------------------------------------------------------------------------+-----------+

I have 62 rows in oc_authtoken. I see some people complain about that table overflowing with data.
Really I don’t see anything strange. All the tables I’ve queried, I see the response, can you think of a specific table to look at?

I pulled MariaDB tags 10.9 and 10.8 with the same login loop. Returned back to latest (10.9.3 currently)

I don’t know if it’s useful info, but Davx5 works fine, even on a new installation (of Davx5) after the problem started happening. I am not too sure how to use browser tools, which tab etc.

ligal · September 23, 2022, 5:45pm

I don’t know the Nextcloud tables very well but I would search for the table with your auth token/cookies - so maybe oc_authtoken -.
And I will verify if my browser is receiving the cookies in there. Because if not when trying to access for your account private pages all your requests will be denied.

For the browser you should look at two things : console and network tab.
The first should show you error messages that might help you understanding the problem or Google the solution.
And the last will give you an understanding of the response that your server is returning. Look specifically for the HTTP response code first.

And finally Davx5 might help here. Because it uses cookies too - I think since it’s based on HTTP(s) -.

Sorry I know all of this is not easy when you have to discover them.

adm1ns · September 23, 2022, 9:07pm

Basically I could see a bunch of 401 (unauthorized) codes.
On the Network tab there is a lot going on, but 401 stand out. This is one of the request/response that showed up after I logged in:

PUT https://nc.domain.main/apps/user_status/heartbeat
[HTTP/2 401 Unauthorized 195ms]

	
PUT
	https://nc.domain.main/apps/user_status/heartbeat
Status
401
Unauthorized
VersionHTTP/2
Transferred945 B (43 B size)
Referrer Policyno-referrer

	
cache-control
	no-cache, no-store, must-revalidate
content-length
	43
content-security-policy
	default-src 'none';base-uri 'none';manifest-src 'self';frame-ancestors 'none'
content-type
	application/json; charset=utf-8
date
	Fri, 23 Sep 2022 20:58:05 GMT
expires
	Thu, 19 Nov 1981 08:52:00 GMT
feature-policy
	autoplay 'none';camera 'none';fullscreen 'none';geolocation 'none';microphone 'none';payment 'none'
pragma
	no-cache
referrer-policy
	no-referrer
server
	openresty
set-cookie
	ocgek0n00t0x=f8655210c8693abc56737756d3dfef4a; path=/; secure; HttpOnly; SameSite=Lax
strict-transport-security
	max-age=63072000; preload
x-content-type-options
	nosniff
X-Firefox-Spdy
	h2
x-frame-options
	SAMEORIGIN
x-permitted-cross-domain-policies
	none
x-powered-by
	PHP/8.0.23
x-request-id
	W9KWC6YGPzTbnRoIPlq8
x-robots-tag
	none
x-xss-protection
	1; mode=block
	
Accept
	application/json, text/plain, */*
Accept-Encoding
	gzip, deflate, br
Accept-Language
	en-US,en;q=0.5
Connection
	keep-alive
Content-Length
	19
Content-Type
	application/json
Cookie
	__Host-nc_sameSiteCookielax=true; __Host-nc_sameSiteCookiestrict=true; ocgek0n00t0x=ecbc5fcfb608c6ef2e75d5ba0333333O; oc_sessionPassphrase=u4Mu8bsjnUwlMENQTTczM6loc8fVhSRotLGsaMifHucaSNG3p9jga0JvooRUlDo%2FnN4EAWo0gCLG7xdwTFGJiWB7BsCsWgeQpFgH1JiezfX0nxPj0IgQ%2FYuBSY8hMGu3
DNT
	1
Host
	nc.domain.main
Origin
	https://nc.domain.main
requesttoken
	YdZwvg/JYOofx1IGe6OqAFRdoKMbOwMuhwzjBZZkMKM=:C43ZEDiraK8ngwpqDfXnaiAx1PZKDc5zZDuUdqUzVdo=
Sec-Fetch-Dest
	empty
Sec-Fetch-Mode
	cors
Sec-Fetch-Site
	same-origin
User-Agent
	Mozilla/5.0 (X11; Linux x86_64; rv:105.0) Gecko/20100101 Firefox/105.0

There is more from the Console page right after it logs me out:

InstallTrigger is deprecated and will be removed in the future. constants.js:50:14
Content Security Policy: The page’s settings blocked the loading of a resource at inline (“script-src”). moz-extension:40:65
Content Security Policy: The page’s settings blocked the loading of a resource at inline (“script-src”). moz-extension:40:65
No OC found index.js:46:4
Proxying an event bus of version 2.1.1 with 1.3.0 index.es.js:2337:14
JQMIGRATE: Migrate is installed, version 3.4.0 jquery-migrate.min.js:2:698
jQuery is deprecated: The global jQuery is deprecated. It will be removed in a later versions without another warning. Please ship your own. globals.js:62:15
$ is deprecated: The global jQuery is deprecated. It will be removed in a later versions without another warning. Please ship your own. globals.js:62:15
Backbone is deprecated: please ship your own, this will be removed in Nextcloud 20 globals.js:62:15
Handlebars is deprecated: please ship your own, this will be removed in Nextcloud 20 globals.js:62:15
Proxying an event bus of version 2.1.1 with 1.3.0 index.es.js:2337:14
jQuery is deprecated: The global jQuery is deprecated. It will be removed in a later versions without another warning. Please ship your own. globals.js:62:15
jQuery is deprecated: The global jQuery is deprecated. It will be removed in a later versions without another warning. Please ship your own. 9 globals.js:62:15
session heartbeat polling started

And this is what the network looks like:

ligal · September 24, 2022, 10:03am

Okay perfect !

The network part just validates what we thought : there is a problem with the authentication of your Nextcloud. You can virtually pass the authentication but in fact your client (browser) is not receiving the good (or any) token that represents the authentication of the client.

I am not an expert on Nextcloud system and I never encounter this error (so no experience with it). So I can only recommend that you go on Google and follow this type of topic : Nextcloud behind reverse Apache proxy - 401 unauthorized - #4 by Erebos

Back at the console, at lots of logs are just warning and it’s normal. But these two lines:

Citation
Content Security Policy: The page’s settings blocked the loading of a resource at inline (“script-src”). moz-extension:40:65
Content Security Policy: The page’s settings blocked the loading of a resource at inline (“script-src”). moz-extension:40:65

It might be normal with your installation, but Content Security Policy should be set very strictly because it’s security measures. Maybe you have a misconfiguration somewhere regarding that policy ?

adm1ns · September 24, 2022, 4:37pm

Wow, still can’t wrap my head around on how those security policies have changed…

I use Nginx Proxy Manager. I saw that topic yesterday and couldn’t see where I would put those rewrite policies, especially on a different reverse proxy (they’re talking Apache, and I use NginxPM). I also saw a few topics on GitHub with a similar behavior to mine, and added recommended settings that fixed it for some users:

location / {
    proxy_pass http://nextcloud;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For
    proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;"
    }

I put those in the Advanced → Custom Nginx Configuration. The best result was the same login loop, the worst is that Nginx PM said the proxy is “Disabled”. So I removed everything from the Advanced tab, and I never had any config there while NC was working from the beginning. I tried to disable HTTP/2 in Nginx PM, and looks like I can log in on the first try, but nothing inside, still get lots of 401 responses.
Nginx PM container hasn’t been updated since march. I have added a few proxy redirects, but that’s about it.
I am not extreme on security, nor I am too knowledgeable, I have a piece of mind knowing that only a few IPs can access my NC instance, that’s enough for me.

OK, problem partially solved… I attribute the root cause to my reverse proxy.
I decided to shut down all the other containers, but leave Nextcloud, MariaDB and Redis running, and oh miracle: I can now log in normally. I started to think that the issue could be in all my containers running on the same internal network. I run Libreddit and Nitter+Redis through the reverse proxy (SSL, TLS, HTTP2 all the good stuff). Now, when Libreddit was shut down, NginxPM refused even to start saying “host libreddit could not be found”. I started it, and my Nextcloud working again. I created a separate network just for the Nextcloud stack - then NginxPM refused to start saying “host nextcloud could not be found”. NginxPM is on 172.17.0.0/16 and Nextcloud is on 172.20.0.0/16 - no clue what the problem is. I returned NC on the main bridge network, and put Nitter+Redis on its own, now I get 502 for nitter, which is something I can mess with later.
I guess I can’t have 2 redis instances on the same network, is that the case? I used to have 2 MariaDB and they were working, not sure how… They were both listening on the same port, but working.

ligal I really appreciate your help here, I’m glad I didn’t wipe my NC in frustration, even though I was about to… If you have any comments on how to make THEM ALL work, please, I’d be glad to hear, if not - thanks a lot, and I will be moving to the nginx or docker community where it will be more relevant. For me NC is more important to work than Nitter.