Nextcloud keeps dropping into Maintenance mode with no rhyme, reason or clue as to why (YAMMI)

YAMMI: Yet Another Maintenance Mode Issue. Why? Because I did due diligence and search the forums for it and there is a plethora of them. Alas none describe my current issue as far as I could tell (as in there are so many of them historically I haven’t examined all of them admittedly just a lot of them).

My Issue: My Nextcloud, suddenly, for for no clear reason, no clues left, nothing I can find in logs, and nothing I did, keeps dropping, like twice a day or more, into maintenance mode. I can turn it off and do with occ maintenance:mode --off every time I notice, but hey, I don’t need a full time job dropping a Nextcloud out of maintenance mode nor to write a monitor to automate it :wink: The Nextcloud is fully functional the moment i drop it out of maintenace mode. I see no issues.

The only thing my logs are riddled with (the log at /index.php/settings/admin/logging) is:

[webdav] Fatal: System is in maintenance mode.

Raw dump:

{"reqId":"zwhSJSEQCaiRzhtWFrWA","level":4,"time":"2023-01-16T08:44:50+11:00","remoteAddr":"192.168.0.11","user":"--","app":"webdav","method":"PROPFIND","url":"/remote.php/dav/addressbooks/users/bernd/contacts/","message":"System is in maintenance mode.","userAgent":"Thunderbird CardBook/83.7","version":"23.0.0.10","exception":{"Exception":"Sabre\\DAV\\Exception\\ServiceUnavailable","Message":"System is in maintenance mode.","Code":0,"Trace":[{"file":"/var/www/html/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"checkMaintenanceMode","class":"OCA\\DAV\\Connector\\Sabre\\MaintenancePlugin","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":456,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/html/nextcloud/apps/dav/lib/Server.php","line":339,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/html/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/var/www/html/nextcloud/remote.php","line":166,"args":["/var/www/html/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/html/nextcloud/apps/dav/lib/Connector/Sabre/MaintenancePlugin.php","Line":85,"CustomMessage":"--"},"id":"63c47828e63ee"}

of which there a LOT and looks to be because it’s in maintance mode not an explanation as to why.

and

[core] Error: Following symlinks is not allowed (‘/backup/backup/var/lock’ → ‘/run/lock/’ not inside ‘/backup/’)

Raw dump:

{"reqId":"AURUoSoRFT7JbritiZ2D","level":3,"time":"2023-01-16T09:05:06+11:00","remoteAddr":"","user":"--","app":"core","method":"","url":"--","message":"Following symlinks is not allowed ('/backup/backup/var/lock' -> '/run/lock/' not inside '/backup/')","userAgent":"--","version":"23.0.0.10","id":"63c47893b68ff"}

which seems unrelated (or?) and not sure what it’s about. Seemingly backup, which I do have configured:

And from /index.php/settings/admin/serverinfo

Operating System: Linux 5.4.0-135-generic x86_64
CPU: Intel(R) Core™ i5-4250U CPU @ 1.30GHz (4 cores)
Memory: 15.53 GB
Server time: Mon Jan 16 10:05:50 AEDT 2023
Uptime: 23 days, 14 hours, 55 minutes, 31 seconds

And from /index.php/settings/admin/overview

Nextcloud Hub II (23.0.0)

The version you are running is not maintained anymore. Please make sure to update to a supported version as soon as possible.

A new version is available: Nextcloud 23.0.12
1 app has no compatible version for this Nextcloud version available

So I guess I’ll be updating first (hesitant as it has an unfortunate habit of costing a bucketload of fixing stuff that breaks but hey, I’ll try).

In the mean time what I want traction on, is how can one diagnose these repeated unexplained drops into mainentance mode. If it turns out to be some obnoxious side effect of the plea to update I’ll scream.

Hi, can you disable the backup app and check if that solves your issue?

I can indeed. And will try. I also did the upgrade, and it went without hitch this time ;-). So will see over today if it falls back into maintenance mode.

That said, I remain curious on the question of diagnosing cause (as opposed to empirically experimenting with upgrades, app enable/disable etc.). The latter of course I will do as I need to free myself of this bother, but doesn’t address the deeper question of how to diagnose it.

Perhaps there is cause to expect any drop into maintenance mode to record the caller and impose upon the caller a requirement to supply a reason (and maybe ID if the caller can’t be identified ;-). And logging that. But that itself is a feature idea, not a diagnosis.

Well I upgraded, not once, not twice, but thrice and nothing actually went awry for a change. The Backup app is in fact disabled (not tested with latest NC) and so I get that test at same time. None of which ultimately addresses the diagnostics question. But we shall see.

And it has not dropped into Maintenance mode since (7 days now). So the issue is gone. Though, the request for means to diagnose aberrant maintenance mode drops stands. If and hen it happens, how does one determine the cause?