Since I installed Nextcloud 28.0.1, after working fine for many hours, suddenly the CRON job takes a very long time, more than an hour.
I had to restart the APACHE server in order to stop the task, then everything worked fine for several hours until the CRON task took a very long time again.
Everything was working fine with Nextcloud 27.
You’ll need to provide more details about your situation. Cron jobs are going to be impacted by:
- Apps you have enabled
- Configuration (e.g external storage, preview generation, etc.)
What do your logs indicate (particularly if you turn on debugging temporarily)?
You may want to not use webcron mode since it’s not particularly reliably for anything other than the simplest jobs[1].
You might also find the command occ background-job:list informative to see all the jobs active in your environment.
[1] Background jobs — Nextcloud latest Administration Manual latest documentation
Thank you for your reply.
I do not use webcron.
I have scheduled an occ job every 5 minutes.
I receive a mail from my server when something occurs abnormally.
And, when doing nothing with Nextcloud, suddenly, I receive every 5 minutes a mail from my server telling me that it was not possible to run the job, as there was still a job running.
More than standard applications, I have enabled OnlyOffice and Talk.
I had no such problem with NextCloud 27, and I have only migrated from 27 to 28.
Difficult for me to identify what in Logs during the problem, because since 28, there is a lot of lines in the Logs than I dit not have before, little sample:
Déboguage dav No calendar events found for status check 25 déc. 2023 à 16:06:05
Déboguage no app in context Logging out 25 déc. 2023 à 16:05:47
Déboguage dav No calendar events found for status check 25 déc. 2023 à 16:04:13
Déboguage dav No calendar events found for status check 25 déc. 2023 à 16:03:54
Déboguage dav No calendar events found for status check 25 déc. 2023 à 16:03:54
Déboguage dav No calendar events found for status check 25 déc. 2023 à 16:01:50
Déboguage no app in context Running check OCA\Settings\SetupChecks\InternetConnectivity 25 déc. 2023 à 16:01:17
Déboguage no app in context Running check OCA\Settings\SetupChecks\ForwardedForHeaders 25 déc. 2023 à 16:01:17
Déboguage no app in context Running check OCA\Settings\SetupChecks\FileLocking 25 déc. 2023 à 16:01:17
Déboguage no app in context Running check OCA\Settings\SetupChecks\EmailTestSuccessful 25 déc. 2023 à 16:01:17
Déboguage no app in context Running check OCA\Settings\SetupChecks\DefaultPhoneRegionSet 25 déc. 2023 à 16:01:17
Déboguage no app in context Running check OCA\Settings\SetupChecks\DatabaseHasMissingPrimaryKeys 25 déc. 2023 à 16:01:16
Déboguage no app in context Running check OCA\Settings\SetupChecks\DatabaseHasMissingIndices 25 déc. 2023 à 16:01:16
Déboguage no app in context Running check OCA\Settings\SetupChecks\DatabaseHasMissingColumns 25 déc. 2023 à 16:01:16
Déboguage no app in context Running check OCA\Settings\SetupChecks\CheckUserCertificates 25 déc. 2023 à 16:01:16
Déboguage no app in context Running check OCA\Settings\SetupChecks\BruteForceThrottler 25 déc. 2023 à 16:01:16
Déboguage no app in context Running check OCA\Notifications\Settings\SetupWarningOnRateLimitReached 25 déc. 2023 à 16:01:16
Déboguage no app in context Running check OCA\LogReader\SetupChecks\LogErrors 25 déc. 2023 à 16:01:15
Déboguage no app in context Running check OCA\DAV\SetupChecks\NeedsSystemAddressBookSync 25 déc. 2023 à 16:01:15
Déboguage no app in context 3 well known handlers registered 25 déc. 2023 à 16:01:11
Déboguage no app in context 3 well known handlers registered 25 déc. 2023 à 16:01:11
Déboguage dav No calendar events found for status check 25 déc. 2023 à 16:01:11
Déboguage dav No calendar events found for status check 25 déc. 2023 à 16:01:11
Déboguage dav No calendar events found for status check 25 déc. 2023 à 16:00:59
Déboguage dav No calendar events found for status check 25 déc. 2023 à 15:58:53
Déboguage dav No calendar events found for status check 25 déc. 2023 à 15:58:53
Déboguage dav No calendar events found for status check 25 déc. 2023 à 15:58:47
Déboguage dav No calendar events found for status check 25 déc. 2023 à 15:58:47
Déboguage dav No calendar events found for status check 25 déc. 2023 à 15:57:23
Déboguage dav No calendar events found for status check 25 déc. 2023 à 15:57:23
Déboguage dav No calendar events found for status check 25 déc. 2023 à 15:57:13
Hey, I experience the same behaviour. since a short time ago, the first cron run takes around 10 hours, compared to a few seconds before.
once the first run is through, it will be fast again, until next reboot.
I have 2 parallel nextcloud docker installations, which show the same behaviour.
@PierreGLV9 did you find something?
Or can someone give me a hint on how to find what the cron actually does and how to debug?
This cannot be correct behaviour…
@slimjim
Sorry, I did not find anything, so I decided to restore NC 27.0.3 and to upgrade it to 27.0.5 instead of 28.0.1, waiting the application to propose me to upgrade to NC 28 to test again, hopping this will have been debugged.
Hi @PierreGLV9 thanks for the feedback, glad you could confirm that 27 did not have this problem in your exact setup. good to know!
Restoring 27 is unfortunately not an option for me however, as nextcloud prohibits downgrading.
I decided for now to unschedule the cron, let’s see what problem this causes, until I find time to analyze the db logs more in detail. If anybody knows some technical documentation of the cron or anything else that might help, feedback is very much appreciated.
I found the long runtime being reproducible in case that helps. my schedule is to call it every 5 minutes, without concurrent runs.
first run of a day after reboot takes very long.
I am rebooting each night at 5 AM.
so for example, first run after 5 AM might take 13 hours. Then next runs after 6 PM are fast, even if I reboot on the same day they are fast. also during the time from midnight to 5 AM they are fast, but then after the schedules reboot on day 2, a long run occurs.
Issue still persists as of 28.0.2. In my specific case, following the arch wiki for scheduling background tasks, the cron service is hanging and maxing out a single thread on my server indefinitely until I stop the service. After stopping the service, the timer will reinitialize the cron task successfully. It seems to hang once every 24 hours.
Nothing in the logs when it happens, either. In fact, everything reports all good.
app list - https://pastes.io/pnjta3sfdr
I’ll disable some apps for now and see if anything changes.
Hi all, for me, issue seems fixed with 28.0.2!!
yesterday, after the upgrade, I started Cron for my 2 instances, ran about 4 hours.
Today I scheduled it again every 5 minutes, first run about 1 minute, now back to a few seconds per run.
Will observe for a few days and come back if I see the issue reocurring.
It’s apparently fixed for me now as well, however, it was acting up as I mentioned prior on 28.02. However, to try and debug it, I forced every scheduled task to run one by one to see if any of them made the cron task hang, While none of them caused it, the issue also hasn’t come back since doing that. I’ve given it 3 days to act up again and it hasn’t.
Updated to 28.0.3 today and the issue returned within 2 hours. I checked the logs and had an error with the Circles app repeating up to 10 times per minute. I had that app disabled at the time. I’ve reenabled it now, even though I’ll never use it and it doesn’t appear it can be removed. Maybe it was the problem all along?
The error in logs -
[related_resources] Info: Could not resolve OCA\Circles\CirclesManager! Class "OCA\Circles\CirclesManager" does not exist
from ? by -- at Mar 1, 2024, 12:50:35 PM
With every update Nexctloud get significantly worse. I installed Nextcloud Hub 8 (29.0.3) today (manual install) on fresh Ubuntu 24 server on machine that used to run older NC 27 mostly fine for a year (without updates because updates break something each time), which I regret now, should have kept it running as it was, now I have no functional cloud… I have a very basic set up, single user, 7 basic apps, I had to disable previews because some app was flooding the log with MP3 file errors. The server ran for half a day and now, the cron jobs gets stuck, never finish, I have 3-5 cron jobs running now, using up all CPU power, synchronization slows to a crawl. If I kill those jobs, sync picks up speed again for few minutes until cron jobs start running again, but it does’t look like the sync will ever finish at this rate. There is nothing in the log, no idea why the jobs hang.
No memcache, redis started spitting out errors too after couple of hours and causing “internal server error” preventing Nextloud from running, had to disable memcache and remove redis. I never used memcache anyway before.
BTW, it hangs when I run it manually from cmd too. I don’t know how to do verbose, -v or -vvv does not work.
Core i5 6500, 16GB RAM, SATA SSD, Gigabit LAN, nothing else is running on it.
So yeah, Nextcloud is getting increasingly bloated and buggy.
Sync never finished yesterday… even after disabling the cron and rebooting the server and the client, sync performance deteriorated to the point of being unusable.
So I reinstalled at the end of the day Ubuntu and Nextcloud, no redis this time, disabled all previews, remove all apps. It started OK and seemed to be doing fine for few hours. Left 760GB syncing overnight uninterrupted, uploading data from a Windows 11 client over gigabit, Nextcloud itself is empty. After 10+ hours, still not synced… three stuck cron jobs, 100% CPU use, less than 30GB left, but ETA is 6 hours. No errors in the log. Just incredibly slow because CPU is maxed out by stuck cron jobs.
What is going on? Nextcloud 27 synced in under 3 hours without errors on the same hardware. Can I still get 27 somewhere?
Seriously bad… I don’t want to kill those jobs this time, see if they ever finish, but until then the server is unusable:
