# Background Job ScanFiles triggered via cron.php breaks my system **URL:** \ **Category:** ℹ️ Support\ **Tags:** nc19, cron\ **Created:** [October 23, 2020, 6:00am UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971 "2020-10-23T06:00:08Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![h3rb3rt](https://help.nextcloud.com/user_avatar/help.nextcloud.com/h3rb3rt/32/9824_2.png) [@h3rb3rt](https://help.nextcloud.com/u/h3rb3rt)\ **Post date:** [October 23, 2020, 6:00am UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/1 "2020-10-23T06:00:08Z")
Hi, I have realized that my cron is taken quite some time to finish. After some investigation I found that the job `OCA\Files\BackgroundJob\ScanFiles` is triggered via the `oc_jobs` table in my database. The job does not run every time cron is started (which would be every 5 minutes) but at least twice a day. Why ? I mean, why is it scanning all my files for new / changes files. My server hosts around 1,5 TB of data, this takes forever and I am not getting the idea of it. I would understand if I have external storage activated (I had previously btw). or if I upload files without using any NC interface (web, client, webdav) but this is not the case. **Biggest concern or unanswered question** **WHY is `OCA\Files\BackgroundJob\ScanFiles` even necessary, and why does it scan through all files even the previews ?** Thanks for any help and advice, perhaps it is a leftover from the times I used the external storage app. _Update 03.11.2020_ Some links, somehow related - [Cron task ScanFiles takes hours to finish](https://help.nextcloud.com/t/cron-task-scanfiles-takes-hours-to-finish/95718) - [https://github.com/nextcloud/server/issues/22846](https://github.com/nextcloud/server/issues/22846) - [The cron.php tasks run for hours, constant CPU load, queries on oc\_filecache](https://help.nextcloud.com/t/the-cron-php-tasks-run-for-hours-constant-cpu-load-queries-on-oc-filecache/85054) - [Cron.php takes too long and can end up running multiple times](https://help.nextcloud.com/t/cron-php-takes-too-long-and-can-end-up-running-multiple-times/85024) - [Cron does not terminate and need a lot of ram](https://help.nextcloud.com/t/cron-does-not-terminate-and-need-a-lot-of-ram/87222) - [https://github.com/nextcloud/server/issues/19078](https://github.com/nextcloud/server/issues/19078) _Update 20.11.2020_ `strace -p PID` helped me to figure out what the cron is doing in the background. For me there where two issues I could pinpoint - One was the problem of the `scanner.php` not able to follow symlinks, which happen if you but a Windows Backup into a NC directory. - [https://github.com/nextcloud/server/issues/22846](https://github.com/nextcloud/server/issues/22846) - The bug is not yet resolved but can be easily done yourself using [https://github.com/nextcloud/server/pull/21723](https://github.com/nextcloud/server/pull/21723) - The second problem was the scan of my `appdata/preview` folder. I have many pictures, and I also use the NC instance for many years already. Long story short over 3 500 000 entries in the `files_cache` table only for the previews. - I deleted the preview folder and the entries in the database based on this [Remove preview files without occ and without posix](https://help.nextcloud.com/t/remove-preview-files-without-occ-and-without-posix/31634/3) - A tip from my side, if you do so and have a huge preview folder, rename it before starting to delete it, it takes forever. The process would be 1. Put your instance in maintenance mode 2. Rename the `appdata/preview` folder to `preview_old` 3. Start deleting the folder `preview_old` 4. Delete the table entries using `DELETE FROM `oc\_filecache`WHERE`path` LIKE '%appdata_%/preview/%'` 5. After the table is clean, you can switch off the maintenance mode, a new preview folder will be created and the old one is deleted in parallel. ---
**Author:** ![h3rb3rt](https://help.nextcloud.com/user_avatar/help.nextcloud.com/h3rb3rt/32/9824_2.png) [@h3rb3rt](https://help.nextcloud.com/u/h3rb3rt)\ **Post date:** [October 27, 2020, 11:26am UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/2 "2020-10-27T11:26:58Z")
Can someone tell my why this job is necessary ? Even without external storage app in use? ---
**Author:** ![h3rb3rt](https://help.nextcloud.com/user_avatar/help.nextcloud.com/h3rb3rt/32/9824_2.png) [@h3rb3rt](https://help.nextcloud.com/u/h3rb3rt)\ **Post date:** [November 2, 2020, 6:59am UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/3 "2020-11-02T06:59:24Z")
I need to push this again, someone must know if this is normal, if this should be in the database for a standard installation or if this is kept in the database from an uninstalled app. With around 2 TB of files, this takes ages to finish and results in weird NC behavior. ---
**Author:** ![h3rb3rt](https://help.nextcloud.com/user_avatar/help.nextcloud.com/h3rb3rt/32/9824_2.png) [@h3rb3rt](https://help.nextcloud.com/u/h3rb3rt)\ **Post date:** [November 3, 2020, 7:01am UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/4 "2020-11-03T07:01:34Z")
Running `occ files:scan --all` finishes within 30 minutes and without any error. The `oc_jobs` tables tells me that the cron, after fixing some issues I had with `occ files:scan`, still takes several hours to finish. Is there a way to figure out what `cron.php` ist doing for hours, even days? I also tried to enforce only one cron job running, but this stops crons from running over days using for example `flock` The big question is, what is cron.php doing while it reserves CPU cycles and starts messing with mysqld? ---
**Author:** ![Paka](https://help.nextcloud.com/user_avatar/help.nextcloud.com/paka/32/10349_2.png) [@Paka](https://help.nextcloud.com/u/Paka)\ **Post date:** [November 7, 2020, 12:14pm UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/5 "2020-11-07T12:14:09Z")
I’ve been reading your posts from various threads because I’ve had the same issue since upgrading to v19.x. Twice a day CPU usage becomes excessive resulting in swapping which, at least once a day, grinds the VPS to a halt. Notification of excessive swapping. I’m rather surprised this issue hasn’t been resolved by now. Having spent a good amount of time on this cron.php problem I’ve, to be honest, expended far too much time with no movement toward fixing this. I’m posting here mostly to bump this … again. ---
**Author:** ![h3rb3rt](https://help.nextcloud.com/user_avatar/help.nextcloud.com/h3rb3rt/32/9824_2.png) [@h3rb3rt](https://help.nextcloud.com/u/h3rb3rt)\ **Post date:** [November 9, 2020, 8:17am UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/6 "2020-11-09T08:17:46Z")
Thanks, at least someone else having the same issue, for now I made a quite radical workaround killing all “php” processes every hour, 3 minutes after the full hour. This is not nice, but everything I could think off to keep my system running. I still do not understand why the `scanfiles` job takes so long - I even have not idea why it is even necessary. This is what I added to the `cron` of `www-data` ``` sudo -u www-data crontab -e # WorkAround # used to kill the php job which gets stuck and eats up CPU 3 */1 * * * killall php ``` ---
**Author:** ![Paka](https://help.nextcloud.com/user_avatar/help.nextcloud.com/paka/32/10349_2.png) [@Paka](https://help.nextcloud.com/u/Paka)\ **Post date:** [November 16, 2020, 11:16am UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/7 "2020-11-16T11:16:09Z")
Wow. That’s a rather severe (if sadly necessary) solution to keep Nextcloud useful! 😲 Just FWIW, here’s a report generated using Netdata to monitor the server which is running Nextcloud: system.load Chart **load average 5 = 53.3 load** five-minute load average Alarm load Family CRITICAL Severity Mon Nov 16 01:30:50 GMT 2020 Time To be honest, I’m rather puzzled why a solution or explanation of this major issue hasn’t surfaced. Perhaps v20 will resolve this. ---
**Author:** ![h3rb3rt](https://help.nextcloud.com/user_avatar/help.nextcloud.com/h3rb3rt/32/9824_2.png) [@h3rb3rt](https://help.nextcloud.com/u/h3rb3rt)\ **Post date:** [November 16, 2020, 11:51am UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/8 "2020-11-16T11:51:17Z")
upgrading to NC 20 is not possible for me yet, there is a bug with “NC Mail” which breaks my setup > > > Expected behavior > I was hoping to find my apps after the migration. > When activating Mail it should activate the app > When clicking "download and... ---
**Author:** ![Paka](https://help.nextcloud.com/user_avatar/help.nextcloud.com/paka/32/10349_2.png) [@Paka](https://help.nextcloud.com/u/Paka)\ **Post date:** [November 17, 2020, 12:32pm UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/9 "2020-11-17T12:32:04Z")
I’m looking for some way to do that with [monit](https://mmonit.com/monit/) when the server load starts increasing. Will post back here when I’ve got it functioning properly. ---
**Author:** ![h3rb3rt](https://help.nextcloud.com/user_avatar/help.nextcloud.com/h3rb3rt/32/9824_2.png) [@h3rb3rt](https://help.nextcloud.com/u/h3rb3rt)\ **Post date:** [November 20, 2020, 7:06am UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/10 "2020-11-20T07:06:17Z")
I have fixed my issue with the cron, at least for now. Using `strace -p PID` for the process never finishing I realised that it was going over my `appdata_xxx/preview` folder and was never able to finish this. Based on the `files_cache` database I had more than 3 158 958 entries linking to the preview. I assume I had even more files in there. I followed a “not so recommended” procedure to get rid of the folder and the database entries and it worked (but took forever). > [@Remove preview files without occ and without posix](https://help.nextcloud.com/t/remove-preview-files-without-occ-and-without-posix/31634/3): > > I have done the following procedure with success: backup everyting (datadir + database) before you attempt this remove folder data/appdata\_ocpb8d7vbr73/preview (appdata\_ocpxxxxxxxx/preview) (only the preview dir!) on the database, execute: delete from oc\_filecache where path like ‘appdata\_ocpb8d7vbr73/preview/%’; Now the previews are gone freeing the diskspace. When a user visits the folder and diplays previews, the previews are regenerated for that folder. Note that this slows down nextc… Still there are open questions - Why the hack do I have millions of entries for preview in there? I have lots of pictures, but if this is an “overall” bottleneck this should be solved differently. - Why the hack are all files, including previews, scanned twice a day using the `jobs` database? OR is it even more often? This is my biggest concern, question, whatsover part of this - WHY ? Bigger system must suffer from this too, I just don’t get it. - How does the `jobs` database work, what do the entries mean, any documentation appreciated. - And why is nobody jumping in from the DEVs helping out, I thought this is a support forum where even DEVs from the Nextcloud Team help out? ---
**Author:** ![Paka](https://help.nextcloud.com/user_avatar/help.nextcloud.com/paka/32/10349_2.png) [@Paka](https://help.nextcloud.com/u/Paka)\ **Post date:** [November 26, 2020, 5:30pm UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/11 "2020-11-26T17:30:54Z")
I’ve ended up using: 1. Process Resource Manager ([PRM](https://www.rfxn.com/projects/process-resource-monitor/)) 2. [Monit](https://mmonit.com/monit/). Editing existing config files. **PRM:** Edited PRM config files and needed and added on in the …/rules directory for the user that ran the Nextcloud cron job. Thus the file _ **usr123.user** _ contained: ``` IGNORE="" MAX_CPU="20" MAX_MEM="20" MAX_PROC="50" # we dont care about the process run time, set value 0 to disable check MAX_ETIME="0" IGNORE_ROOT="0" KILL_TRIG="1" KILL_WAIT="1" KILL_PARENT="1" KILL_SIG="9" # KILL_RESTART_CMD="service php7.4-fpm restart" KILL_RESTART_CMD="/sbin/reboot" ``` The PRM log showed this when the Nextcloud cron job ran away: ``` Nov 26 15:03:30 Tweedledee prm[25005]: HARD FAIL MAX_MEM use:74 limit:20 mode:killall ppid:23031 pidlist:23036 user:usr123 cmd:php7.4 -f /var/www/domain.com/html/cron.php restart-cmd:/sbin/reboot Nov 26 15:03:29 Tweedledee prm[25005]: soft fail #1 MAX_MEM use:74 limit:20 pid:23036 user:usr123 cmd:php7.4 -f /var/www/domain.com/html/cron.php ``` I’d tried restarting php7.4 as you see, but the server still ground to a hold. So I resorted to rebooting which worked. **Monit:** /etc/monit/monitrc ``` check system $HOST [..] if loadavg (5min) > 10 then restart if swap usage > 50% for 2 cycles then restart if cpu usage (system) > 90% for 1 cycles then restart stop program = "/sbin/reboot" [..] ``` I’ve not done the process with clearing out `appdata_xxx/preview` yet. ---
**Author:** ![Paka](https://help.nextcloud.com/user_avatar/help.nextcloud.com/paka/32/10349_2.png) [@Paka](https://help.nextcloud.com/u/Paka)\ **Post date:** [November 27, 2020, 12:11pm UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/12 "2020-11-27T12:11:34Z")
FYI: Bug report on this issue has been filed at: **[Query on oc\_filecache uses wrong index - Cron job runs very long #24401](https://help.nextcloud.com/t/the-cron-php-tasks-run-for-hours-constant-cpu-load-queries-on-oc-filecache/85054/30)** ---
**Author:** ![wwe](https://help.nextcloud.com/user_avatar/help.nextcloud.com/wwe/32/72963_2.png) [@wwe](https://help.nextcloud.com/u/wwe)\ **Post date:** [December 12, 2024, 7:13pm UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/13 "2024-12-12T19:13:07Z")
---
**Author:** ![wwe](https://help.nextcloud.com/user_avatar/help.nextcloud.com/wwe/32/72963_2.png) [@wwe](https://help.nextcloud.com/u/wwe)\ **Post date:** [December 12, 2024, 7:13pm UTC](https://help.nextcloud.com/t/background-job-scanfiles-triggered-via-cron-php-breaks-my-system/95971/14 "2024-12-12T19:13:09Z")