Oc_jobs table values count

krakazyabra · November 5, 2020, 3:07pm

Hi all!
I’ve just noticed, that my table oc_jobs has huge job values for some classes:

mysql> select count(id) as jobs, class from oc_jobs group by class order by jobs desc;
+-------+-----------------------------------------------------------------+
| jobs  | class                                                           |
+-------+-----------------------------------------------------------------+
| 64017 | OCA\DAV\BackgroundJob\UploadCleanup                             |
| 17288 | OC\Command\CommandJob                                           |
|   806 | OCA\QuotaWarning\Job\User                                       |
|   799 | OCA\Maps\BackgroundJob\UserInstallScanJob                       |
|   790 | OCA\LookupServerConnector\BackgroundJobs\RetryJob               |
|   730 | OCA\Settings\BackgroundJobs\VerifyUserData                      |
|   239 | OCA\FirstRunWizard\Notification\BackgroundJob                   |
|   155 | OC\Settings\BackgroundJobs\VerifyUserData                       |
|    45 | OCA\Mail\BackgroundJob\SyncJob                                  |
|    45 | OCA\Mail\BackgroundJob\TrainImportanceClassifierJob             |
|     8 | OCA\Maps\BackgroundJob\AddPhotoJob                              |
|     6 | OCA\DAV\BackgroundJob\RefreshWebcalJob                          |
|     3 | OCA\FederatedFileSharing\BackgroundJob\RetryJob                 |
|     1 | OCA\DAV\BackgroundJob\CleanupDirectLinksJob                     |
|     1 | OC\Core\BackgroundJobs\CleanupLoginFlowV2                       |
|     1 | OCA\Passman\BackgroundJob\ExpireCredentials                     |
|     1 | OCA\DAV\BackgroundJob\CleanupInvitationTokenJob                 |
|     1 | OC\Log\Rotate                                                   |
|     1 | OCA\Passwords\Cron\BackupJob                                    |
|     1 | OCA\DAV\BackgroundJob\EventReminderJob                          |
|     1 | OC\Preview\BackgroundCleanupJob                                 |
|     1 | OCA\Passwords\Cron\CheckNightlyUpdates                          |
|     1 | OCA\Passwords\Cron\CheckPasswordsJob                            |
|     1 | OCA\DAV\BackgroundJob\UpdateCalendarResourcesRoomsBackgroundJob |
|     1 | OCA\Passwords\Cron\CleanUpSessions                              |
|     1 | OCA\Passwords\Cron\ProcessDeletedEntities                       |
|     1 | OCA\Deck\Cron\CardDescriptionActivity                           |
|     1 | OCA\Passwords\Cron\ProcessDeletedUsers                          |
|     1 | OCA\Deck\Cron\DeleteCron                                        |
|     1 | OCA\Passwords\Cron\SendServerSurvey                             |
|     1 | OCA\Deck\Cron\ScheduledNotifications                            |
|     1 | OCA\Passwords\Cron\SynchronizeShares                            |
|     1 | OCA\PhoneTrack\Cron\AutoExport                                  |
|     1 | OCA\Federation\SyncJob                                          |
|     1 | OCA\Polls\Cron\NotificationCron                                 |
|     1 | OCA\Files\BackgroundJob\CleanupDirectEditingTokens              |
|     1 | OCA\Files\BackgroundJob\CleanupFileLocks                        |
|     1 | OCA\RansomwareDetection\BackgroundJob\CleanUpJob                |
|     1 | OCA\Files\BackgroundJob\DeleteOrphanedItems                     |
|     1 | OCA\Richdocuments\Backgroundjobs\ObtainCapabilities             |
|     1 | OCA\Files\BackgroundJob\ScanFiles                               |
|     1 | OCA\Files_Antivirus\BackgroundJob\BackgroundScanner             |
|     1 | OCA\Support\BackgroundJobs\CheckSubscription                    |
|     1 | OCA\Files_Sharing\BackgroundJob\FederatedSharesDiscoverJob      |
|     1 | OCA\Talk\BackgroundJob\CheckReferenceIdColumn                   |
|     1 | OCA\Files_Sharing\DeleteOrphanedSharesJob                       |
|     1 | OCA\Talk\BackgroundJob\ExpireSignalingMessage                   |
|     1 | OCA\Files_Sharing\ExpireSharesJob                               |
|     1 | OCA\Talk\BackgroundJob\RemoveEmptyRooms                         |
|     1 | OCA\Talk\BackgroundJob\ResetAssignedSignalingServer             |
|     1 | OCA\Files_Trashbin\BackgroundJob\ExpireTrash                    |
|     1 | OCA\Text\Cron\Cleanup                                           |
|     1 | OCA\Files_Versions\BackgroundJob\ExpireVersions                 |
|     1 | OCA\TwoFactorBackupCodes\BackgroundJob\CheckBackupCodes         |
|     1 | OCA\Activity\BackgroundJob\EmailNotification                    |
|     1 | OCA\UpdateNotification\Notification\BackgroundJob               |
|     1 | OCA\FullTextSearch\Cron\Index                                   |
|     1 | OCA\Activity\BackgroundJob\ExpireActivities                     |
|     1 | OCA\UpdateNotification\ResetTokenBackgroundJob                  |
|     1 | OCA\GroupFolders\BackgroundJob\ExpireGroupVersions              |
|     1 | OCA\AdminAudit\BackgroundJobs\Rotate                            |
|     1 | OCA\User_LDAP\Jobs\CleanUp                                      |
|     1 | OCA\Analytics\BackgroundJob\Daily                               |
|     1 | OCA\User_LDAP\Jobs\Sync                                         |
|     1 | OCA\Analytics\BackgroundJob\Hourly                              |
|     1 | OCA\User_LDAP\Jobs\UpdateGroups                                 |
|     1 | OCA\Bookmarks\BackgroundJobs\CrawlJob                           |
|     1 | OCA\WorkflowEngine\BackgroundJobs\Rotate                        |
|     1 | OCA\Bookmarks\BackgroundJobs\PreviewsJob                        |
|     1 | OC\Authentication\Token\DefaultTokenCleanupJob                  |
|     1 | OCA\Maps\BackgroundJob\LookupMissingGeoJob                      |
|     1 | OCA\Circles\Cron\ContactsExistingShares                         |
|     1 | OC\BackgroundJob\Legacy\RegularJob                              |
|     1 | OCA\ContactsInteraction\BackgroundJob\CleanupJob                |
|     1 | OCA\News\Cron\Updater                                           |
|     1 | OCA\Contacts\Cron\SocialUpdateRegistration                      |
|     1 | OC\Core\BackgroundJobs\BackgroundCleanupUpdaterBackupsJob       |
|     1 | OCA\NextcloudAnnouncements\Cron\Crawler                         |
+-------+-----------------------------------------------------------------+
78 rows in set (0.14 sec)

Cron.php is running in while loop, average time is 5min

The question is: are such values ok or not? If not, what should I do?

j-ed · November 5, 2020, 3:23pm

Unfortunately counting the class occurrences isn’t the whole story. The “argument” column needs to be taken into account too

krakazyabra · November 5, 2020, 4:19pm

hi! sorry, I don’t understand what do you mean. Add column with argument to output table?

krakazyabra · November 5, 2020, 4:21pm

select count(argument) as argument, count(id) as jobs, class from oc_jobs group by class order by jobs desc;
Argument column shows the same value, as jobs.

+----------+-------+-----------------------------------------------------------------+
| argument | jobs  | class                                                           |
+----------+-------+-----------------------------------------------------------------+
|    49520 | 49520 | OCA\DAV\BackgroundJob\UploadCleanup                             |
|    15057 | 15057 | OC\Command\CommandJob                                           |
|      804 |   804 | OCA\QuotaWarning\Job\User                                       |
|      799 |   799 | OCA\Maps\BackgroundJob\UserInstallScanJob                       |
|      790 |   790 | OCA\LookupServerConnector\BackgroundJobs\RetryJob               |
|      711 |   711 | OCA\Settings\BackgroundJobs\VerifyUserData                      |
|      216 |   216 | OCA\FirstRunWizard\Notification\BackgroundJob                   |
|      155 |   155 | OC\Settings\BackgroundJobs\VerifyUserData                       |
|       45 |    45 | OCA\Mail\BackgroundJob\SyncJob                                  |
|       45 |    45 | OCA\Mail\BackgroundJob\TrainImportanceClassifierJob             |
|        8 |     8 | OCA\Maps\BackgroundJob\AddPhotoJob                              |
|        6 |     6 | OCA\DAV\BackgroundJob\RefreshWebcalJob                          |
|        3 |     3 | OCA\FederatedFileSharing\BackgroundJob\RetryJob                 |
|        1 |     1 | OCA\Talk\BackgroundJob\ExpireSignalingMessage                   |
...
+----------+-------+-----------------------------------------------------------------+
75 rows in set (0.15 sec)

j-ed · November 5, 2020, 4:54pm

Check-out the raw table content and you will see that you cannot count all occurrences of e.g. “OCA\Mail\BackgroundJob\SyncJob” because they have different values in the arguments column wink

MariaDB [nextcloud]> select id,class,argument from oc_jobs;
±------±----------------------------------------------------------------±------------------------------------------------------+
| id | class | argument |
±------±----------------------------------------------------------------±------------------------------------------------------+
…
| 47120 | OCA\Mail\BackgroundJob\SyncJob | {“accountId”:1} |
| 47123 | OCA\Mail\BackgroundJob\SyncJob | {“accountId”:4} |
| 47124 | OCA\Mail\BackgroundJob\SyncJob | {“accountId”:5} |
| 47125 | OCA\Mail\BackgroundJob\SyncJob | {“accountId”:6} |
…

krakazyabra · November 5, 2020, 5:18pm

of courses, arguments are different for each job. but I have there >1k users. and noticed, that cronjob wasn’t running for 3 months.
I manually launched it, after successful ending, checked the db and saw these values.

j-ed · November 5, 2020, 5:30pm

I cannot tell you if it’s better to wait until the table cleaned automatically or if it’s better to clean it manually. Due to the fact that your systems hosts more than 1k users, you should already have an official support contract in place so that you can address this question to the Nextcloud buisness support. This purpose of this forum is to help private and small business users with their issues.

krakazyabra · November 6, 2020, 9:47am

No. There is no rule, that I have to buy any contracts if I have such amount of users. It is still private cloud.

The question was: is it ok or not. if it is not ok, can I erase this table manually? What problems can occur, if i perform erasing?

wwe · November 6, 2020, 8:44pm

for sure there is no such rule as Nextcloud is free and open source software.

But you can trust on really simple rule of thumb: free support in the forum most likely has little or no experience with enterprise grade installations and issues you get there. This is a simple reason why professional support is recommended if you host big amount of users. There may be reasons to choose free support but then you can’t expect any answers, every answer is a gift.

flammekueche · November 8, 2020, 2:58pm

Got similar problem with 210000+ jobs in queue, changing background task mode (ajax,cron or webcron) didn’t work, cause cron is never realy executed whatever I do, still don’t know why.
I run in a shell an infinite loop with “php cron.php” and the queue decrease, 3 days later it was ok, now only 70 jobs
BTW I run nextcloud on docker, I create an hourly schedule from main host:
“docker exec --user nc_admin nc_container php cron.php” and everything is ok.