External Storage Performance | occ files:scan --all

Hey, just found something that might be of interest:

check which SMB-Extention you are using.
I also has smbclient activated. I removed it and installed php-smbclient

in the OC git-issues you find a lot of hints related to the term “icewind”.
it seems to be a lot quicker because it can do more requests in parallel. on a first sight - not having so much data - it looks pretty promising!

In all of my Synology´s is SMB 2 activated. Possible is also SMB 3 and SMB 1.

Where have you owncloud up and running?
I meant the OC server - not the storage.

I had to install the php-smbclient on my PI and now the connect to qnap works better

I use only Synology servers for all. Installation of nextCloud/ownCloud and also storage of data. Because of mainly I use the storage system of Synology, I use the “External Storage” system of nextCloud/ownCloud to implement.

My production system is a Synology DS1815+ server, a good performance system. In which way this difference is possible ? Scantime to start was 17:30 for ownCloud, 17:33 for nextCloud :

nextCloud v9.0.52 daily

Starting scan for user 1 out of 1 (rescan)

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 5079    | 31593 | 00:17:05     |
+---------+-------+--------------+

ownCloud v9.0.3

Starting scan for user 1 out of 1 (rescan)

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 5079    | 31592 | 00:03:10     |
+---------+-------+--------------+

On my Synology DS1815+

nextCloud 10 beta

Starting scan for user 1 out of 1 (rescan)

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 5235    | 31888 | 00:04:03     |
+---------+-------+--------------+

ownCloud 9.1.0.11

Starting scan for user 1 out of 1 (rescan)

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 5235    | 31888 | 00:03:52     |
+---------+-------+--------------+

On my Synology DS412+

nextCloud 10 beta

Starting scan for user 1 out of 1 (rescan)

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 6611    | 38356 | 00:09:32     |
+---------+-------+--------------+

ownCloud 9.1.0.11

Starting scan for user 1 out of 1 (rescan)

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 6648    | 38570 | 00:08:39     |
+---------+-------+--------------+

Very good !!! :wink:

1 Like

@icewind Any idea for the performance drop?

I’m a little surprised at this question. I have nothing changed on my servers. I only updated from NC 9.0.53 to NC 10 Beta, nothing else. On 3 Synology servers I have the same result. At a fourth I have to look at the logs. NC now in the performance is approximately on level of OC. I have certainly believed that appropriate adjustments were made. For me someone have made modifications in this update.

Sometimes, but not everytime I get this kind of message :slight_smile:

Starting scan for user 1 out of 1 (rescan)
Exception while scanning: "files/def5872d965ed8e1f99d250c02cf0e0d" is locked
#0 /volume1/web/nextcloud/lib/private/Files/Storage/Common.php(650): OC\Lock\DBLockingProvider->acquireLock('files/def5872d9...', 1)
#1 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Common->acquireLock('FileZillaPortab...', 1, Object(OC\Lock\DBLockingProvider))
#2 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Wrapper\Wrapper->acquireLock('FileZillaPortab...', 1, Object(OC\Lock\DBLockingProvider))
#3 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Wrapper\Wrapper->acquireLock('FileZillaPortab...', 1, Object(OC\Lock\DBLockingProvider))
#4 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(140): OC\Files\Storage\Wrapper\Wrapper->acquireLock('FileZillaPortab...', 1, Object(OC\Lock\DBLockingProvider))
#5 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanFile('FileZillaPortab...', 3, '220442', Object(OC\Files\Cache\CacheEntry), true)
#6 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(377): OC\Files\Cache\Scanner->handleChildren('FileZillaPortab...', true, 3, '220442', true, 0)
#7 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '220442', true)
#8 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '219849', true)
#9 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '219459', true)
#10 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '219415', true)
#11 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '219400', true)
#12 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '196329', true)
#13 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(311): OC\Files\Cache\Scanner->scanChildren('', true, 3, 279, true)
#14 /volume1/web/nextcloud/lib/private/Files/Utils/Scanner.php(193): OC\Files\Cache\Scanner->scan('', true, 3)
#15 /volume1/web/nextcloud/apps/files/lib/Command/Scan.php(155): OC\Files\Utils\Scanner->scan('/rescan')
#16 /volume1/web/nextcloud/apps/files/lib/Command/Scan.php(223): OCA\Files\Command\Scan->scanFiles('rescan', '/rescan', false, Object(Symfony\Component\Console\Output\ConsoleOutput), false)
#17 /volume1/web/nextcloud/3rdparty/symfony/console/Command/Command.php(259): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /volume1/web/nextcloud/core/Command/Base.php(158): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(844): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(192): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(123): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 /volume1/web/nextcloud/lib/private/Console/Application.php(145): Symfony\Component\Console\Application->run(NULL, NULL)
#23 /volume1/web/nextcloud/console.php(92): OC\Console\Application->run()
#24 /volume1/web/nextcloud/occ(11): require_once('/volume1/web/ne...')
#25 {main}

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 2339    | 14516 | 00:05:16     |
+---------+-------+--------------+

Starting scan for user 1 out of 1 (rescan)
Exception while scanning: "files/fc3251a6816e8a141a1d6c40fdccff59" is locked
#0 /volume1/web/nextcloud/lib/private/Files/Storage/Common.php(650): OC\Lock\DBLockingProvider->acquireLock('files/fc3251a68...', 1)
#1 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Common->acquireLock('FreeCommanderPo...', 1, Object(OC\Lock\DBLockingProvider))
#2 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Wrapper\Wrapper->acquireLock('FreeCommanderPo...', 1, Object(OC\Lock\DBLockingProvider))
#3 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Wrapper\Wrapper->acquireLock('FreeCommanderPo...', 1, Object(OC\Lock\DBLockingProvider))
#4 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(140): OC\Files\Storage\Wrapper\Wrapper->acquireLock('FreeCommanderPo...', 1, Object(OC\Lock\DBLockingProvider))
#5 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanFile('FreeCommanderPo...', 3, '196338', Object(OC\Files\Cache\CacheEntry), true)
#6 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(377): OC\Files\Cache\Scanner->handleChildren('FreeCommanderPo...', true, 3, '196338', true, 12652036)
#7 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FreeCommanderPo...', true, 3, '196338', true)
#8 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(311): OC\Files\Cache\Scanner->scanChildren('', true, 3, 279, true)
#9 /volume1/web/nextcloud/lib/private/Files/Utils/Scanner.php(193): OC\Files\Cache\Scanner->scan('', true, 3)
#10 /volume1/web/nextcloud/apps/files/lib/Command/Scan.php(155): OC\Files\Utils\Scanner->scan('/rescan')
#11 /volume1/web/nextcloud/apps/files/lib/Command/Scan.php(223): OCA\Files\Command\Scan->scanFiles('rescan', '/rescan', false, Object(Symfony\Component\Console\Output\ConsoleOutput), false)
#12 /volume1/web/nextcloud/3rdparty/symfony/console/Command/Command.php(259): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /volume1/web/nextcloud/core/Command/Base.php(158): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(844): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(192): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(123): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /volume1/web/nextcloud/lib/private/Console/Application.php(145): Symfony\Component\Console\Application->run(NULL, NULL)
#18 /volume1/web/nextcloud/console.php(92): OC\Console\Application->run()
#19 /volume1/web/nextcloud/occ(11): require_once('/volume1/web/ne...')
#20 {main}

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 2712    | 16661 | 00:05:17     |
+---------+-------+--------------+

Starting scan for user 1 out of 1 (rescan)
Exception while scanning: "files/db4bd5dccc8756f5f9d4869f37fcb742" is locked
#0 /volume1/web/nextcloud/lib/private/Files/Storage/Common.php(650): OC\Lock\DBLockingProvider->acquireLock('files/db4bd5dcc...', 1)
#1 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Common->acquireLock('SyncthingGTK/et...', 1, Object(OC\Lock\DBLockingProvider))
#2 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Wrapper\Wrapper->acquireLock('SyncthingGTK/et...', 1, Object(OC\Lock\DBLockingProvider))
#3 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Wrapper\Wrapper->acquireLock('SyncthingGTK/et...', 1, Object(OC\Lock\DBLockingProvider))
#4 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(140): OC\Files\Storage\Wrapper\Wrapper->acquireLock('SyncthingGTK/et...', 1, Object(OC\Lock\DBLockingProvider))
#5 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanFile('SyncthingGTK/et...', 3, '244333', Object(OC\Files\Cache\CacheEntry), true)
#6 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(377): OC\Files\Cache\Scanner->handleChildren('SyncthingGTK/et...', true, 3, '244333', true, 274346)
#7 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('SyncthingGTK/et...', true, 3, '244333', true)
#8 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('SyncthingGTK/et...', true, 3, '244314', true)
#9 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('SyncthingGTK/et...', true, 3, '244300', true)
#10 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('SyncthingGTK', true, 3, '196362', true)
#11 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(311): OC\Files\Cache\Scanner->scanChildren('', true, 3, 279, true)
#12 /volume1/web/nextcloud/lib/private/Files/Utils/Scanner.php(193): OC\Files\Cache\Scanner->scan('', true, 3)
#13 /volume1/web/nextcloud/apps/files/lib/Command/Scan.php(155): OC\Files\Utils\Scanner->scan('/rescan')
#14 /volume1/web/nextcloud/apps/files/lib/Command/Scan.php(223): OCA\Files\Command\Scan->scanFiles('rescan', '/rescan', false, Object(Symfony\Component\Console\Output\ConsoleOutput), false)
#15 /volume1/web/nextcloud/3rdparty/symfony/console/Command/Command.php(259): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /volume1/web/nextcloud/core/Command/Base.php(158): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(844): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(192): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(123): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /volume1/web/nextcloud/lib/private/Console/Application.php(145): Symfony\Component\Console\Application->run(NULL, NULL)
#21 /volume1/web/nextcloud/console.php(92): OC\Console\Application->run()
#22 /volume1/web/nextcloud/occ(11): require_once('/volume1/web/ne...')
#23 {main}

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 4181    | 26616 | 01:00:05     |
+---------+-------+--------------+

Starting scan for user 1 out of 1 (rescan)
Exception while scanning: "files/1350ef4b35630a977ad51415961f46c8" is locked
#0 /volume1/web/nextcloud/lib/private/Files/Storage/Common.php(650): OC\Lock\DBLockingProvider->acquireLock('files/1350ef4b3...', 1)
#1 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Common->acquireLock('FileZillaPortab...', 1, Object(OC\Lock\DBLockingProvider))
#2 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Wrapper\Wrapper->acquireLock('FileZillaPortab...', 1, Object(OC\Lock\DBLockingProvider))
#3 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Wrapper\Wrapper->acquireLock('FileZillaPortab...', 1, Object(OC\Lock\DBLockingProvider))
#4 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(140): OC\Files\Storage\Wrapper\Wrapper->acquireLock('FileZillaPortab...', 1, Object(OC\Lock\DBLockingProvider))
#5 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanFile('FileZillaPortab...', 3, '219445', Object(OC\Files\Cache\CacheEntry), true)
#6 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(377): OC\Files\Cache\Scanner->handleChildren('FileZillaPortab...', true, 3, '219445', true, 0)
#7 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '219445', true)
#8 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '219412', true)
#9 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '219400', true)
#10 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '196329', true)
#11 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(311): OC\Files\Cache\Scanner->scanChildren('', true, 3, 279, true)
#12 /volume1/web/nextcloud/lib/private/Files/Utils/Scanner.php(193): OC\Files\Cache\Scanner->scan('', true, 3)
#13 /volume1/web/nextcloud/apps/files/lib/Command/Scan.php(155): OC\Files\Utils\Scanner->scan('/rescan')
#14 /volume1/web/nextcloud/apps/files/lib/Command/Scan.php(223): OCA\Files\Command\Scan->scanFiles('rescan', '/rescan', false, Object(Symfony\Component\Console\Output\ConsoleOutput), false)
#15 /volume1/web/nextcloud/3rdparty/symfony/console/Command/Command.php(259): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /volume1/web/nextcloud/core/Command/Base.php(158): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(844): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(192): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(123): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /volume1/web/nextcloud/lib/private/Console/Application.php(145): Symfony\Component\Console\Application->run(NULL, NULL)
#21 /volume1/web/nextcloud/console.php(92): OC\Console\Application->run()
#22 /volume1/web/nextcloud/occ(11): require_once('/volume1/web/ne...')
#23 {main}

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 2248    | 14052 | 00:05:07     |
+---------+-------+--------------+

Starting scan for user 1 out of 1 (rescan)
Exception while scanning: "files/333eabd174703963cdb13a2e343c9ef1" is locked
#0 /volume1/web/nextcloud/lib/private/Files/Storage/Common.php(650): OC\Lock\DBLockingProvider->acquireLock('files/333eabd17...', 1)
#1 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Common->acquireLock('FileZillaPortab...', 1, Object(OC\Lock\DBLockingProvider))
#2 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Wrapper\Wrapper->acquireLock('FileZillaPortab...', 1, Object(OC\Lock\DBLockingProvider))
#3 /volume1/web/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(583): OC\Files\Storage\Wrapper\Wrapper->acquireLock('FileZillaPortab...', 1, Object(OC\Lock\DBLockingProvider))
#4 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(140): OC\Files\Storage\Wrapper\Wrapper->acquireLock('FileZillaPortab...', 1, Object(OC\Lock\DBLockingProvider))
#5 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanFile('FileZillaPortab...', 3, '219955', Object(OC\Files\Cache\CacheEntry), true)
#6 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(377): OC\Files\Cache\Scanner->handleChildren('FileZillaPortab...', true, 3, '219955', true, 30915)
#7 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '219955', true)
#8 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '219842', true)
#9 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '219459', true)
#10 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '219415', true)
#11 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '219400', true)
#12 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(380): OC\Files\Cache\Scanner->scanChildren('FileZillaPortab...', true, 3, '196329', true)
#13 /volume1/web/nextcloud/lib/private/Files/Cache/Scanner.php(311): OC\Files\Cache\Scanner->scanChildren('', true, 3, 279, true)
#14 /volume1/web/nextcloud/lib/private/Files/Utils/Scanner.php(193): OC\Files\Cache\Scanner->scan('', true, 3)
#15 /volume1/web/nextcloud/apps/files/lib/Command/Scan.php(155): OC\Files\Utils\Scanner->scan('/rescan')
#16 /volume1/web/nextcloud/apps/files/lib/Command/Scan.php(223): OCA\Files\Command\Scan->scanFiles('rescan', '/rescan', false, Object(Symfony\Component\Console\Output\ConsoleOutput), false)
#17 /volume1/web/nextcloud/3rdparty/symfony/console/Command/Command.php(259): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /volume1/web/nextcloud/core/Command/Base.php(158): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(844): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(192): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /volume1/web/nextcloud/3rdparty/symfony/console/Application.php(123): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 /volume1/web/nextcloud/lib/private/Console/Application.php(145): Symfony\Component\Console\Application->run(NULL, NULL)
#23 /volume1/web/nextcloud/console.php(92): OC\Console\Application->run()
#24 /volume1/web/nextcloud/occ(11): require_once('/volume1/web/ne...')
#25 {main}

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 2318    | 14297 | 00:05:07     |
+---------+-------+--------------+

@Andy64: I took the freedom of re-formatting your post: all message text showed up far too

#BIG :wink:

due to the hashes preceeding the lines

Thanks. I try this also, but without success…:grimacing:

1 Like

Are you using redis as file-locking cache? mysql (used by default) should work as well but there are a number of reports (File is locked - how to unlock) and it seems redis runs better.

10x better.


I only have Synology servers. I don´t know anything about “redis”. In which way I can check this?

There seems to be a way to install it: http://www.iholken.com/index.php/2016/03/16/install-redis-server-and-phpredis-extension-into-synology-nas-running-dsm-6-without-bootstrapping/

NC 11 also reduced the number of DB queries a lot (not sure about the scanning process). Perhaps worth a try.

I have 8 HDDS where 4 are external usb3 drivers in my nextcloud .
I installed the latest nextcloud in my server , but when o start the files scan it takes a very long time and i have no idea why .
On the web there is nothing related to this issue on how to fix it , since owncloud and nextcloud are similar platforms , a fix for 1 can work on the other .

I am using redis server as memcache .
_I had to do a ctrl+c on the command because i was already almost 3 hours waiting without my Disk4 even start to be scanned . (usually you know that by watching the led on that HDD .

Starting scan for user 1 out of 1 (pedro)
^CInterrupted by user

+---------+--------+--------------+
| Folders | Files  | Elapsed time |
+---------+--------+--------------+
| 5922    | 263447 | 02:47:48     |
+---------+--------+--------------+

Is there any solution to speed up the process ?

Finally i let my server doing the files scan all night , the result was :

Folders : 46814
Files : 785005
Elapsed Time : 13h 15m 31s

I am trying to figure why it took so long to catalog those files in db .
If anyone knows how to speedup the process then post here how to do it .

AFAIK scan spreed has nothing to duo with redis being or not being used.

Either it is due to drives latency or slow MySQL database.
For your amount of files, I guess that database is large. Perhaps the MySQL cache is too small to handle these kind of mass calls efficiently? But just a guess, as I lack insights :wink:.

mysqltuner or similar scripts could give some hints.

If there is a difference like 20 min vs 90 min, it would probably be interesting to know how much sql-queries are needed.

Trying to fix a NC15.0.8 issue.
occ scan Performance on a dual core intel Synology DS218+ is appalling (hours and hours).
CPU usage 2%, RAM 25%

Any ideas how to speed up?