External storage app extremely slow down the system

Hello,

I was troubleshooting an issue when desktop client was replication REALLY slow in local LAN. I was unable to complete initial full sync of the client during 3 days for ~30GB data/70k files. After lot of wrong assumptions I was able to isolate the official “external storage” app with 3 mounted SMB shares to be the issue. the shares itself a not replicated. disabling the app immediately made the replication running fast.

Nextcloud version (eg, 19.0.3):
Operating system and version (QNAP QTS 4.4.3+Docker):
Apache or nginx version (Docker image 19.0.3):
CPU: Celeron J1900 @ 1.99GHz (4*cores)
RAM: 8GB

Initially posted as

My initial issue was visible when replication a directory with lot of small files, so to show the issue I created 106 small files with random context with total size of 716 kB. I made a test when I uploaded exactly the same files 3 times to my dev nextcloud instance with different setups:

  1. external storage disabled (finished 21:23, duration 44,491 msec)
  2. external storage enabled and 1 SMB share is mounted (finished 21:34 duration 70638 msec)
  3. external storage enabled and 3 SMB shares are mounted (finished 21:56 duration 135663 msec)

as a conclusion: replication of the same amount of data took 3 times longer as with external storage app disabled!

here is a grep of the client log

Additional observation: external storage with 3 active SMB mounts slow down all parts of the system, even such you would not imagine like “News” and “Mail” apps

from the full log you can see how replication (and uploading the files) is affected by external storage and the number of external storages (1/3 ext storage run logs shortened due to forum restriction - trust me I uploaded exactly the same data).

#=#=#=# Syncrun finished 2020-10-29T21:16:16Z (last step: 15701 msec, total: 17195 msec)
#=#=#=# Syncrun started 2020-10-29T21:22:07Z
#=#=#=#=# Propagation starts 2020-10-29T21:22:07Z (last step: 489 msec, total: 489 msec)
||Documents/test wo external storage|INST_NEW|Down|1604006519|5f9b3277dc7f4|0|00000281oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:22:08Z (last step: 127 msec, total: 617 msec)
#=#=#=# Syncrun started 2020-10-29T21:23:02Z
#=#=#=#=# Propagation starts 2020-10-29T21:23:05Z (last step: 2546 msec, total: 2546 msec)
21:23:05||Documents/test wo external storage/random-11158.txt|INST_NEW|Down|1599449413|73779e4dbcd88413e2c3a45946028629|4471|00000365oc52dthqts8g|4||0|0|0||||
21:23:05||Documents/test wo external storage/random-10716.txt|INST_NEW|Down|1603529913|984c0cc9c01da3712e5d5675c1a47812|3519|00000587oc52dthqts8g|4||0|0|0||||
21:23:06||Documents/test wo external storage/random-14003.txt|INST_NEW|Down|1609005316|45b70ba19ddfbe7e491a2a83f1fe10e3|3394|00000613oc52dthqts8g|4||0|0|0||||
21:23:05||Documents/test wo external storage/random-11242.txt|INST_NEW|Down|1608947719|ca21d39cca3e1e984737b323558644e7|4232|00000458oc52dthqts8g|4||0|0|0||||
21:23:06||Documents/test wo external storage/random-13160.txt|INST_NEW|Down|1606150094|b06531953c47dc6cc74ee49cf7e8c4b8|3548|00000569oc52dthqts8g|4||0|0|0||||
21:23:05||Documents/test wo external storage/random-12015.txt|INST_NEW|Down|1599918698|14ce5513c393a2c64cc6b04ddcde091a|2572|00000748oc52dthqts8g|4||0|0|0||||
21:23:07||Documents/test wo external storage/random-14403.txt|INST_NEW|Down|1602639861|80a9a011eb3897367580b42288eafe1d|2870|00000644oc52dthqts8g|4||0|0|0||||
21:23:08||Documents/test wo external storage/random-14829.txt|INST_NEW|Down|1603471699|728fe639110a2dc3cc3d4c5b45ce9a88|3934|00000506oc52dthqts8g|4||0|0|0||||
21:23:08||Documents/test wo external storage/random-15862.txt|INST_NEW|Down|1605600737|aa61021674df5a7d53e02f8d1cc68158|2597|00000679oc52dthqts8g|4||0|0|0||||
21:23:09||Documents/test wo external storage/random-16483.txt|INST_NEW|Down|1606102584|35709a0a5d9039e5c19526ccc8fea0d4|2770|00000685oc52dthqts8g|4||0|0|0||||
21:23:08||Documents/test wo external storage/random-14673.txt|INST_NEW|Down|1603819777|e8cf01a31b25219122f633c14b11fced|3519|00000601oc52dthqts8g|4||0|0|0||||
21:23:10||Documents/test wo external storage/random-22203.txt|INST_NEW|Down|1601979212|15caf91b0dfec6ec2f224d1d68e3c6b2|21638|00000286oc52dthqts8g|4||0|0|0||||
21:23:10||Documents/test wo external storage/random-22449.txt|INST_NEW|Down|1602504314|e7f3b511baeb39cd24511100804f944b|4021|00000504oc52dthqts8g|4||0|0|0||||
21:23:10||Documents/test wo external storage/random-23376.txt|INST_NEW|Down|1607289570|ebfb066f72a874e31dbb517a65cb8560|4379|00000416oc52dthqts8g|4||0|0|0||||
21:23:11||Documents/test wo external storage/random-30814.txt|INST_NEW|Down|1600072162|785553c96c0cdb71ea0d29fdf80682cf|2820|00000687oc52dthqts8g|4||0|0|0||||
21:23:11||Documents/test wo external storage/random-23736.txt|INST_NEW|Down|1602288029|79afd94d82b466c1994c145fba32d8ce|3281|00000612oc52dthqts8g|4||0|0|0||||
21:23:11||Documents/test wo external storage/random-26479.txt|INST_NEW|Down|1604493517|08b5f444d488f9738ed7d191999adf7d|4291|00000428oc52dthqts8g|4||0|0|0||||
21:23:12||Documents/test wo external storage/random-32544.txt|INST_NEW|Down|1607735069|e6d623d0df203d9fa8a09ea135cb2262|2909|00000645oc52dthqts8g|4||0|0|0||||
21:23:12||Documents/test wo external storage/random-36191.txt|INST_NEW|Down|1608805707|2ec6f7c883d7c53e0e8f4eb6b82aee16|4430|00000402oc52dthqts8g|4||0|0|0||||
21:23:13||Documents/test wo external storage/random-36596.txt|INST_NEW|Down|1602219213|9662cf088f0efcee75d0674213d56738|2408|00000754oc52dthqts8g|4||0|0|0||||
21:23:13||Documents/test wo external storage/random-37480.txt|INST_NEW|Down|1609056496|43240ce3f5f4a9f5e6fbcb6e999501e0|3439|00000610oc52dthqts8g|4||0|0|0||||
21:23:13||Documents/test wo external storage/random-42801.txt|INST_NEW|Down|1607882295|0a5f8c1b7e1c1c8ff8255454eab20329|2796|00000678oc52dthqts8g|4||0|0|0||||
21:23:13||Documents/test wo external storage/random-42635.txt|INST_NEW|Down|1607514339|2579a07ee7263941fbb669d24f0342e9|3662|00000554oc52dthqts8g|4||0|0|0||||
21:23:14||Documents/test wo external storage/random-43797.txt|INST_NEW|Down|1607244445|88c888cd88204801bdca0829eb59b79d|4283|00000426oc52dthqts8g|4||0|0|0||||
21:23:14||Documents/test wo external storage/random-46755.txt|INST_NEW|Down|1605979704|19d6413239731804d34267a391289d5c|3595|00000558oc52dthqts8g|4||0|0|0||||
21:23:14||Documents/test wo external storage/random-47438.txt|INST_NEW|Down|1606470499|17654614086118a5ba2c9aa7d141733e|3592|00000591oc52dthqts8g|4||0|0|0||||
21:23:14||Documents/test wo external storage/random-47900.txt|INST_NEW|Down|1598808457|2a1e21115177db63f7a1d6bbeda056ab|16563|00000285oc52dthqts8g|4||0|0|0||||
21:23:15||Documents/test wo external storage/random-52295.txt|INST_NEW|Down|1609017086|9ac4cb4515fd72e1b2f0fdd60e37d640|4466|00000397oc52dthqts8g|4||0|0|0||||
21:23:15||Documents/test wo external storage/random-55329.txt|INST_NEW|Down|1602101549|72ed81a109a61f38b7a1d178eb6c9e30|4115|00000487oc52dthqts8g|4||0|0|0||||
21:23:15||Documents/test wo external storage/random-5331.txt|INST_NEW|Down|1607515217|b83eaf532b97a67644f5bae7ded8cdfa|4161|00000461oc52dthqts8g|4||0|0|0||||
21:23:16||Documents/test wo external storage/random-56163.txt|INST_NEW|Down|1607534713|e22efc469ac758be00b74b8760d9e505|4621|00000335oc52dthqts8g|4||0|0|0||||
21:23:16||Documents/test wo external storage/random-55348.txt|INST_NEW|Down|1606723834|d3632561c53d02ff6ca703fabb30d3fc|3451|00000611oc52dthqts8g|4||0|0|0||||
21:23:16||Documents/test wo external storage/random-58595.txt|INST_NEW|Down|1602516036|d61d818d84647a155e3c481a91e793b5|5078|00000287oc52dthqts8g|4||0|0|0||||
21:23:16||Documents/test wo external storage/random-5621.txt|INST_NEW|Down|1602005318|26c282c61a9e13b3a1f7488e7c92eb9b|3803|00000534oc52dthqts8g|4||0|0|0||||
21:23:17||Documents/test wo external storage/random-58703.txt|INST_NEW|Down|1601634262|2cbb3f36248d1a5e400c906d394281ec|4711|00000331oc52dthqts8g|4||0|0|0||||
21:23:17||Documents/test wo external storage/random-60148.txt|INST_NEW|Down|1599658160|56da39a7125ae4695452d2e49e9eb0e2|2861|00000670oc52dthqts8g|4||0|0|0||||
21:23:17||Documents/test wo external storage/random-60504.txt|INST_NEW|Down|1600164507|f634e88a398d7df708b56ad61e0134b1|3210|00000630oc52dthqts8g|4||0|0|0||||
21:23:18||Documents/test wo external storage/random-60515.txt|INST_NEW|Down|1607881054|f82278923b15b3c8eef6e19ef43562df|29105|00000289oc52dthqts8g|4||0|0|0||||
21:23:18||Documents/test wo external storage/random-60971.txt|INST_NEW|Down|1602458254|ae7e3d3f24da40da1ca9cf5048c63103|5054|00000292oc52dthqts8g|4||0|0|0||||
21:23:19||Documents/test wo external storage/random-62350.txt|INST_NEW|Down|1602677003|97f5862cbffb0c98113bd7771719898f|2869|00000654oc52dthqts8g|4||0|0|0||||
21:23:19||Documents/test wo external storage/random-63480.txt|INST_NEW|Down|1602805775|d596dbc258921a17852fb8a0fbdc615f|2986|00000643oc52dthqts8g|4||0|0|0||||
21:23:20||Documents/test wo external storage/random-63890.txt|INST_NEW|Down|1608745343|c79b19ed145781a2ae4130d7c9254e71|2942|00000765oc52dthqts8g|4||0|0|0||||
21:23:20||Documents/test wo external storage/random-6411.txt|INST_NEW|Down|1599339091|64b43536a8ab967f23c5d75bd7d0decd|4251|00000444oc52dthqts8g|4||0|0|0||||
21:23:19||Documents/test wo external storage/random-6240.txt|INST_NEW|Down|1601979756|0772c1400d61d1ce9cc76a125c84aaf3|4774|00000300oc52dthqts8g|4||0|0|0||||
21:23:20||Documents/test wo external storage/random-64774.txt|INST_NEW|Down|1600697319|a6bcf5fc1893c89da0b53721f035d118|3251|00000660oc52dthqts8g|4||0|0|0||||
21:23:21||Documents/test wo external storage/random-6822.txt|INST_NEW|Down|1599803540|a6b6b090668d7a33c1e34def39eeed7b|3935|00000507oc52dthqts8g|4||0|0|0||||
21:23:21||Documents/test wo external storage/random-69484.txt|INST_NEW|Down|1605946761|e59ed4d1ae5e1d18a8ccc6c60834038d|3128|00000636oc52dthqts8g|4||0|0|0||||
21:23:21||Documents/test wo external storage/random-65438.txt|INST_NEW|Down|1600969036|a527593a2b2cbf380f2d3d90e4aff9c0|4075|00000482oc52dthqts8g|4||0|0|0||||
21:23:22||Documents/test wo external storage/random-70002.txt|INST_NEW|Down|1604304198|c078589bdbcec533a574ac9c335deccd|3916|00000521oc52dthqts8g|4||0|0|0||||
21:23:23||Documents/test wo external storage/random-73397.txt|INST_NEW|Down|1598754397|b32513b6755749bb58bf3cedbefa4fe9|3728|00000543oc52dthqts8g|4||0|0|0||||
21:23:24||Documents/test wo external storage/random-73508.txt|INST_NEW|Down|1606181838|8c9245ccef33ac3983b91b43874d4d2b|4593|00000344oc52dthqts8g|4||0|0|0||||
21:23:24||Documents/test wo external storage/random-76461.txt|INST_NEW|Down|1604997212|38357383a75bc088c5ba27174a6491aa|4732|00000321oc52dthqts8g|4||0|0|0||||
21:23:24||Documents/test wo external storage/random-75433.txt|INST_NEW|Down|1604176362|01577417bb49bbe2bcc0d735c4a2fe3f|48922|00000283oc52dthqts8g|4||0|0|0||||
21:23:25||Documents/test wo external storage/random-77423.txt|INST_NEW|Down|1601100927|a851a700e2cdbe5bd32cd7f725bd6a37|4487|00000354oc52dthqts8g|4||0|0|0||||
21:23:25||Documents/test wo external storage/random-76482.txt|INST_NEW|Down|1603414240|363dadd7cb3fa5f08d9aa22e8a034cc9|5071|00000290oc52dthqts8g|4||0|0|0||||
21:23:25||Documents/test wo external storage/random-78694.txt|INST_NEW|Down|1599574228|d46d5a5fc6fddc9070c7c0d548924650|3781|00000537oc52dthqts8g|4||0|0|0||||
21:23:26||Documents/test wo external storage/random-82920.txt|INST_NEW|Down|1599479019|ab470d4f12e88d1494252a7392c41abd|3644|00000568oc52dthqts8g|4||0|0|0||||
21:23:26||Documents/test wo external storage/random-82599.txt|INST_NEW|Down|1602440672|38fbf4a23b4e96eca99fbfd83ad1b297|4129|00000478oc52dthqts8g|4||0|0|0||||
21:23:26||Documents/test wo external storage/random-83157.txt|INST_NEW|Down|1600938871|9b720d97ede6b7dc98ffca2a975c72e2|3499|00000609oc52dthqts8g|4||0|0|0||||
21:23:28||Documents/test wo external storage/random-8360.txt|INST_NEW|Down|1603864118|dae5e9d033b2d4fd420f73f5842f5cc9|3136|00000686oc52dthqts8g|4||0|0|0||||
21:23:29||Documents/test wo external storage/random-83762.txt|INST_NEW|Down|1601969311|f61ec291aaec2618c80bca93e56aaefa|3204|00000624oc52dthqts8g|4||0|0|0||||
21:23:29||Documents/test wo external storage/random-89056.txt|INST_NEW|Down|1601589210|f97feaffd402a5bd9434558db4042376|4326|00000421oc52dthqts8g|4||0|0|0||||
21:23:29||Documents/test wo external storage/random-91273.txt|INST_NEW|Down|1606540115|c05efeb42078d140cadb1b04b922971f|13695|00000291oc52dthqts8g|4||0|0|0||||
21:23:29||Documents/test wo external storage/random-90705.txt|INST_NEW|Down|1600142845|591f6aa9b54f8daf6eab6a1edf809f05|4727|00000326oc52dthqts8g|4||0|0|0||||
21:23:29||Documents/test wo external storage/random-87723.txt|INST_NEW|Down|1606462092|c17108107b2c7b4d8a158133bfeb8eed|3185|00000642oc52dthqts8g|4||0|0|0||||
21:23:30||Documents/test wo external storage/random-91627.txt|INST_NEW|Down|1599747088|668ca7355a60579d6926c0d3e0b63b93|4872|00000288oc52dthqts8g|4||0|0|0||||
21:23:31||Documents/test wo external storage/random-9831.txt|INST_NEW|Down|1603883235|16d9af6830ce4c08d398900ca51a3e12|3602|00000557oc52dthqts8g|4||0|0|0||||
21:23:30||Documents/test wo external storage/random-96831.txt|INST_NEW|Down|1599814611|490cfe8934ce9397d3cb5aa844683c3d|304827|00000284oc52dthqts8g|4||0|0|0||||
21:23:31||Documents/test wo external storage/random-98521.txt|INST_NEW|Down|1603187963|cc30f8aec755f43487b4dcea0915ae62|2266|00000714oc52dthqts8g|4||0|0|0||||
21:23:31||Documents/test wo external storage/random-99876.txt|INST_NEW|Down|1602548844|d73c7cc5efbc04440b4dd1f64a0b91e0|4044|00000495oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:23:33Z (last step: 28197 msec, total: 30743 msec)
#=#=#=# Syncrun started 2020-10-29T21:23:42Z
#=#=#=#=# Propagation starts 2020-10-29T21:23:45Z (last step: 3089 msec, total: 3089 msec)
21:23:45||Documents/test wo external storage/random-13695.txt|INST_NEW|Down|1606232612|a62b864bc2e46cb24905f13eeb3bbf6c|1582|00000928oc52dthqts8g|4||0|0|0||||
21:23:45||Documents/test wo external storage/random-13779.txt|INST_NEW|Down|1599542306|2b366187707fd3ab2318f738dd1ded24|1406|00000925oc52dthqts8g|4||0|0|0||||
21:23:45||Documents/test wo external storage/random-17455.txt|INST_NEW|Down|1600227510|c87ded5d4c956da85e25cd934804a025|1174|00000988oc52dthqts8g|4||0|0|0||||
21:23:45||Documents/test wo external storage/random-22397.txt|INST_NEW|Down|1599001577|55109fd063356fea13e5792fb80b3d4f|1060|00001001oc52dthqts8g|4||0|0|0||||
21:23:45||Documents/test wo external storage/random-17977.txt|INST_NEW|Down|1604614816|24987bc5da258756d7d845305402b327|1961|00000870oc52dthqts8g|4||0|0|0||||
21:23:45||Documents/test wo external storage/random-23433.txt|INST_NEW|Down|1606233500|44e15876af71a07e080303d9025217d9|1430|00000910oc52dthqts8g|4||0|0|0||||
21:23:46||Documents/test wo external storage/random-23871.txt|INST_NEW|Down|1602860678|fd38e7e47f2741e09dd3eb24a7e6bddd|1186|00000948oc52dthqts8g|4||0|0|0||||
21:23:47||Documents/test wo external storage/random-27669.txt|INST_NEW|Down|1601800774|c7403a74c5b08e0024c9d73d51db4b5b|1151|00001103oc52dthqts8g|4||0|0|0||||
21:23:48||Documents/test wo external storage/random-33708.txt|INST_NEW|Down|1607232021|f1fb85e0846a7a0ef5c8bed1af21f318|2234|00000882oc52dthqts8g|4||0|0|0||||
21:23:48||Documents/test wo external storage/random-33911.txt|INST_NEW|Down|1599192035|3c7d2f472f102f1eaae9eee516705fe8|2167|00001116oc52dthqts8g|4||0|0|0||||
21:23:48||Documents/test wo external storage/random-34663.txt|INST_NEW|Down|1604770577|2475adbf2e2095061793b7b6738c6626|1126|00001150oc52dthqts8g|4||0|0|0||||
21:23:48||Documents/test wo external storage/random-36286.txt|INST_NEW|Down|1607719757|7ebad019d95cd9792ddeb7fe5966c533|1393|00000943oc52dthqts8g|4||0|0|0||||
21:23:49||Documents/test wo external storage/random-45429.txt|INST_NEW|Down|1607677346|024a589994b83f92f3bd0bbbca32a4c8|1373|00000985oc52dthqts8g|4||0|0|0||||
21:23:48||Documents/test wo external storage/random-44051.txt|INST_NEW|Down|1608788182|dcd02576c078055f69953c5838469b7c|1849|00000871oc52dthqts8g|4||0|0|0||||
21:23:49||Documents/test wo external storage/random-47041.txt|INST_NEW|Down|1608609560|18564ec6836752e7a91889ad798e84af|1585|00000890oc52dthqts8g|4||0|0|0||||
21:23:50||Documents/test wo external storage/random-50513.txt|INST_NEW|Down|1604051847|351d1f0e5d22819c897d0bc59c7df0a4|2040|00000834oc52dthqts8g|4||0|0|0||||
21:23:50||Documents/test wo external storage/random-5593.txt|INST_NEW|Down|1603324410|bf234231c8d542b766252f23e1fd9fc6|1718|00000916oc52dthqts8g|4||0|0|0||||
21:23:50||Documents/test wo external storage/random-56666.txt|INST_NEW|Down|1603655959|0cbc28f184c1205064f8bf2bf32ecd79|1907|00000873oc52dthqts8g|4||0|0|0||||
21:23:50||Documents/test wo external storage/random-56207.txt|INST_NEW|Down|1605764253|16838f80b3465535ab850c31f488e43f|2731|00000813oc52dthqts8g|4||0|0|0||||
21:23:50||Documents/test wo external storage/random-58150.txt|INST_NEW|Down|1608398729|b74eac270665a0b1b59bbf37742bb9cc|2210|00000780oc52dthqts8g|4||0|0|0||||
21:23:50||Documents/test wo external storage/random-56466.txt|INST_NEW|Down|1609024694|b0993348613b7ed35805bffd5965429c|1987|00000832oc52dthqts8g|4||0|0|0||||
21:23:51||Documents/test wo external storage/random-6801.txt|INST_NEW|Down|1605921753|b0ee3ad3668f7ce45c629146e7fa469e|1106|00001038oc52dthqts8g|4||0|0|0||||
21:23:51||Documents/test wo external storage/random-61041.txt|INST_NEW|Down|1608708574|7f4b1ff4aec596d05602f03bd3ba142d|1678|00000893oc52dthqts8g|4||0|0|0||||
21:23:51||Documents/test wo external storage/random-62172.txt|INST_NEW|Down|1600622795|9939a9d285e0cdac666bee6985714708|2205|00000774oc52dthqts8g|4||0|0|0||||
21:23:52||Documents/test wo external storage/random-78507.txt|INST_NEW|Down|1601137156|f289643e4475a0d8c0b057902418ea20|1199|00000946oc52dthqts8g|4||0|0|0||||
21:23:52||Documents/test wo external storage/random-89228.txt|INST_NEW|Down|1604229575|fbff283eb1b6af3e302a422fe82f4edc|1900|00000853oc52dthqts8g|4||0|0|0||||
21:23:52||Documents/test wo external storage/random-818.txt|INST_NEW|Down|1608285193|bb21834f30ba1994ea67c08f249fde86|1231|00000947oc52dthqts8g|4||0|0|0||||
21:23:52||Documents/test wo external storage/random-88941.txt|INST_NEW|Down|1605398483|e07e450689338fed6c7b9631a192c37e|1246|00000940oc52dthqts8g|4||0|0|0||||
21:23:52||Documents/test wo external storage/random-85449.txt|INST_NEW|Down|1601589744|3525981bc7fb038425a0ee128e353776|1040|00001068oc52dthqts8g|4||0|0|0||||
21:23:53||Documents/test wo external storage/random-90147.txt|INST_NEW|Down|1604666774|2e2e81e704b7a0597109fedba7c55704|2658|00000839oc52dthqts8g|4||0|0|0||||
21:23:54||Documents/test wo external storage/random-91442.txt|INST_NEW|Down|1604227815|63964535a11d59179490b2b10dbe676d|1381|00000932oc52dthqts8g|4||0|0|0||||
21:23:54||Documents/test wo external storage/random-90728.txt|INST_NEW|Down|1605270674|b2f2124563a215c8b826e4fa40ef3fbf|1119|00000982oc52dthqts8g|4||0|0|0||||
21:23:54||Documents/test wo external storage/random-90160.txt|INST_NEW|Down|1600481662|9ed5ad1c739a601ffc49e0c9648cc7f8|1846|00000896oc52dthqts8g|4||0|0|0||||
21:23:54||Documents/test wo external storage/random-93732.txt|INST_NEW|Down|1603050865|326e023fb4e47077b805f29d8fa6c404|2252|00000798oc52dthqts8g|4||0|0|0||||
21:23:54||Documents/test wo external storage/random-95756.txt|INST_NEW|Down|1602951214|f69ecd42bb4d275a3438240658758f71|1711|00000891oc52dthqts8g|4||0|0|0||||
21:23:54||Documents/test wo external storage/random-99959.txt|INST_NEW|Down|1599396865|81e86aa1f63861c18abf933848ae8b99|1547|00000977oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:23:56Z (last step: 10658 msec, total: 13748 msec)
#=#=#=# Syncrun started 2020-10-29T21:24:02Z
#=#=#=#=# Propagation starts 2020-10-29T21:24:02Z (last step: 408 msec, total: 408 msec)
#=#=#=# Syncrun finished 2020-10-29T21:24:02Z (last step: 76 msec, total: 485 msec)
#=#=#=# Syncrun started 2020-10-29T21:31:32Z
#=#=#=#=# Propagation starts 2020-10-29T21:31:32Z (last step: 505 msec, total: 505 msec)
#=#=#=# Syncrun finished 2020-10-29T21:31:32Z (last step: 52 msec, total: 558 msec)
#=#=#=# Syncrun started 2020-10-29T21:32:35Z
#=#=#=#=# Propagation starts 2020-10-29T21:32:39Z (last step: 3317 msec, total: 3317 msec)
||Documents/test with 1 external storage|INST_NEW|Down|1604007156|5f9b34f521c70|0|00001381oc52dthqts8g|4||0|0|0||||
21:32:39||Documents/test with 1 external storage/random-10716.txt|INST_NEW|Down|1603529913|473c805d8810f131cc48ee1bdbc859cd|3519|00001493oc52dthqts8g|4||0|0|0||||
..
21:33:44||Documents/test with 1 external storage/random-99959.txt|INST_NEW|Down|1599396865|3bfb222f4f5e4653253253f5f46f1c52|1547|00002042oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:33:48Z (last step: 9513 msec, total: 11791 msec)
#=#=#=# Syncrun started 2020-10-29T21:33:53Z
#=#=#=#=# Propagation starts 2020-10-29T21:33:56Z (last step: 2812 msec, total: 2812 msec)
21:33:56||Documents/test with 1 external storage/random-23871.txt|INST_NEW|Down|1602860678|c894c21304637be35e15362340ca9442|1186|00002253oc52dthqts8g|4||0|0|0||||
21:33:56||Documents/test with 1 external storage/random-22397.txt|INST_NEW|Down|1599001577|448bd10b842a5f12df62565437105e7e|1060|00002121oc52dthqts8g|4||0|0|0||||
21:33:56||Documents/test with 1 external storage/random-6801.txt|INST_NEW|Down|1605921753|6102b1097d9b66c08677b6f2359e6dda|1106|00002115oc52dthqts8g|4||0|0|0||||
21:33:56||Documents/test with 1 external storage/random-56207.txt|INST_NEW|Down|1605764253|39cb5681f637c8e0593092611ad1ae6d|2731|00002196oc52dthqts8g|4||0|0|0||||
21:33:57||Documents/test with 1 external storage/random-85449.txt|INST_NEW|Down|1601589744|708b6849d2e2932bec87dd1e26150360|1040|00002205oc52dthqts8g|4||0|0|0||||
21:33:56||Documents/test with 1 external storage/random-27669.txt|INST_NEW|Down|1601800774|682fa647a154a3d47e38bcb67edf21e4|1151|00002127oc52dthqts8g|4||0|0|0||||
21:33:56||Documents/test with 1 external storage/random-78507.txt|INST_NEW|Down|1601137156|8a5125539443973fbc8b8e793b003375|1199|00002188oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:34:00Z (last step: 4409 msec, total: 7221 msec)
#=#=#=# Syncrun started 2020-10-29T21:34:05Z
#=#=#=#=# Propagation starts 2020-10-29T21:34:10Z (last step: 5192 msec, total: 5192 msec)
21:34:10||Documents/test with 1 external storage/random-12015.txt|INST_NEW|Down|1599918698|4c7106c589d2ff2aa2373e13928c3281|2572|00002343oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:34:11Z (last step: 893 msec, total: 6086 msec)
#=#=#=# Syncrun started 2020-10-29T21:34:15Z
#=#=#=#=# Propagation starts 2020-10-29T21:34:16Z (last step: 838 msec, total: 838 msec)
#=#=#=# Syncrun finished 2020-10-29T21:34:16Z (last step: 37 msec, total: 875 msec)
#=#=#=# Syncrun started 2020-10-29T21:46:02Z
#=#=#=#=# Propagation starts 2020-10-29T21:46:04Z (last step: 1199 msec, total: 1199 msec)
#=#=#=# Syncrun finished 2020-10-29T21:46:04Z (last step: 43 msec, total: 1242 msec)
#=#=#=# Syncrun started 2020-10-29T21:47:02Z
#=#=#=#=# Propagation starts 2020-10-29T21:47:06Z (last step: 3409 msec, total: 3409 msec)
||Documents/test with 3 external storages|INST_NEW|Down|1609017086|5f9b3857d9aa6|0|00002452oc52dthqts8g|4||0|0|0||||
21:47:06||Documents/test with 3 external storages/random-11158.txt|INST_NEW|Down|1599449413|15ab0dbad7607d9adc46a5254942fe67|4471|00002463oc52dthqts8g|4||0|0|0||||
21:47:06||Documents/test with 3 external storages/random-22203.txt|INST_NEW|Down|1601979212|9aad44aea3dbb694975d73ae03e1b602|21638|00002455oc52dthqts8g|4||0|0|0||||
21:47:06||Documents/test with 3 external storages/random-58595.txt|INST_NEW|Down|1602516036|44a3c773ae9ff755030dd87794132fde|5078|00002454oc52dthqts8g|4||0|0|0||||
21:47:07||Documents/test with 3 external storages/random-60515.txt|INST_NEW|Down|1607881054|339a28cf2af568cb1ca96956f09c1d3d|29105|00002459oc52dthqts8g|4||0|0|0||||
21:47:08||Documents/test with 3 external storages/random-6240.txt|INST_NEW|Down|1601979756|600cb965ee0f324fc9304ee5d8d44726|4774|00002465oc52dthqts8g|4||0|0|0||||
21:47:09||Documents/test with 3 external storages/random-75433.txt|INST_NEW|Down|1604176362|ce35eb55e49985384415edb75e184b81|48922|00002456oc52dthqts8g|4||0|0|0||||
21:47:06||Documents/test with 3 external storages/random-52295.txt|INST_NEW|Down|1609017086|f9ced3122b63e0b5fbb202918732603a|4466|00002466oc52dthqts8g|4||0|0|0||||
21:47:06||Documents/test with 3 external storages/random-58703.txt|INST_NEW|Down|1601634262|1737a9a4bcaa09ecfaf3d80bdbd8cbcd|4711|00002461oc52dthqts8g|4||0|0|0||||
21:47:09||Documents/test with 3 external storages/random-73508.txt|INST_NEW|Down|1606181838|3189823737919ebfa2b763144226641a|4593|00002467oc52dthqts8g|4||0|0|0||||
21:47:06||Documents/test with 3 external storages/random-56163.txt|INST_NEW|Down|1607534713|9c3dbe6954269979c5ad7fad759217dc|4621|00002462oc52dthqts8g|4||0|0|0||||
21:47:10||Documents/test with 3 external storages/random-76482.txt|INST_NEW|Down|1603414240|964a16c5bac0fbe5718e3190c70cb865|5071|00002457oc52dthqts8g|4||0|0|0||||
21:47:13||Documents/test with 3 external storages/random-96831.txt|INST_NEW|Down|1599814611|1e7480bbc3c7e407a100e7daa4e1cc11|304827|00002453oc52dthqts8g|4||0|0|0||||
21:47:12||Documents/test with 3 external storages/random-91273.txt|INST_NEW|Down|1606540115|0758cd4336640521e620de427a10ae01|13695|00002458oc52dthqts8g|4||0|0|0||||
21:47:11||Documents/test with 3 external storages/random-90705.txt|INST_NEW|Down|1600142845|dd7a8ff9465ce4e19ccac6c57ef89765|4727|00002460oc52dthqts8g|4||0|0|0||||
21:47:12||Documents/test with 3 external storages/random-91627.txt|INST_NEW|Down|1599747088|da80d000ca2fdf5521c3e2ddff057e29|4872|00002464oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:47:17Z (last step: 11431 msec, total: 14841 msec)
#=#=#=# Syncrun started 2020-10-29T21:47:23Z
#=#=#=#=# Propagation starts 2020-10-29T21:47:26Z (last step: 2993 msec, total: 2993 msec)
21:47:27||Documents/test with 3 external storages/random-22449.txt|INST_NEW|Down|1602504314|35695744af17ec692108e9b2fc21c280|4021|00002512oc52dthqts8g|4||0|0|0||||
21:47:27||Documents/test with 3 external storages/random-47900.txt|INST_NEW|Down|1598808457|ac4c1079ac515bfd11cad66e3d3e6f8e|16563|00002478oc52dthqts8g|4||0|0|0||||
21:47:27||Documents/test with 3 external storages/random-23376.txt|INST_NEW|Down|1607289570|6363dd754dbcbb019d1b9a3c2e90a791|4379|00002487oc52dthqts8g|4||0|0|0||||
21:47:28||Documents/test with 3 external storages/random-5331.txt|INST_NEW|Down|1607515217|309b724c89941576576e6398a5ec2b85|4161|00002480oc52dthqts8g|4||0|0|0||||
21:47:27||Documents/test with 3 external storages/random-26479.txt|INST_NEW|Down|1604493517|3afdde54662666bdc03d74d181a720bc|4291|00002474oc52dthqts8g|4||0|0|0||||
21:47:28||Documents/test with 3 external storages/random-55329.txt|INST_NEW|Down|1602101549|ec39f60d8ee0b3e56a2bba3b60106511|4115|00002496oc52dthqts8g|4||0|0|0||||
21:47:27||Documents/test with 3 external storages/random-11242.txt|INST_NEW|Down|1608947719|2c16657ab94e1cd038bcd94ba1d20e4f|4232|00002494oc52dthqts8g|4||0|0|0||||
21:47:29||Documents/test with 3 external storages/random-5621.txt|INST_NEW|Down|1602005318|7f037ec321ce0c8c767e931b8f7e8943|3803|00002493oc52dthqts8g|4||0|0|0||||
21:47:30||Documents/test with 3 external storages/random-60971.txt|INST_NEW|Down|1602458254|bf19f4904d473257d6526db132670438|5054|00002491oc52dthqts8g|4||0|0|0||||
21:47:32||Documents/test with 3 external storages/random-73397.txt|INST_NEW|Down|1598754397|0a79187ae229afaa6017e059469e355b|3728|00002511oc52dthqts8g|4||0|0|0||||
21:47:31||Documents/test with 3 external storages/random-6411.txt|INST_NEW|Down|1599339091|76db86a0d2790e0aefd9447cbd644cb6|4251|00002477oc52dthqts8g|4||0|0|0||||
21:47:27||Documents/test with 3 external storages/random-36191.txt|INST_NEW|Down|1608805707|03836fb3acdd5325ed270c6ba67005f9|4430|00002481oc52dthqts8g|4||0|0|0||||
21:47:33||Documents/test with 3 external storages/random-82599.txt|INST_NEW|Down|1602440672|e73657f6510902ca275b786ca454569e|4129|00002482oc52dthqts8g|4||0|0|0||||
21:47:31||Documents/test with 3 external storages/random-65438.txt|INST_NEW|Down|1600969036|3feec1a68b05dc7a9426ef854c63824d|4075|00002489oc52dthqts8g|4||0|0|0||||
21:47:34||Documents/test with 3 external storages/random-89056.txt|INST_NEW|Down|1601589210|179fc4c2600a300f72d275b9e72364b0|4326|00002490oc52dthqts8g|4||0|0|0||||
21:47:33||Documents/test with 3 external storages/random-77423.txt|INST_NEW|Down|1601100927|64b1219b62c57eee0debec25a62c9b87|4487|00002475oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:47:38Z (last step: 11865 msec, total: 14859 msec)
#=#=#=# Syncrun started 2020-10-29T21:47:45Z
#=#=#=#=# Propagation starts 2020-10-29T21:47:48Z (last step: 3434 msec, total: 3434 msec)
21:47:48||Documents/test with 3 external storages/random-13160.txt|INST_NEW|Down|1606150094|2ec8bf8683162f68d4548aef04d91e15|3548|00002539oc52dthqts8g|4||0|0|0||||
21:47:48||Documents/test with 3 external storages/random-46755.txt|INST_NEW|Down|1605979704|fcddc59bf2bf827c5d29bc0b5bd5b3de|3595|00002524oc52dthqts8g|4||0|0|0||||
21:47:48||Documents/test with 3 external storages/random-14673.txt|INST_NEW|Down|1603819777|5d3fe90e8fb6bb7b23cc845acf056bdd|3519|00002547oc52dthqts8g|4||0|0|0||||
21:47:48||Documents/test with 3 external storages/random-23736.txt|INST_NEW|Down|1602288029|25c87c19dd5c0bfc168e42501782e6e6|3281|00002558oc52dthqts8g|4||0|0|0||||
21:47:48||Documents/test with 3 external storages/random-43797.txt|INST_NEW|Down|1607244445|5d2e87519d3f4de8d0ceeeebb428fef8|4283|00002523oc52dthqts8g|4||0|0|0||||
21:47:51||Documents/test with 3 external storages/random-6822.txt|INST_NEW|Down|1599803540|e7a11de970de8dc957bbc41bcf18de41|3935|00002515oc52dthqts8g|4||0|0|0||||

...

21:50:10||Documents/test with 3 external storages/random-22397.txt|INST_NEW|Down|1599001577|39e285c3e3ee4c4342dc8ca20f1cab7b|1060|00003083oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:50:13Z (last step: 2793 msec, total: 6545 msec)
#=#=#=# Syncrun started 2020-10-29T21:55:54Z
#=#=#=#=# Propagation starts 2020-10-29T21:55:59Z (last step: 4548 msec, total: 4548 msec)
21:55:59||Documents/test with 3 external storages/random-13695.txt|INST_NEW|Down|1606232612|44cdc55a8a1f12baf5e9d60e6cf22022|1582|00003342oc52dthqts8g|4||0|0|0||||
21:55:59||Documents/test with 3 external storages/random-27669.txt|INST_NEW|Down|1601800774|2a8b5343cc15de97d25acc0fb5e8f692|1151|00003113oc52dthqts8g|4||0|0|0||||
21:55:59||Documents/test with 3 external storages/random-60504.txt|INST_NEW|Down|1600164507|ead2e1e7be7d392d2119dbfc8ac61dc7|3210|00003109oc52dthqts8g|4||0|0|0||||
21:55:59||Documents/test with 3 external storages/random-98521.txt|INST_NEW|Down|1603187963|93885afccea3d1e9094c7544b4e057bf|2266|00003181oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:56:02Z (last step: 3235 msec, total: 7783 msec)
#=#=#=# Syncrun started 2020-10-29T21:57:41Z
#=#=#=#=# Propagation starts 2020-10-29T21:57:42Z (last step: 1358 msec, total: 1358 msec)
#=#=#=# Syncrun finished 2020-10-29T21:57:42Z (last step: 64 msec, total: 1423 msec)
2 Likes

Got the same problems with smb, no matter if nextcloud 19 or 20

2 Likes

Thank you for identifying this, @wwe

Can report that I see the same behaviour with v20.0.7 and External Storage Support v1.11.1 - with a local mounted volume, in my case.

1 Like

+1

Using v21.0.0 (docker) and External Storage Support v1.12 running on a Raspberry Pi 3B+. I’m also mounting an ext4 volume from a local USB drive. MySQL as database backend.

1 Like

Same issue here. Nextcloud 20.0.8.

SMB External storage, syncing to the client is taking an insanely long time.

1 Like

What are the advice here to maybe mount the share locally at the server, and then serve it as a local folder from NextCloud? I also tend to face a performance impact with the External Storage in my test-setup.

Same issue from version 20 to the current version 22