Hello,
I was troubleshooting an issue when desktop client was replication REALLY slow in local LAN. I was unable to complete initial full sync of the client during 3 days for ~30GB data/70k files. After lot of wrong assumptions I was able to isolate the official “external storage” app with 3 mounted SMB shares to be the issue. the shares itself a not replicated. disabling the app immediately made the replication running fast.
Nextcloud version (eg, 19.0.3):
Operating system and version (QNAP QTS 4.4.3+Docker):
Apache or nginx version (Docker image 19.0.3):
CPU: Celeron J1900 @ 1.99GHz (4*cores)
RAM: 8GB
Initially posted as
My initial issue was visible when replication a directory with lot of small files, so to show the issue I created 106 small files with random context with total size of 716 kB. I made a test when I uploaded exactly the same files 3 times to my dev nextcloud instance with different setups:
- external storage disabled (finished 21:23, duration 44,491 msec)
- external storage enabled and 1 SMB share is mounted (finished 21:34 duration 70638 msec)
- external storage enabled and 3 SMB shares are mounted (finished 21:56 duration 135663 msec)
as a conclusion: replication of the same amount of data took 3 times longer as with external storage app disabled!
here is a grep of the client log
Additional observation: external storage with 3 active SMB mounts slow down all parts of the system, even such you would not imagine like “News” and “Mail” apps
from the full log you can see how replication (and uploading the files) is affected by external storage and the number of external storages (1/3 ext storage run logs shortened due to forum restriction - trust me I uploaded exactly the same data).
#=#=#=# Syncrun finished 2020-10-29T21:16:16Z (last step: 15701 msec, total: 17195 msec)
#=#=#=# Syncrun started 2020-10-29T21:22:07Z
#=#=#=#=# Propagation starts 2020-10-29T21:22:07Z (last step: 489 msec, total: 489 msec)
||Documents/test wo external storage|INST_NEW|Down|1604006519|5f9b3277dc7f4|0|00000281oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:22:08Z (last step: 127 msec, total: 617 msec)
#=#=#=# Syncrun started 2020-10-29T21:23:02Z
#=#=#=#=# Propagation starts 2020-10-29T21:23:05Z (last step: 2546 msec, total: 2546 msec)
21:23:05||Documents/test wo external storage/random-11158.txt|INST_NEW|Down|1599449413|73779e4dbcd88413e2c3a45946028629|4471|00000365oc52dthqts8g|4||0|0|0||||
21:23:05||Documents/test wo external storage/random-10716.txt|INST_NEW|Down|1603529913|984c0cc9c01da3712e5d5675c1a47812|3519|00000587oc52dthqts8g|4||0|0|0||||
21:23:06||Documents/test wo external storage/random-14003.txt|INST_NEW|Down|1609005316|45b70ba19ddfbe7e491a2a83f1fe10e3|3394|00000613oc52dthqts8g|4||0|0|0||||
21:23:05||Documents/test wo external storage/random-11242.txt|INST_NEW|Down|1608947719|ca21d39cca3e1e984737b323558644e7|4232|00000458oc52dthqts8g|4||0|0|0||||
21:23:06||Documents/test wo external storage/random-13160.txt|INST_NEW|Down|1606150094|b06531953c47dc6cc74ee49cf7e8c4b8|3548|00000569oc52dthqts8g|4||0|0|0||||
21:23:05||Documents/test wo external storage/random-12015.txt|INST_NEW|Down|1599918698|14ce5513c393a2c64cc6b04ddcde091a|2572|00000748oc52dthqts8g|4||0|0|0||||
21:23:07||Documents/test wo external storage/random-14403.txt|INST_NEW|Down|1602639861|80a9a011eb3897367580b42288eafe1d|2870|00000644oc52dthqts8g|4||0|0|0||||
21:23:08||Documents/test wo external storage/random-14829.txt|INST_NEW|Down|1603471699|728fe639110a2dc3cc3d4c5b45ce9a88|3934|00000506oc52dthqts8g|4||0|0|0||||
21:23:08||Documents/test wo external storage/random-15862.txt|INST_NEW|Down|1605600737|aa61021674df5a7d53e02f8d1cc68158|2597|00000679oc52dthqts8g|4||0|0|0||||
21:23:09||Documents/test wo external storage/random-16483.txt|INST_NEW|Down|1606102584|35709a0a5d9039e5c19526ccc8fea0d4|2770|00000685oc52dthqts8g|4||0|0|0||||
21:23:08||Documents/test wo external storage/random-14673.txt|INST_NEW|Down|1603819777|e8cf01a31b25219122f633c14b11fced|3519|00000601oc52dthqts8g|4||0|0|0||||
21:23:10||Documents/test wo external storage/random-22203.txt|INST_NEW|Down|1601979212|15caf91b0dfec6ec2f224d1d68e3c6b2|21638|00000286oc52dthqts8g|4||0|0|0||||
21:23:10||Documents/test wo external storage/random-22449.txt|INST_NEW|Down|1602504314|e7f3b511baeb39cd24511100804f944b|4021|00000504oc52dthqts8g|4||0|0|0||||
21:23:10||Documents/test wo external storage/random-23376.txt|INST_NEW|Down|1607289570|ebfb066f72a874e31dbb517a65cb8560|4379|00000416oc52dthqts8g|4||0|0|0||||
21:23:11||Documents/test wo external storage/random-30814.txt|INST_NEW|Down|1600072162|785553c96c0cdb71ea0d29fdf80682cf|2820|00000687oc52dthqts8g|4||0|0|0||||
21:23:11||Documents/test wo external storage/random-23736.txt|INST_NEW|Down|1602288029|79afd94d82b466c1994c145fba32d8ce|3281|00000612oc52dthqts8g|4||0|0|0||||
21:23:11||Documents/test wo external storage/random-26479.txt|INST_NEW|Down|1604493517|08b5f444d488f9738ed7d191999adf7d|4291|00000428oc52dthqts8g|4||0|0|0||||
21:23:12||Documents/test wo external storage/random-32544.txt|INST_NEW|Down|1607735069|e6d623d0df203d9fa8a09ea135cb2262|2909|00000645oc52dthqts8g|4||0|0|0||||
21:23:12||Documents/test wo external storage/random-36191.txt|INST_NEW|Down|1608805707|2ec6f7c883d7c53e0e8f4eb6b82aee16|4430|00000402oc52dthqts8g|4||0|0|0||||
21:23:13||Documents/test wo external storage/random-36596.txt|INST_NEW|Down|1602219213|9662cf088f0efcee75d0674213d56738|2408|00000754oc52dthqts8g|4||0|0|0||||
21:23:13||Documents/test wo external storage/random-37480.txt|INST_NEW|Down|1609056496|43240ce3f5f4a9f5e6fbcb6e999501e0|3439|00000610oc52dthqts8g|4||0|0|0||||
21:23:13||Documents/test wo external storage/random-42801.txt|INST_NEW|Down|1607882295|0a5f8c1b7e1c1c8ff8255454eab20329|2796|00000678oc52dthqts8g|4||0|0|0||||
21:23:13||Documents/test wo external storage/random-42635.txt|INST_NEW|Down|1607514339|2579a07ee7263941fbb669d24f0342e9|3662|00000554oc52dthqts8g|4||0|0|0||||
21:23:14||Documents/test wo external storage/random-43797.txt|INST_NEW|Down|1607244445|88c888cd88204801bdca0829eb59b79d|4283|00000426oc52dthqts8g|4||0|0|0||||
21:23:14||Documents/test wo external storage/random-46755.txt|INST_NEW|Down|1605979704|19d6413239731804d34267a391289d5c|3595|00000558oc52dthqts8g|4||0|0|0||||
21:23:14||Documents/test wo external storage/random-47438.txt|INST_NEW|Down|1606470499|17654614086118a5ba2c9aa7d141733e|3592|00000591oc52dthqts8g|4||0|0|0||||
21:23:14||Documents/test wo external storage/random-47900.txt|INST_NEW|Down|1598808457|2a1e21115177db63f7a1d6bbeda056ab|16563|00000285oc52dthqts8g|4||0|0|0||||
21:23:15||Documents/test wo external storage/random-52295.txt|INST_NEW|Down|1609017086|9ac4cb4515fd72e1b2f0fdd60e37d640|4466|00000397oc52dthqts8g|4||0|0|0||||
21:23:15||Documents/test wo external storage/random-55329.txt|INST_NEW|Down|1602101549|72ed81a109a61f38b7a1d178eb6c9e30|4115|00000487oc52dthqts8g|4||0|0|0||||
21:23:15||Documents/test wo external storage/random-5331.txt|INST_NEW|Down|1607515217|b83eaf532b97a67644f5bae7ded8cdfa|4161|00000461oc52dthqts8g|4||0|0|0||||
21:23:16||Documents/test wo external storage/random-56163.txt|INST_NEW|Down|1607534713|e22efc469ac758be00b74b8760d9e505|4621|00000335oc52dthqts8g|4||0|0|0||||
21:23:16||Documents/test wo external storage/random-55348.txt|INST_NEW|Down|1606723834|d3632561c53d02ff6ca703fabb30d3fc|3451|00000611oc52dthqts8g|4||0|0|0||||
21:23:16||Documents/test wo external storage/random-58595.txt|INST_NEW|Down|1602516036|d61d818d84647a155e3c481a91e793b5|5078|00000287oc52dthqts8g|4||0|0|0||||
21:23:16||Documents/test wo external storage/random-5621.txt|INST_NEW|Down|1602005318|26c282c61a9e13b3a1f7488e7c92eb9b|3803|00000534oc52dthqts8g|4||0|0|0||||
21:23:17||Documents/test wo external storage/random-58703.txt|INST_NEW|Down|1601634262|2cbb3f36248d1a5e400c906d394281ec|4711|00000331oc52dthqts8g|4||0|0|0||||
21:23:17||Documents/test wo external storage/random-60148.txt|INST_NEW|Down|1599658160|56da39a7125ae4695452d2e49e9eb0e2|2861|00000670oc52dthqts8g|4||0|0|0||||
21:23:17||Documents/test wo external storage/random-60504.txt|INST_NEW|Down|1600164507|f634e88a398d7df708b56ad61e0134b1|3210|00000630oc52dthqts8g|4||0|0|0||||
21:23:18||Documents/test wo external storage/random-60515.txt|INST_NEW|Down|1607881054|f82278923b15b3c8eef6e19ef43562df|29105|00000289oc52dthqts8g|4||0|0|0||||
21:23:18||Documents/test wo external storage/random-60971.txt|INST_NEW|Down|1602458254|ae7e3d3f24da40da1ca9cf5048c63103|5054|00000292oc52dthqts8g|4||0|0|0||||
21:23:19||Documents/test wo external storage/random-62350.txt|INST_NEW|Down|1602677003|97f5862cbffb0c98113bd7771719898f|2869|00000654oc52dthqts8g|4||0|0|0||||
21:23:19||Documents/test wo external storage/random-63480.txt|INST_NEW|Down|1602805775|d596dbc258921a17852fb8a0fbdc615f|2986|00000643oc52dthqts8g|4||0|0|0||||
21:23:20||Documents/test wo external storage/random-63890.txt|INST_NEW|Down|1608745343|c79b19ed145781a2ae4130d7c9254e71|2942|00000765oc52dthqts8g|4||0|0|0||||
21:23:20||Documents/test wo external storage/random-6411.txt|INST_NEW|Down|1599339091|64b43536a8ab967f23c5d75bd7d0decd|4251|00000444oc52dthqts8g|4||0|0|0||||
21:23:19||Documents/test wo external storage/random-6240.txt|INST_NEW|Down|1601979756|0772c1400d61d1ce9cc76a125c84aaf3|4774|00000300oc52dthqts8g|4||0|0|0||||
21:23:20||Documents/test wo external storage/random-64774.txt|INST_NEW|Down|1600697319|a6bcf5fc1893c89da0b53721f035d118|3251|00000660oc52dthqts8g|4||0|0|0||||
21:23:21||Documents/test wo external storage/random-6822.txt|INST_NEW|Down|1599803540|a6b6b090668d7a33c1e34def39eeed7b|3935|00000507oc52dthqts8g|4||0|0|0||||
21:23:21||Documents/test wo external storage/random-69484.txt|INST_NEW|Down|1605946761|e59ed4d1ae5e1d18a8ccc6c60834038d|3128|00000636oc52dthqts8g|4||0|0|0||||
21:23:21||Documents/test wo external storage/random-65438.txt|INST_NEW|Down|1600969036|a527593a2b2cbf380f2d3d90e4aff9c0|4075|00000482oc52dthqts8g|4||0|0|0||||
21:23:22||Documents/test wo external storage/random-70002.txt|INST_NEW|Down|1604304198|c078589bdbcec533a574ac9c335deccd|3916|00000521oc52dthqts8g|4||0|0|0||||
21:23:23||Documents/test wo external storage/random-73397.txt|INST_NEW|Down|1598754397|b32513b6755749bb58bf3cedbefa4fe9|3728|00000543oc52dthqts8g|4||0|0|0||||
21:23:24||Documents/test wo external storage/random-73508.txt|INST_NEW|Down|1606181838|8c9245ccef33ac3983b91b43874d4d2b|4593|00000344oc52dthqts8g|4||0|0|0||||
21:23:24||Documents/test wo external storage/random-76461.txt|INST_NEW|Down|1604997212|38357383a75bc088c5ba27174a6491aa|4732|00000321oc52dthqts8g|4||0|0|0||||
21:23:24||Documents/test wo external storage/random-75433.txt|INST_NEW|Down|1604176362|01577417bb49bbe2bcc0d735c4a2fe3f|48922|00000283oc52dthqts8g|4||0|0|0||||
21:23:25||Documents/test wo external storage/random-77423.txt|INST_NEW|Down|1601100927|a851a700e2cdbe5bd32cd7f725bd6a37|4487|00000354oc52dthqts8g|4||0|0|0||||
21:23:25||Documents/test wo external storage/random-76482.txt|INST_NEW|Down|1603414240|363dadd7cb3fa5f08d9aa22e8a034cc9|5071|00000290oc52dthqts8g|4||0|0|0||||
21:23:25||Documents/test wo external storage/random-78694.txt|INST_NEW|Down|1599574228|d46d5a5fc6fddc9070c7c0d548924650|3781|00000537oc52dthqts8g|4||0|0|0||||
21:23:26||Documents/test wo external storage/random-82920.txt|INST_NEW|Down|1599479019|ab470d4f12e88d1494252a7392c41abd|3644|00000568oc52dthqts8g|4||0|0|0||||
21:23:26||Documents/test wo external storage/random-82599.txt|INST_NEW|Down|1602440672|38fbf4a23b4e96eca99fbfd83ad1b297|4129|00000478oc52dthqts8g|4||0|0|0||||
21:23:26||Documents/test wo external storage/random-83157.txt|INST_NEW|Down|1600938871|9b720d97ede6b7dc98ffca2a975c72e2|3499|00000609oc52dthqts8g|4||0|0|0||||
21:23:28||Documents/test wo external storage/random-8360.txt|INST_NEW|Down|1603864118|dae5e9d033b2d4fd420f73f5842f5cc9|3136|00000686oc52dthqts8g|4||0|0|0||||
21:23:29||Documents/test wo external storage/random-83762.txt|INST_NEW|Down|1601969311|f61ec291aaec2618c80bca93e56aaefa|3204|00000624oc52dthqts8g|4||0|0|0||||
21:23:29||Documents/test wo external storage/random-89056.txt|INST_NEW|Down|1601589210|f97feaffd402a5bd9434558db4042376|4326|00000421oc52dthqts8g|4||0|0|0||||
21:23:29||Documents/test wo external storage/random-91273.txt|INST_NEW|Down|1606540115|c05efeb42078d140cadb1b04b922971f|13695|00000291oc52dthqts8g|4||0|0|0||||
21:23:29||Documents/test wo external storage/random-90705.txt|INST_NEW|Down|1600142845|591f6aa9b54f8daf6eab6a1edf809f05|4727|00000326oc52dthqts8g|4||0|0|0||||
21:23:29||Documents/test wo external storage/random-87723.txt|INST_NEW|Down|1606462092|c17108107b2c7b4d8a158133bfeb8eed|3185|00000642oc52dthqts8g|4||0|0|0||||
21:23:30||Documents/test wo external storage/random-91627.txt|INST_NEW|Down|1599747088|668ca7355a60579d6926c0d3e0b63b93|4872|00000288oc52dthqts8g|4||0|0|0||||
21:23:31||Documents/test wo external storage/random-9831.txt|INST_NEW|Down|1603883235|16d9af6830ce4c08d398900ca51a3e12|3602|00000557oc52dthqts8g|4||0|0|0||||
21:23:30||Documents/test wo external storage/random-96831.txt|INST_NEW|Down|1599814611|490cfe8934ce9397d3cb5aa844683c3d|304827|00000284oc52dthqts8g|4||0|0|0||||
21:23:31||Documents/test wo external storage/random-98521.txt|INST_NEW|Down|1603187963|cc30f8aec755f43487b4dcea0915ae62|2266|00000714oc52dthqts8g|4||0|0|0||||
21:23:31||Documents/test wo external storage/random-99876.txt|INST_NEW|Down|1602548844|d73c7cc5efbc04440b4dd1f64a0b91e0|4044|00000495oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:23:33Z (last step: 28197 msec, total: 30743 msec)
#=#=#=# Syncrun started 2020-10-29T21:23:42Z
#=#=#=#=# Propagation starts 2020-10-29T21:23:45Z (last step: 3089 msec, total: 3089 msec)
21:23:45||Documents/test wo external storage/random-13695.txt|INST_NEW|Down|1606232612|a62b864bc2e46cb24905f13eeb3bbf6c|1582|00000928oc52dthqts8g|4||0|0|0||||
21:23:45||Documents/test wo external storage/random-13779.txt|INST_NEW|Down|1599542306|2b366187707fd3ab2318f738dd1ded24|1406|00000925oc52dthqts8g|4||0|0|0||||
21:23:45||Documents/test wo external storage/random-17455.txt|INST_NEW|Down|1600227510|c87ded5d4c956da85e25cd934804a025|1174|00000988oc52dthqts8g|4||0|0|0||||
21:23:45||Documents/test wo external storage/random-22397.txt|INST_NEW|Down|1599001577|55109fd063356fea13e5792fb80b3d4f|1060|00001001oc52dthqts8g|4||0|0|0||||
21:23:45||Documents/test wo external storage/random-17977.txt|INST_NEW|Down|1604614816|24987bc5da258756d7d845305402b327|1961|00000870oc52dthqts8g|4||0|0|0||||
21:23:45||Documents/test wo external storage/random-23433.txt|INST_NEW|Down|1606233500|44e15876af71a07e080303d9025217d9|1430|00000910oc52dthqts8g|4||0|0|0||||
21:23:46||Documents/test wo external storage/random-23871.txt|INST_NEW|Down|1602860678|fd38e7e47f2741e09dd3eb24a7e6bddd|1186|00000948oc52dthqts8g|4||0|0|0||||
21:23:47||Documents/test wo external storage/random-27669.txt|INST_NEW|Down|1601800774|c7403a74c5b08e0024c9d73d51db4b5b|1151|00001103oc52dthqts8g|4||0|0|0||||
21:23:48||Documents/test wo external storage/random-33708.txt|INST_NEW|Down|1607232021|f1fb85e0846a7a0ef5c8bed1af21f318|2234|00000882oc52dthqts8g|4||0|0|0||||
21:23:48||Documents/test wo external storage/random-33911.txt|INST_NEW|Down|1599192035|3c7d2f472f102f1eaae9eee516705fe8|2167|00001116oc52dthqts8g|4||0|0|0||||
21:23:48||Documents/test wo external storage/random-34663.txt|INST_NEW|Down|1604770577|2475adbf2e2095061793b7b6738c6626|1126|00001150oc52dthqts8g|4||0|0|0||||
21:23:48||Documents/test wo external storage/random-36286.txt|INST_NEW|Down|1607719757|7ebad019d95cd9792ddeb7fe5966c533|1393|00000943oc52dthqts8g|4||0|0|0||||
21:23:49||Documents/test wo external storage/random-45429.txt|INST_NEW|Down|1607677346|024a589994b83f92f3bd0bbbca32a4c8|1373|00000985oc52dthqts8g|4||0|0|0||||
21:23:48||Documents/test wo external storage/random-44051.txt|INST_NEW|Down|1608788182|dcd02576c078055f69953c5838469b7c|1849|00000871oc52dthqts8g|4||0|0|0||||
21:23:49||Documents/test wo external storage/random-47041.txt|INST_NEW|Down|1608609560|18564ec6836752e7a91889ad798e84af|1585|00000890oc52dthqts8g|4||0|0|0||||
21:23:50||Documents/test wo external storage/random-50513.txt|INST_NEW|Down|1604051847|351d1f0e5d22819c897d0bc59c7df0a4|2040|00000834oc52dthqts8g|4||0|0|0||||
21:23:50||Documents/test wo external storage/random-5593.txt|INST_NEW|Down|1603324410|bf234231c8d542b766252f23e1fd9fc6|1718|00000916oc52dthqts8g|4||0|0|0||||
21:23:50||Documents/test wo external storage/random-56666.txt|INST_NEW|Down|1603655959|0cbc28f184c1205064f8bf2bf32ecd79|1907|00000873oc52dthqts8g|4||0|0|0||||
21:23:50||Documents/test wo external storage/random-56207.txt|INST_NEW|Down|1605764253|16838f80b3465535ab850c31f488e43f|2731|00000813oc52dthqts8g|4||0|0|0||||
21:23:50||Documents/test wo external storage/random-58150.txt|INST_NEW|Down|1608398729|b74eac270665a0b1b59bbf37742bb9cc|2210|00000780oc52dthqts8g|4||0|0|0||||
21:23:50||Documents/test wo external storage/random-56466.txt|INST_NEW|Down|1609024694|b0993348613b7ed35805bffd5965429c|1987|00000832oc52dthqts8g|4||0|0|0||||
21:23:51||Documents/test wo external storage/random-6801.txt|INST_NEW|Down|1605921753|b0ee3ad3668f7ce45c629146e7fa469e|1106|00001038oc52dthqts8g|4||0|0|0||||
21:23:51||Documents/test wo external storage/random-61041.txt|INST_NEW|Down|1608708574|7f4b1ff4aec596d05602f03bd3ba142d|1678|00000893oc52dthqts8g|4||0|0|0||||
21:23:51||Documents/test wo external storage/random-62172.txt|INST_NEW|Down|1600622795|9939a9d285e0cdac666bee6985714708|2205|00000774oc52dthqts8g|4||0|0|0||||
21:23:52||Documents/test wo external storage/random-78507.txt|INST_NEW|Down|1601137156|f289643e4475a0d8c0b057902418ea20|1199|00000946oc52dthqts8g|4||0|0|0||||
21:23:52||Documents/test wo external storage/random-89228.txt|INST_NEW|Down|1604229575|fbff283eb1b6af3e302a422fe82f4edc|1900|00000853oc52dthqts8g|4||0|0|0||||
21:23:52||Documents/test wo external storage/random-818.txt|INST_NEW|Down|1608285193|bb21834f30ba1994ea67c08f249fde86|1231|00000947oc52dthqts8g|4||0|0|0||||
21:23:52||Documents/test wo external storage/random-88941.txt|INST_NEW|Down|1605398483|e07e450689338fed6c7b9631a192c37e|1246|00000940oc52dthqts8g|4||0|0|0||||
21:23:52||Documents/test wo external storage/random-85449.txt|INST_NEW|Down|1601589744|3525981bc7fb038425a0ee128e353776|1040|00001068oc52dthqts8g|4||0|0|0||||
21:23:53||Documents/test wo external storage/random-90147.txt|INST_NEW|Down|1604666774|2e2e81e704b7a0597109fedba7c55704|2658|00000839oc52dthqts8g|4||0|0|0||||
21:23:54||Documents/test wo external storage/random-91442.txt|INST_NEW|Down|1604227815|63964535a11d59179490b2b10dbe676d|1381|00000932oc52dthqts8g|4||0|0|0||||
21:23:54||Documents/test wo external storage/random-90728.txt|INST_NEW|Down|1605270674|b2f2124563a215c8b826e4fa40ef3fbf|1119|00000982oc52dthqts8g|4||0|0|0||||
21:23:54||Documents/test wo external storage/random-90160.txt|INST_NEW|Down|1600481662|9ed5ad1c739a601ffc49e0c9648cc7f8|1846|00000896oc52dthqts8g|4||0|0|0||||
21:23:54||Documents/test wo external storage/random-93732.txt|INST_NEW|Down|1603050865|326e023fb4e47077b805f29d8fa6c404|2252|00000798oc52dthqts8g|4||0|0|0||||
21:23:54||Documents/test wo external storage/random-95756.txt|INST_NEW|Down|1602951214|f69ecd42bb4d275a3438240658758f71|1711|00000891oc52dthqts8g|4||0|0|0||||
21:23:54||Documents/test wo external storage/random-99959.txt|INST_NEW|Down|1599396865|81e86aa1f63861c18abf933848ae8b99|1547|00000977oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:23:56Z (last step: 10658 msec, total: 13748 msec)
#=#=#=# Syncrun started 2020-10-29T21:24:02Z
#=#=#=#=# Propagation starts 2020-10-29T21:24:02Z (last step: 408 msec, total: 408 msec)
#=#=#=# Syncrun finished 2020-10-29T21:24:02Z (last step: 76 msec, total: 485 msec)
#=#=#=# Syncrun started 2020-10-29T21:31:32Z
#=#=#=#=# Propagation starts 2020-10-29T21:31:32Z (last step: 505 msec, total: 505 msec)
#=#=#=# Syncrun finished 2020-10-29T21:31:32Z (last step: 52 msec, total: 558 msec)
#=#=#=# Syncrun started 2020-10-29T21:32:35Z
#=#=#=#=# Propagation starts 2020-10-29T21:32:39Z (last step: 3317 msec, total: 3317 msec)
||Documents/test with 1 external storage|INST_NEW|Down|1604007156|5f9b34f521c70|0|00001381oc52dthqts8g|4||0|0|0||||
21:32:39||Documents/test with 1 external storage/random-10716.txt|INST_NEW|Down|1603529913|473c805d8810f131cc48ee1bdbc859cd|3519|00001493oc52dthqts8g|4||0|0|0||||
..
21:33:44||Documents/test with 1 external storage/random-99959.txt|INST_NEW|Down|1599396865|3bfb222f4f5e4653253253f5f46f1c52|1547|00002042oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:33:48Z (last step: 9513 msec, total: 11791 msec)
#=#=#=# Syncrun started 2020-10-29T21:33:53Z
#=#=#=#=# Propagation starts 2020-10-29T21:33:56Z (last step: 2812 msec, total: 2812 msec)
21:33:56||Documents/test with 1 external storage/random-23871.txt|INST_NEW|Down|1602860678|c894c21304637be35e15362340ca9442|1186|00002253oc52dthqts8g|4||0|0|0||||
21:33:56||Documents/test with 1 external storage/random-22397.txt|INST_NEW|Down|1599001577|448bd10b842a5f12df62565437105e7e|1060|00002121oc52dthqts8g|4||0|0|0||||
21:33:56||Documents/test with 1 external storage/random-6801.txt|INST_NEW|Down|1605921753|6102b1097d9b66c08677b6f2359e6dda|1106|00002115oc52dthqts8g|4||0|0|0||||
21:33:56||Documents/test with 1 external storage/random-56207.txt|INST_NEW|Down|1605764253|39cb5681f637c8e0593092611ad1ae6d|2731|00002196oc52dthqts8g|4||0|0|0||||
21:33:57||Documents/test with 1 external storage/random-85449.txt|INST_NEW|Down|1601589744|708b6849d2e2932bec87dd1e26150360|1040|00002205oc52dthqts8g|4||0|0|0||||
21:33:56||Documents/test with 1 external storage/random-27669.txt|INST_NEW|Down|1601800774|682fa647a154a3d47e38bcb67edf21e4|1151|00002127oc52dthqts8g|4||0|0|0||||
21:33:56||Documents/test with 1 external storage/random-78507.txt|INST_NEW|Down|1601137156|8a5125539443973fbc8b8e793b003375|1199|00002188oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:34:00Z (last step: 4409 msec, total: 7221 msec)
#=#=#=# Syncrun started 2020-10-29T21:34:05Z
#=#=#=#=# Propagation starts 2020-10-29T21:34:10Z (last step: 5192 msec, total: 5192 msec)
21:34:10||Documents/test with 1 external storage/random-12015.txt|INST_NEW|Down|1599918698|4c7106c589d2ff2aa2373e13928c3281|2572|00002343oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:34:11Z (last step: 893 msec, total: 6086 msec)
#=#=#=# Syncrun started 2020-10-29T21:34:15Z
#=#=#=#=# Propagation starts 2020-10-29T21:34:16Z (last step: 838 msec, total: 838 msec)
#=#=#=# Syncrun finished 2020-10-29T21:34:16Z (last step: 37 msec, total: 875 msec)
#=#=#=# Syncrun started 2020-10-29T21:46:02Z
#=#=#=#=# Propagation starts 2020-10-29T21:46:04Z (last step: 1199 msec, total: 1199 msec)
#=#=#=# Syncrun finished 2020-10-29T21:46:04Z (last step: 43 msec, total: 1242 msec)
#=#=#=# Syncrun started 2020-10-29T21:47:02Z
#=#=#=#=# Propagation starts 2020-10-29T21:47:06Z (last step: 3409 msec, total: 3409 msec)
||Documents/test with 3 external storages|INST_NEW|Down|1609017086|5f9b3857d9aa6|0|00002452oc52dthqts8g|4||0|0|0||||
21:47:06||Documents/test with 3 external storages/random-11158.txt|INST_NEW|Down|1599449413|15ab0dbad7607d9adc46a5254942fe67|4471|00002463oc52dthqts8g|4||0|0|0||||
21:47:06||Documents/test with 3 external storages/random-22203.txt|INST_NEW|Down|1601979212|9aad44aea3dbb694975d73ae03e1b602|21638|00002455oc52dthqts8g|4||0|0|0||||
21:47:06||Documents/test with 3 external storages/random-58595.txt|INST_NEW|Down|1602516036|44a3c773ae9ff755030dd87794132fde|5078|00002454oc52dthqts8g|4||0|0|0||||
21:47:07||Documents/test with 3 external storages/random-60515.txt|INST_NEW|Down|1607881054|339a28cf2af568cb1ca96956f09c1d3d|29105|00002459oc52dthqts8g|4||0|0|0||||
21:47:08||Documents/test with 3 external storages/random-6240.txt|INST_NEW|Down|1601979756|600cb965ee0f324fc9304ee5d8d44726|4774|00002465oc52dthqts8g|4||0|0|0||||
21:47:09||Documents/test with 3 external storages/random-75433.txt|INST_NEW|Down|1604176362|ce35eb55e49985384415edb75e184b81|48922|00002456oc52dthqts8g|4||0|0|0||||
21:47:06||Documents/test with 3 external storages/random-52295.txt|INST_NEW|Down|1609017086|f9ced3122b63e0b5fbb202918732603a|4466|00002466oc52dthqts8g|4||0|0|0||||
21:47:06||Documents/test with 3 external storages/random-58703.txt|INST_NEW|Down|1601634262|1737a9a4bcaa09ecfaf3d80bdbd8cbcd|4711|00002461oc52dthqts8g|4||0|0|0||||
21:47:09||Documents/test with 3 external storages/random-73508.txt|INST_NEW|Down|1606181838|3189823737919ebfa2b763144226641a|4593|00002467oc52dthqts8g|4||0|0|0||||
21:47:06||Documents/test with 3 external storages/random-56163.txt|INST_NEW|Down|1607534713|9c3dbe6954269979c5ad7fad759217dc|4621|00002462oc52dthqts8g|4||0|0|0||||
21:47:10||Documents/test with 3 external storages/random-76482.txt|INST_NEW|Down|1603414240|964a16c5bac0fbe5718e3190c70cb865|5071|00002457oc52dthqts8g|4||0|0|0||||
21:47:13||Documents/test with 3 external storages/random-96831.txt|INST_NEW|Down|1599814611|1e7480bbc3c7e407a100e7daa4e1cc11|304827|00002453oc52dthqts8g|4||0|0|0||||
21:47:12||Documents/test with 3 external storages/random-91273.txt|INST_NEW|Down|1606540115|0758cd4336640521e620de427a10ae01|13695|00002458oc52dthqts8g|4||0|0|0||||
21:47:11||Documents/test with 3 external storages/random-90705.txt|INST_NEW|Down|1600142845|dd7a8ff9465ce4e19ccac6c57ef89765|4727|00002460oc52dthqts8g|4||0|0|0||||
21:47:12||Documents/test with 3 external storages/random-91627.txt|INST_NEW|Down|1599747088|da80d000ca2fdf5521c3e2ddff057e29|4872|00002464oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:47:17Z (last step: 11431 msec, total: 14841 msec)
#=#=#=# Syncrun started 2020-10-29T21:47:23Z
#=#=#=#=# Propagation starts 2020-10-29T21:47:26Z (last step: 2993 msec, total: 2993 msec)
21:47:27||Documents/test with 3 external storages/random-22449.txt|INST_NEW|Down|1602504314|35695744af17ec692108e9b2fc21c280|4021|00002512oc52dthqts8g|4||0|0|0||||
21:47:27||Documents/test with 3 external storages/random-47900.txt|INST_NEW|Down|1598808457|ac4c1079ac515bfd11cad66e3d3e6f8e|16563|00002478oc52dthqts8g|4||0|0|0||||
21:47:27||Documents/test with 3 external storages/random-23376.txt|INST_NEW|Down|1607289570|6363dd754dbcbb019d1b9a3c2e90a791|4379|00002487oc52dthqts8g|4||0|0|0||||
21:47:28||Documents/test with 3 external storages/random-5331.txt|INST_NEW|Down|1607515217|309b724c89941576576e6398a5ec2b85|4161|00002480oc52dthqts8g|4||0|0|0||||
21:47:27||Documents/test with 3 external storages/random-26479.txt|INST_NEW|Down|1604493517|3afdde54662666bdc03d74d181a720bc|4291|00002474oc52dthqts8g|4||0|0|0||||
21:47:28||Documents/test with 3 external storages/random-55329.txt|INST_NEW|Down|1602101549|ec39f60d8ee0b3e56a2bba3b60106511|4115|00002496oc52dthqts8g|4||0|0|0||||
21:47:27||Documents/test with 3 external storages/random-11242.txt|INST_NEW|Down|1608947719|2c16657ab94e1cd038bcd94ba1d20e4f|4232|00002494oc52dthqts8g|4||0|0|0||||
21:47:29||Documents/test with 3 external storages/random-5621.txt|INST_NEW|Down|1602005318|7f037ec321ce0c8c767e931b8f7e8943|3803|00002493oc52dthqts8g|4||0|0|0||||
21:47:30||Documents/test with 3 external storages/random-60971.txt|INST_NEW|Down|1602458254|bf19f4904d473257d6526db132670438|5054|00002491oc52dthqts8g|4||0|0|0||||
21:47:32||Documents/test with 3 external storages/random-73397.txt|INST_NEW|Down|1598754397|0a79187ae229afaa6017e059469e355b|3728|00002511oc52dthqts8g|4||0|0|0||||
21:47:31||Documents/test with 3 external storages/random-6411.txt|INST_NEW|Down|1599339091|76db86a0d2790e0aefd9447cbd644cb6|4251|00002477oc52dthqts8g|4||0|0|0||||
21:47:27||Documents/test with 3 external storages/random-36191.txt|INST_NEW|Down|1608805707|03836fb3acdd5325ed270c6ba67005f9|4430|00002481oc52dthqts8g|4||0|0|0||||
21:47:33||Documents/test with 3 external storages/random-82599.txt|INST_NEW|Down|1602440672|e73657f6510902ca275b786ca454569e|4129|00002482oc52dthqts8g|4||0|0|0||||
21:47:31||Documents/test with 3 external storages/random-65438.txt|INST_NEW|Down|1600969036|3feec1a68b05dc7a9426ef854c63824d|4075|00002489oc52dthqts8g|4||0|0|0||||
21:47:34||Documents/test with 3 external storages/random-89056.txt|INST_NEW|Down|1601589210|179fc4c2600a300f72d275b9e72364b0|4326|00002490oc52dthqts8g|4||0|0|0||||
21:47:33||Documents/test with 3 external storages/random-77423.txt|INST_NEW|Down|1601100927|64b1219b62c57eee0debec25a62c9b87|4487|00002475oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:47:38Z (last step: 11865 msec, total: 14859 msec)
#=#=#=# Syncrun started 2020-10-29T21:47:45Z
#=#=#=#=# Propagation starts 2020-10-29T21:47:48Z (last step: 3434 msec, total: 3434 msec)
21:47:48||Documents/test with 3 external storages/random-13160.txt|INST_NEW|Down|1606150094|2ec8bf8683162f68d4548aef04d91e15|3548|00002539oc52dthqts8g|4||0|0|0||||
21:47:48||Documents/test with 3 external storages/random-46755.txt|INST_NEW|Down|1605979704|fcddc59bf2bf827c5d29bc0b5bd5b3de|3595|00002524oc52dthqts8g|4||0|0|0||||
21:47:48||Documents/test with 3 external storages/random-14673.txt|INST_NEW|Down|1603819777|5d3fe90e8fb6bb7b23cc845acf056bdd|3519|00002547oc52dthqts8g|4||0|0|0||||
21:47:48||Documents/test with 3 external storages/random-23736.txt|INST_NEW|Down|1602288029|25c87c19dd5c0bfc168e42501782e6e6|3281|00002558oc52dthqts8g|4||0|0|0||||
21:47:48||Documents/test with 3 external storages/random-43797.txt|INST_NEW|Down|1607244445|5d2e87519d3f4de8d0ceeeebb428fef8|4283|00002523oc52dthqts8g|4||0|0|0||||
21:47:51||Documents/test with 3 external storages/random-6822.txt|INST_NEW|Down|1599803540|e7a11de970de8dc957bbc41bcf18de41|3935|00002515oc52dthqts8g|4||0|0|0||||
...
21:50:10||Documents/test with 3 external storages/random-22397.txt|INST_NEW|Down|1599001577|39e285c3e3ee4c4342dc8ca20f1cab7b|1060|00003083oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:50:13Z (last step: 2793 msec, total: 6545 msec)
#=#=#=# Syncrun started 2020-10-29T21:55:54Z
#=#=#=#=# Propagation starts 2020-10-29T21:55:59Z (last step: 4548 msec, total: 4548 msec)
21:55:59||Documents/test with 3 external storages/random-13695.txt|INST_NEW|Down|1606232612|44cdc55a8a1f12baf5e9d60e6cf22022|1582|00003342oc52dthqts8g|4||0|0|0||||
21:55:59||Documents/test with 3 external storages/random-27669.txt|INST_NEW|Down|1601800774|2a8b5343cc15de97d25acc0fb5e8f692|1151|00003113oc52dthqts8g|4||0|0|0||||
21:55:59||Documents/test with 3 external storages/random-60504.txt|INST_NEW|Down|1600164507|ead2e1e7be7d392d2119dbfc8ac61dc7|3210|00003109oc52dthqts8g|4||0|0|0||||
21:55:59||Documents/test with 3 external storages/random-98521.txt|INST_NEW|Down|1603187963|93885afccea3d1e9094c7544b4e057bf|2266|00003181oc52dthqts8g|4||0|0|0||||
#=#=#=# Syncrun finished 2020-10-29T21:56:02Z (last step: 3235 msec, total: 7783 msec)
#=#=#=# Syncrun started 2020-10-29T21:57:41Z
#=#=#=#=# Propagation starts 2020-10-29T21:57:42Z (last step: 1358 msec, total: 1358 msec)
#=#=#=# Syncrun finished 2020-10-29T21:57:42Z (last step: 64 msec, total: 1423 msec)