Errors when trying to migrate from Dropbox to Nextcloud

[bluepr0]

Support intro

Sorry to hear you’re facing problems

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can

Nextcloud version 15.0.2:
Operating system and version unRAID 6.6.7: on docker

The issue you are facing:
I’m trying to move around 200GB of data from Dropbox to Nextcloud (all local network). I originally tried to just copy and paste the folders (just like Dropbox works) but it seems you need to upload all files through the web app.

Anyway when drag and drop all files from my Dropbox folder to Nextcloud webapp (Using Chrome 73) I’m always getting errors such as An unknown error has ocurred or Could not create folder. This only happens when I try to move my entire Dropbox folder it seems, if I go with just a few folders then it usually works fine

Is this the first time you’ve seen this error? Yes:

Steps to replicate it:

1. Drag huge amount of folders with files from local directory to Nextcloud webapp using Chrome 73

The output of your Nextcloud log in Admin > Logging:

{"reqId":"TKvchYbG7WvqBjPbi8ER","level":4,"time":"2019-03-29T09:17:05+00:00","remoteAddr":"10.0.1.21","user":"jonatan","app":"webdav","method":"MKCOL","url":"/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","message":{"Exception":"Doctrine\\DBAL\\Exception\\DriverException","Message":"An exception occurred while executing 'UPDATE `oc_file_locks` SET `lock` = `lock` + 1, `ttl` = ? WHERE `key` = ? AND `lock` >= 0' with params [1553854625, \"files\\/94fd320b6359acc490e1a6f373a6098b\"]:\n\nSQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction","Code":0,"Trace":[{"file":"/config/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php","line":128,"function":"convertException","class":"Doctrine\\DBAL\\Driver\\AbstractMySQLDriver","type":"->","args":["An exception occurred while executing 'UPDATE `oc_file_locks` SET `lock` = `lock` + 1, `ttl` = ? WHERE `key` = ? AND `lock` >= 0' with params [1553854625, \"files\\/94fd320b6359acc490e1a6f373a6098b\"]:\n\nSQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction",{"errorInfo":["40001",1213,"Deadlock found when trying to get lock; try restarting transaction"],"__class__":"Doctrine\\DBAL\\Driver\\PDOException"}]},{"file":"/config/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1015,"function":"driverExceptionDuringQuery","class":"Doctrine\\DBAL\\DBALException","type":"::","args":[{"__class__":"Doctrine\\DBAL\\Driver\\PDOMySql\\Driver"},{"errorInfo":["40001",1213,"Deadlock found when trying to get lock; try restarting transaction"],"__class__":"Doctrine\\DBAL\\Driver\\PDOException"},"UPDATE `oc_file_locks` SET `lock` = `lock` + 1, `ttl` = ? WHERE `key` = ? AND `lock` >= 0",{"1":1553854625,"2":"files/94fd320b6359acc490e1a6f373a6098b"}]},{"file":"/config/www/nextcloud/lib/private/DB/Connection.php","line":216,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->","args":["UPDATE `oc_file_locks` SET `lock` = `lock` + 1, `ttl` = ? WHERE `key` = ? AND `lock` >= 0",[1553854625,"files/94fd320b6359acc490e1a6f373a6098b"],[]]},{"file":"/config/www/nextcloud/lib/private/Lock/DBLockingProvider.php","line":196,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->","args":["UPDATE `oc_file_locks` SET `lock` = `lock` + 1, `ttl` = ? WHERE `key` = ? AND `lock` >= 0",[1553854625,"files/94fd320b6359acc490e1a6f373a6098b"]]},{"file":"/config/www/nextcloud/lib/private/Files/Storage/Common.php","line":714,"function":"acquireLock","class":"OC\\Lock\\DBLockingProvider","type":"->","args":["files/94fd320b6359acc490e1a6f373a6098b",1]},{"file":"/config/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":593,"function":"acquireLock","class":"OC\\Files\\Storage\\Common","type":"->","args":["files/Projects (Selective Sync Conflict)",1,{"__class__":"OC\\Lock\\DBLockingProvider"}]},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":1923,"function":"acquireLock","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["files/Projects (Selective Sync Conflict)",1,{"__class__":"OC\\Lock\\DBLockingProvider"}]},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":2037,"function":"lockPath","class":"OC\\Files\\View","type":"->","args":["/Projects (Selective Sync Conflict)",1]},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":1132,"function":"lockFile","class":"OC\\Files\\View","type":"->","args":["/Projects (Selective Sync Conflict)/Squire (Selective Sync Conflict)/Squire iPad/PaintCode/SVG logos",1]},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":268,"function":"basicOperation","class":"OC\\Files\\View","type":"->","args":["mkdir","/Projects (Selective Sync Conflict)/Squire (Selective Sync Conflict)/Squire iPad/PaintCode/SVG logos",["create","write"]]},{"file":"/config/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":189,"function":"mkdir","class":"OC\\Files\\View","type":"->","args":["/Projects (Selective Sync Conflict)/Squire (Selective Sync Conflict)/Squire iPad/PaintCode/SVG logos"]},{"file":"/config/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1210,"function":"createDirectory","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->","args":["SVG logos"]},{"file":"/config/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":590,"function":"createCollection","class":"Sabre\\DAV\\Server","type":"->","args":["files/jonatan/Projects (Selective Sync Conflict)/Squire (Selective Sync Conflict)/Squire iPad/PaintCode/SVG logos",{"__class__":"Sabre\\DAV\\MkCol"}]},{"function":"httpMkcol","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"absoluteUrl":"https://10.0.1.38:444/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/config/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php","line":105,"function":"call_user_func_array","args":[[{"__class__":"Sabre\\DAV\\CorePlugin"},"httpMkcol"],[{"absoluteUrl":"https://10.0.1.38:444/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"/config/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":479,"function":"emit","class":"Sabre\\Event\\EventEmitter","type":"->","args":["method:MKCOL",[{"absoluteUrl":"https://10.0.1.38:444/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"/config/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":254,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"absoluteUrl":"https://10.0.1.38:444/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/config/www/nextcloud/apps/dav/lib/Server.php","line":301,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/config/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"/config/www/nextcloud/remote.php","line":163,"args":["/config/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/config/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php","Line":115,"Previous":{"Exception":"Doctrine\\DBAL\\Driver\\PDOException","Message":"SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction","Code":"40001","Trace":[{"file":"/config/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1008,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDOStatement","type":"->","args":[[1553854625,"files/94fd320b6359acc490e1a6f373a6098b"]]},{"file":"/config/www/nextcloud/lib/private/DB/Connection.php","line":216,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->","args":["UPDATE `oc_file_locks` SET `lock` = `lock` + 1, `ttl` = ? WHERE `key` = ? AND `lock` >= 0",[1553854625,"files/94fd320b6359acc490e1a6f373a6098b"],[]]},{"file":"/config/www/nextcloud/lib/private/Lock/DBLockingProvider.php","line":196,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->","args":["UPDATE `oc_file_locks` SET `lock` = `lock` + 1, `ttl` = ? WHERE `key` = ? AND `lock` >= 0",[1553854625,"files/94fd320b6359acc490e1a6f373a6098b"]]},{"file":"/config/www/nextcloud/lib/private/Files/Storage/Common.php","line":714,"function":"acquireLock","class":"OC\\Lock\\DBLockingProvider","type":"->","args":["files/94fd320b6359acc490e1a6f373a6098b",1]},{"file":"/config/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":593,"function":"acquireLock","class":"OC\\Files\\Storage\\Common","type":"->","args":["files/Projects (Selective Sync Conflict)",1,{"__class__":"OC\\Lock\\DBLockingProvider"}]},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":1923,"function":"acquireLock","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["files/Projects (Selective Sync Conflict)",1,{"__class__":"OC\\Lock\\DBLockingProvider"}]},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":2037,"function":"lockPath","class":"OC\\Files\\View","type":"->","args":["/Projects (Selective Sync Conflict)",1]},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":1132,"function":"lockFile","class":"OC\\Files\\View","type":"->","args":["/Projects (Selective Sync Conflict)/Squire (Selective Sync Conflict)/Squire iPad/PaintCode/SVG logos",1]},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":268,"function":"basicOperation","class":"OC\\Files\\View","type":"->","args":["mkdir","/Projects (Selective Sync Conflict)/Squire (Selective Sync Conflict)/Squire iPad/PaintCode/SVG logos",["create","write"]]},{"file":"/config/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":189,"function":"mkdir","class":"OC\\Files\\View","type":"->","args":["/Projects (Selective Sync Conflict)/Squire (Selective Sync Conflict)/Squire iPad/PaintCode/SVG logos"]},{"file":"/config/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1210,"function":"createDirectory","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->","args":["SVG logos"]},{"file":"/config/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":590,"function":"createCollection","class":"Sabre\\DAV\\Server","type":"->","args":["files/jonatan/Projects (Selective Sync Conflict)/Squire (Selective Sync Conflict)/Squire iPad/PaintCode/SVG logos",{"__class__":"Sabre\\DAV\\MkCol"}]},{"function":"httpMkcol","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"absoluteUrl":"https://10.0.1.38:444/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/config/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php","line":105,"function":"call_user_func_array","args":[[{"__class__":"Sabre\\DAV\\CorePlugin"},"httpMkcol"],[{"absoluteUrl":"https://10.0.1.38:444/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"/config/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":479,"function":"emit","class":"Sabre\\Event\\EventEmitter","type":"->","args":["method:MKCOL",[{"absoluteUrl":"https://10.0.1.38:444/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"/config/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":254,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"absoluteUrl":"https://10.0.1.38:444/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/config/www/nextcloud/apps/dav/lib/Server.php","line":301,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/config/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"/config/www/nextcloud/remote.php","line":163,"args":["/config/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/config/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","Line":107,"Previous":{"Exception":"PDOException","Message":"SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction","Code":"40001","Trace":[{"file":"/config/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","line":105,"function":"execute","class":"PDOStatement","type":"->","args":[[1553854625,"files/94fd320b6359acc490e1a6f373a6098b"]]},{"file":"/config/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1008,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDOStatement","type":"->","args":[[1553854625,"files/94fd320b6359acc490e1a6f373a6098b"]]},{"file":"/config/www/nextcloud/lib/private/DB/Connection.php","line":216,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->","args":["UPDATE `oc_file_locks` SET `lock` = `lock` + 1, `ttl` = ? WHERE `key` = ? AND `lock` >= 0",[1553854625,"files/94fd320b6359acc490e1a6f373a6098b"],[]]},{"file":"/config/www/nextcloud/lib/private/Lock/DBLockingProvider.php","line":196,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->","args":["UPDATE `oc_file_locks` SET `lock` = `lock` + 1, `ttl` = ? WHERE `key` = ? AND `lock` >= 0",[1553854625,"files/94fd320b6359acc490e1a6f373a6098b"]]},{"file":"/config/www/nextcloud/lib/private/Files/Storage/Common.php","line":714,"function":"acquireLock","class":"OC\\Lock\\DBLockingProvider","type":"->","args":["files/94fd320b6359acc490e1a6f373a6098b",1]},{"file":"/config/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":593,"function":"acquireLock","class":"OC\\Files\\Storage\\Common","type":"->","args":["files/Projects (Selective Sync Conflict)",1,{"__class__":"OC\\Lock\\DBLockingProvider"}]},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":1923,"function":"acquireLock","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["files/Projects (Selective Sync Conflict)",1,{"__class__":"OC\\Lock\\DBLockingProvider"}]},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":2037,"function":"lockPath","class":"OC\\Files\\View","type":"->","args":["/Projects (Selective Sync Conflict)",1]},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":1132,"function":"lockFile","class":"OC\\Files\\View","type":"->","args":["/Projects (Selective Sync Conflict)/Squire (Selective Sync Conflict)/Squire iPad/PaintCode/SVG logos",1]},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":268,"function":"basicOperation","class":"OC\\Files\\View","type":"->","args":["mkdir","/Projects (Selective Sync Conflict)/Squire (Selective Sync Conflict)/Squire iPad/PaintCode/SVG logos",["create","write"]]},{"file":"/config/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":189,"function":"mkdir","class":"OC\\Files\\View","type":"->","args":["/Projects (Selective Sync Conflict)/Squire (Selective Sync Conflict)/Squire iPad/PaintCode/SVG logos"]},{"file":"/config/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1210,"function":"createDirectory","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->","args":["SVG logos"]},{"file":"/config/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":590,"function":"createCollection","class":"Sabre\\DAV\\Server","type":"->","args":["files/jonatan/Projects (Selective Sync Conflict)/Squire (Selective Sync Conflict)/Squire iPad/PaintCode/SVG logos",{"__class__":"Sabre\\DAV\\MkCol"}]},{"function":"httpMkcol","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"absoluteUrl":"https://10.0.1.38:444/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/config/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php","line":105,"function":"call_user_func_array","args":[[{"__class__":"Sabre\\DAV\\CorePlugin"},"httpMkcol"],[{"absoluteUrl":"https://10.0.1.38:444/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"/config/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":479,"function":"emit","class":"Sabre\\Event\\EventEmitter","type":"->","args":["method:MKCOL",[{"absoluteUrl":"https://10.0.1.38:444/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"/config/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":254,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"absoluteUrl":"https://10.0.1.38:444/remote.php/dav/files/jonatan/Projects%20(Selective%20Sync%20Conflict)/Squire%20(Selective%20Sync%20Conflict)/Squire%20iPad/PaintCode/SVG%20logos","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/config/www/nextcloud/apps/dav/lib/Server.php","line":301,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/config/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"/config/www/nextcloud/remote.php","line":163,"args":["/config/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/config/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","Line":105}},"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36","version":"15.0.2.0","id":"5c9de5b0409e8"}

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'datadirectory' => '/data',
  'instanceid' => 'ockgadl8xwcm',
  'passwordsalt' => 'redacted',
  'secret' => 'redacted',
  'trusted_domains' => 
  array (
    0 => '10.0.1.38:444',
  ),
  'dbtype' => 'mysql',
  'version' => '15.0.2.0',
  'overwrite.cli.url' => 'https://10.0.1.38:444',
  'dbname' => 'nextcloud',
  'dbhost' => '10.0.1.38',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud',
  'dbpassword' => 'redacted',
  'installed' => true,
);

[linucksrox]

Uploading via the web app is not great when dealing with either very large files or a large number of files. What issues did you run into when trying to sync using the desktop client? Syncing through the desktop client is definitely the recommended way to go in this situation because it will handle network interruptions and other issues gracefully.
If you mean you tried to “preseed” the data directly on the server, that’s also possible but requires a few more steps. You would need to place the data in the correct location obviously, but then you need to tell Nextcloud to scan the data so it can update its file cache database. On the server you would run as the web user php /path/to/nextcloud/occ files:scan --all

[bluepr0]

Thanks for your reply @linucksrox!

Gotcha, I figured the desktop app could be indeed a better option, but just wasn’t sure if this errors were a Chrome or a Nextcloud thing!

Also, about uploading data directly to server… that would be ideal to me. I’m running nextcloud on docker so I will need to find out how to do a scan, but at least I’m glad it’s possible.

Thanks once again!

[linucksrox]

No problem. The documentation for how to run occ within the docker container is here: https://github.com/nextcloud/docker#using-the-nextcloud-command-line-interface
It depends on how you deployed the container (manually or using compose/swarm).

[bluepr0]

I’ve tried using the desktop app but still I’m getting tons of errors. Now I’m not entirely sure if these files will be later tried to be uploaded or if they ended up not being synced

Also on the logs

[linucksrox]

Dang, with all those gateway timeout errors and other errors in the log it sounds like there’s a problem with the nextcloud installation or possibly a bad network connection or underpowered hardware. I’m not sure where to start, but I would start investigating those log errors first to see if you can narrow down what would cause them.

[bluepr0]

Thanks so much! I will try again but also try coping directly the data to the server and update later the database

It’s weird because I have quite a few of GB that were not synced and I was hoping the desktop app will take care of timeout errors and try again later

[linucksrox]

The client should retry eventually but I’m not sure how long you should expect to wait. You could click the option on the drop-down for the sync connection and click the force sync option, or try to pause then unpause the connection, or even log out and log back in with the client to get it to try again.

[bluepr0]

I found something that might be affecting, I’ve got setup a default value of 512MB as a max file size, but I do have bigger files than that. I’ve changed that will try again!

[bluepr0]

Alright so I’ve been trying to force sync but still I’m unable to sync all the files. As you can see the Dropbox folder is 251GB but it’s not syncing all of it. I’m actually directly syncing this same Dropbox folder so the data is there.

I’m also not getting any errors or files that are not being synced. Not entirely sure how to debug other than erase all nextcloud data, copying it manually through Samba and then update manually the data base through the docker command

[bluepr0]

Just run the occ scan all files command but I’m still not getting all the files synced. Again, even if the folder is the Dropbox one (so all the files are really there)

[linucksrox]

Any idea which files are not syncing? I might start by checking the ignored files list in the nextcloud client.

[bluepr0]

No, sadly I have no idea which files is not syncing because it’s actually syncing same Dropbox folder.

As far as I’m seeing the desktop app doesn’t have any ignored files or excluded from syncing

[linucksrox]

There are probably other ways, but I would start by comparing one subfolder at a time. Compare one folder on your local machine to the corresponding folder that you download from Nextcloud to make sure the size and number of items matches. Then proceed to the next folder, until you reach a mismatch. At that point go through the subfolders within that folder, etc. until you figure out specifically which files are not on the cloud.
Also, now that the Nextcloud client says it’s fully synced, if you click the Activity tab at the top are there any errors in that list about certain files that could not be uploaded?

[bluepr0]

I found a few folders that are 2gb less each one… but does this means it’s a bug from Nextcloud or how I can debug the error?

No errors on the Activity tab

[linucksrox]

Ok, did you find out exactly which files are affected? If you can isolate one specific file that is not syncing to your server, then look at the sync status for that file and see if it shows as being synced or something else. You could also copy that file to the root of your folder and see if it syncs the copy.

As I was thinking more about this, it might be worth mentioning that hidden files are visible.

[bluepr0]

Sorry for my late reply… but no, I couldn’t find out if there were missing files.

I made another experiment and deleted everything in the nextcloud server. Then I moved all through Samba directly from my local Dropbox folder to my nextcloud folder.

Then executed the database build so all files become visible. However I’m still getting 216gb in the Nextcloud webui, but if I check local folder info on macOS is around 232gb. If I use the file explorer in my nextcloud server it’s also shows 216gb.

Could it be that Linux (where Nextcloud is) and Mac use different methods to calculate folder size?. Even with that it shouldn’t be that different I guess.

Back when I tried the nextcloud desktop client for macOS I also made sure the excluded files on settings were set to none, so theoretically all files should have synced.

I literally have no clue what’s going on. Maybe there’s a way to compare with an app/script all the folder/files structure on both the nextcloud folder and my dropbox folder and see if it’s missing something?

Thanks!

[linucksrox]

I would be comparing file and folder counts at this point. I’m sure there’s some way to automate the comparison of entire directory structures and file sizes, but I’m not familiar with any such tools on macOS. I use WinMerge in Windows, so perhaps you can find an alternative to that.
I have seen large discrepancies between file sizes on different filesystems, and it can vary wildly depending on the size of each filesystem block and how many files you have (a larger number of smaller files can have a greater difference between actual total size and total disk space used for example because of wasted block space).
Having said that I would still want to be sure, so it’s worth the exercise of figuring out exactly what is different. Like I said before, even if you traverse manually and find one difference (one file shows a different size on Nextcloud and Dropbox) that will probably give you a big clue.

[bluepr0]

I have checked quite a few folders that I know they had a lot of files inside and all of them there’s differences between files quantity. Except for the Projects folder which has more files (WTF?) on Nextcloud folder, the rest have less than the original

[linucksrox]

Well that confirms it’s not a filesystem calculation issue. The gateway timeout errors and Sabre/Dav errors in the log are very concerning. I don’t trust that your Nextcloud installation is working correctly, so perhaps you should investigate those errors more closely and maybe start with a smaller set of data that is easier to troubleshoot.