Sync - Some Files are blocked/error during sync and then synced after a second/third try - No App In Context

Hello Everybody,

I got a weird problem on a new Nextcloud Installation, here is the config :
Qnap 6 Disks of 4TB in RAID 6 -
Ubuntu 16.04 (fully updated) in a VM inside the QNAP - 14 GB of RAM
Nextcloud 13.0.2
MariaDB 10.2
Php 7.0
Apache2
Redis 3.0.6 - caching and locking

When someone upload from Nextcloud client a folder with files inside, i got a lot of errors with Database Deadlocks try restarting …
I reviewed several topics and i am in the good mode of READ-Commited and i disabled MariaDB binlog, but still get the same problem…

When i watch with HTOP or Netdata the server i quite all memory taken by Cached Memory, but i don’t know witch process is using it.

On my house server with the same config but with only 4GB of Ram, i don’t have a lot of Ram taken from Cached Memory.

Can someone help me to resolve this, i think it has something to do with Redis in his configuration.


this is Home Server


And this is the server of my client.

Edit here is the log i get :

    {"reqId":"WJSH3j9VOki8a3KF5BRM","level":3,
"time":"2018-04-27T16:38:44+00:00",
"remoteAddr":"10.114.45.5","user":"USER",
"app":"no app in context","method":"PUT",
"url":"\/remote.php\/dav\/files\/USER\/images\/Nouveau%20dossier\/IMGP3441.JPG",
"message":"Exception: {\"Exception\":\"OCP\\\\AppFramework\\\\QueryException\",\
"Message\":\"Could not resolve filesService! Class filesService does not exist\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/AppFramework\\\/Utility\\\/SimpleContainer.php(118): OC\\\\AppFramework\\\\Utility\\\\SimpleContainer->resolve('filesService')\
    #1 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/ServerContainer.php(132): OC\\\\AppFramework\\\\Utility\\\\SimpleContainer->query('filesService')\
    #2 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/AppFramework\\\/DependencyInjection\\\/DIContainer.php(441): OC\\\\ServerContainer->query('filesService')\
    #3 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/AppFramework\\\/Utility\\\/SimpleContainer.php(74): OC\\\\AppFramework\\\\DependencyInjection\\\\DIContainer->query('filesService')\
    #4 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/AppFramework\\\/Utility\\\/SimpleContainer.php(97): OC\\\\AppFramework\\\\Utility\\\\SimpleContainer->buildClass(Object(ReflectionClass))\
    #5 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/AppFramework\\\/Utility\\\/SimpleContainer.php(118): OC\\\\AppFramework\\\\Utility\\\\SimpleContainer->resolve('OCA\\\\\\\\Files_FullT...')\
    #6 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/AppFramework\\\/DependencyInjection\\\/DIContainer.php(467): OC\\\\AppFramework\\\\Utility\\\\SimpleContainer->query('OCA\\\\\\\\Files_FullT...')\
    #7 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/AppFramework\\\/DependencyInjection\\\/DIContainer.php(438): OC\\\\AppFramework\\\\DependencyInjection\\\\DIContainer->queryNoFallback('OCA\\\\\\\\Files_FullT...')\
    #8 \\\/var\\\/www\\\/nextcloud\\\/apps\\\/files_fulltextsearch\\\/lib\\\/Hooks\\\/FilesHooks.php(52): OC\\\\AppFramework\\\\DependencyInjection\\\\DIContainer->query('OCA\\\\\\\\Files_FullT...')\
    #9 \\\/var\\\/www\\\/nextcloud\\\/apps\\\/files_fulltextsearch\\\/lib\\\/Hooks\\\/FilesHooks.php(66): OCA\\\\Files_FullTextSearch\\\\Hooks\\\\FilesHooks::getController()\
    #10 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/legacy\\\/hook.php(106): OCA\\\\Files_FullTextSearch\\\\Hooks\\\\FilesHooks::onNewFile(Array)\
    #11 \\\/var\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/File.php(309): OC_Hook::emit('OC_Filesystem', 'post_create', Array)\
    #12 \\\/var\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/File.php(241): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->emitPostHooks(false)\
    #13 \\\/var\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/Directory.php(151): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->put(Resource id #22)\
    #14 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1096): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\Directory->createFile('IMGP3441.JPG', Resource id #22)\
    #15 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(525): Sabre\\\\DAV\\\\Server->createFile('files\\\/USER...', Resource id #22, NULL)\
    #16 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPut(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\
    #17 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\
    #18 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(479): Sabre\\\\Event\\\\EventEmitter->emit('method:PUT', Array)\
    #19 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\
    #20 \\\/var\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Server.php(283): Sabre\\\\DAV\\\\Server->exec()\
    #21 \\\/var\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(35): OCA\\\\DAV\\\\Server->exec()\
    #22 \\\/var\\\/www\\\/nextcloud\\\/remote.php(164): require_once('\\\/var\\\/www\\\/nextcl...')\
    #23 {main}\",\"File\":\"\\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/AppFramework\\\/Utility\\\/SimpleContainer.php\",\"Line\":103}","userAgent":"Mozilla\/5.0 (Macintosh) mirall\/2.3.3 (build 84) (Nextcloud)","version":"13.0.2.1"}

EDIT 2 : I disable REDIS and do the same test, i got the same amount of errors with the same logs… So REDIS is out of the problem…

UP : I still get a lot of DEADLOCKS when i try to upload several files.

Tough issue and I’m not sure if I can help. One thing I noticed is the part of the log entry mentioned above.
Maybe the issue is caused by Full Text Search. I haven’t tested it yet, but isn’t it BETA and furthermore relying on an additional service? If the latter is the case, then maybe the additional service is not reachable/ not correctly configured or something else?
Would you mind trying to disable Full Text Search and see if the load when uploading files is reduced?

I got this errors before adding this App, i deleted since, so it isn’t the culprit.
I also disabled REDIS caching and locking, still the same…

So although this app is deleted, the app name occurs in the logs? :thinking: :open_mouth:

Can you post your config.php, web server config, list of all enabled apps and latest nextcloud.logs?
Maybe the output of the browser console during upload could be interesting as well.

Sorry, I’m just trying to find some hints where this might coming from.

I’m seeing doing my test with the same folder everytime (2,7Go) i always stuck on the same files…
Some .indd and .jpg… but even if i rename the file it always stuck on them…
After the first sync, they get sync well…

Now it don’t appear look at this :
{“reqId”:“cXninaVmj04X3oCNHtdv”,“level”:4,“time”:“2018-05-08T13:51:19+00:00”,“remoteAddr”:“10.114.45.5”,“user”:“USER”,“app”:“webdav”,“method”:“MKCOL”,“url”:"/remote.php/dav/uploads/USER/638489116",“message”:“Exception: {“Exception”:“Doctrine\\DBAL\\Exception\\DriverException”,“Message”:“An exception occurred while executing ‘INSERT INTO oc_filecache (mimepart,mimetype,mtime,size,etag,storage_mtime,permissions,parent,checksum,path_hash,path,name,storage) SELECT ?,?,?,?,?,?,?,?,?,?,?,?,? FROM oc_filecache WHERE storage = ? AND path_hash = ? HAVING COUNT(*) = 0’ with params [\“1\”, \“2\”, 1525787468, -1, \“5af1ab56cc0b6\”, 1525787468, 31, 727140, \”\”, \“e5017d851f0ff9350548ee6ec8aa2bd0\”, \“uploads\\\/638489116\”, \“638489116\”, 27, 27, \“e5017d851f0ff9350548ee6ec8aa2bd0\”]:
\SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction",“Code”:0,“Trace”:"#0 \/var\/www\/nextcloud\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/DBALException.php(128): Doctrine\\DBAL\\Driver\\AbstractMySQLDriver->convertException(‘An exception oc…’, Object(Doctrine\\DBAL\\Driver\\PDOException))
#1 \/var\/www\/nextcloud\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Connection.php(1015): Doctrine\\DBAL\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\DBAL\\Driver\\PDOMySql\\Driver), Object(Doctrine\\DBAL\\Driver\\PDOException), ‘INSERT INTO oc...', Array)\ #2 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/DB\\\/Connection.php(216): Doctrine\\\\DBAL\\\\Connection->executeUpdate('INSERT INTOoc…’, Array, Array)
#3 \/var\/www\/nextcloud\/lib\/private\/DB\/Adapter.php(114): OC\\DB\\Connection->executeUpdate(‘INSERT INTO `oc…’, Array)
#4 \/var\/www\/nextcloud\/lib\/private\/DB\/Connection.php(254): OC\\DB\\Adapter->insertIfNotExist(‘PREFIXfilecac…’, Array, Array)
#5 \/var\/www\/nextcloud\/lib\/private\/Files\/Cache\/Cache.php(273): OC\\DB\\Connection->insertIfNotExist(‘PREFIXfilecac…’, Array, Array)
#6 \/var\/www\/nextcloud\/lib\/private\/Files\/Cache\/Cache.php(230): OC\\Files\\Cache\\Cache->insert(‘uploads\/6384891…’, Array)
#7 \/var\/www\/nextcloud\/lib\/private\/Files\/Cache\/Scanner.php(292): OC\\Files\\Cache\\Cache->put(‘uploads\/6384891…’, Array)
#8 \/var\/www\/nextcloud\/lib\/private\/Files\/Cache\/Scanner.php(221): OC\\Files\\Cache\\Scanner->addToCache(‘uploads\/6384891…’, Array, -1)
#9 \/var\/www\/nextcloud\/lib\/private\/Files\/Cache\/Scanner.php(335): OC\\Files\\Cache\\Scanner->scanFile(‘uploads\/6384891…’, 3, 727140, false, false)
#10 \/var\/www\/nextcloud\/lib\/private\/Files\/Cache\/Updater.php(124): OC\\Files\\Cache\\Scanner->scan(‘uploads\/6384891…’, false, 3, false)
#11 \/var\/www\/nextcloud\/lib\/private\/Files\/View.php(319): OC\\Files\\Cache\\Updater->update(‘uploads\/6384891…’, 1525787478)
#12 \/var\/www\/nextcloud\/lib\/private\/Files\/View.php(1163): OC\\Files\\View->writeUpdate(Object(OCA\\Files_Trashbin\\Storage), ‘uploads\/6384891…’)
#13 \/var\/www\/nextcloud\/lib\/private\/Files\/View.php(267): OC\\Files\\View->basicOperation(‘mkdir’, ‘\/638489116’, Array)
#14 \/var\/www\/nextcloud\/apps\/dav\/lib\/Connector\/Sabre\/Directory.php(180): OC\\Files\\View->mkdir(’\/\/638489116’)
#15 \/var\/www\/nextcloud\/apps\/dav\/lib\/Upload\/UploadHome.php(47): OCA\\DAV\\Connector\\Sabre\\Directory->createDirectory(‘638489116’)
#16 \/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php(1210): OCA\\DAV\\Upload\\UploadHome->createDirectory(‘638489116’)
#17 \/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/CorePlugin.php(590): Sabre\\DAV\\Server->createCollection(‘uploads\/Service…’, Object(Sabre\\DAV\\MkCol))
#18 [internal function]: Sabre\\DAV\\CorePlugin->httpMkcol(Object(Sabre\\HTTP\\Request), Object(Sabre\\HTTP\\Response))
#19 \/var\/www\/nextcloud\/3rdparty\/sabre\/event\/lib\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)
#20 \/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php(479): Sabre\\Event\\EventEmitter->emit(‘method:MKCOL’, Array)
#21 \/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php(254): Sabre\\DAV\\Server->invokeMethod(Object(Sabre\\HTTP\\Request), Object(Sabre\\HTTP\\Response))
#22 \/var\/www\/nextcloud\/apps\/dav\/lib\/Server.php(283): Sabre\\DAV\\Server->exec()
#23 \/var\/www\/nextcloud\/apps\/dav\/appinfo\/v2\/remote.php(35): OCA\\DAV\\Server->exec()
#24 \/var\/www\/nextcloud\/remote.php(164): require_once(’\/var\/www\/nextcl…’)
#25 {main}",“File”:"\/var\/www\/nextcloud\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Driver\/AbstractMySQLDriver.php",“Line”:115}",“userAgent”:“Mozilla/5.0 (Macintosh) mirall/2.3.3 (build 84) (Nextcloud)”,“version”:“13.0.2.1”}

you could find that here : https://github.com/nextcloud/server/issues/9351

When i started this topic i had FullTextSearch BETA enabled, but i got this issue before install and after uninstall.

Now i don’t have it anymore and i don’t see it in the logs.

Alright, very important point that the logs are different meanwhile. Everybody will always assume full text search is the reason, because this log entry is actually describing another issue. I suggest you edit both your initial posts here in the forum and on github and replace the old log output with the new one to reflect the real issue with filecache - and the title maybe.

While the issue is already reported on github and developers are involved, I believe it is in best hands there. We could just update this thread here with the later solution from github.

1 Like