Internal server error uploading from webbrowser?

Hi,
When I’m trying to upload files trough the webbbrowser many files are uploading but some get’s “internal error”.
This only happens when I’m trying to upload many files at once.

Take this folder for a example, it’s 764 files but I could only upload 762 files.

This is what I could find in the error log, and it seem to bee the same error messages at everyone:

Error	index	Doctrine\DBAL\Exception\DriverException: An exception occurred while executing 'INSERT INTO `oc_filecache` (`mimepart`,`mimetype`,`mtime`,`size`,`etag`,`storage_mtime`,`permissions`,`parent`,`checksum`,`path_hash`,`path`,`name`,`storage`) SELECT ?,?,?,?,?,?,?,?,?,?,?,?,? FROM `oc_filecache` WHERE `storage` = ? AND `path_hash` = ? HAVING COUNT(*) = 0' with params ["10", "12", 1499009068, 0, "4a9574473066234b51d2af261b8bb81e", 1499009068, 27, 6293, "", "b56bdcd759861f1bcfba1e732e307151", "appdata_oci8jjq5tx7d\/preview\/6287\/540-960-max.png", "540-960-max.png", 2, 2, "b56bdcd759861f1bcfba1e732e307151"]: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction

    /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php - line 128: Doctrine\DBAL\Driver\AbstractMySQLDriver->convertException('An exception oc...', Object(Doctrine\DBAL\Driver\PDOException))
    /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php - line 1015: Doctrine\DBAL\DBALException driverExceptionDuringQuery(Object(Doctrine\DBAL\Driver\PDOMySql\Driver), Object(Doctrine\DBAL\Driver\PDOException), 'INSERT INTO `oc...', Array)
    /var/www/nextcloud/lib/private/DB/Connection.php - line 213: Doctrine\DBAL\Connection->executeUpdate('INSERT INTO `oc...', Array, Array)
    /var/www/nextcloud/lib/private/DB/Adapter.php - line 114: OC\DB\Connection->executeUpdate('INSERT INTO `oc...', Array)
    /var/www/nextcloud/lib/private/DB/Connection.php - line 251: OC\DB\Adapter->insertIfNotExist('*PREFIX*filecac...', Array, Array)
    /var/www/nextcloud/lib/private/Files/Cache/Cache.php - line 273: OC\DB\Connection->insertIfNotExist('*PREFIX*filecac...', Array, Array)
    /var/www/nextcloud/lib/private/Files/Cache/Cache.php - line 230: OC\Files\Cache\Cache->insert('appdata_oci8jjq...', Array)
    /var/www/nextcloud/lib/private/Files/Cache/Scanner.php - line 279: OC\Files\Cache\Cache->put('appdata_oci8jjq...', Array)
    /var/www/nextcloud/lib/private/Files/Cache/Scanner.php - line 216: OC\Files\Cache\Scanner->addToCache('appdata_oci8jjq...', Array, -1)
    /var/www/nextcloud/lib/private/Files/Cache/Scanner.php - line 322: OC\Files\Cache\Scanner->scanFile('appdata_oci8jjq...', 3, 6293, false, false)
    /var/www/nextcloud/lib/private/Files/Cache/Updater.php - line 124: OC\Files\Cache\Scanner->scan('appdata_oci8jjq...', false, 3, false)
    /var/www/nextcloud/lib/private/Files/View.php - line 321: OC\Files\Cache\Updater->update('appdata_oci8jjq...', 1499009068)
    /var/www/nextcloud/lib/private/Files/View.php - line 1147: OC\Files\View->writeUpdate(Object(OCA\Files_Trashbin\Storage), 'appdata_oci8jjq...')
    /var/www/nextcloud/lib/private/Files/View.php - line 559: OC\Files\View->basicOperation('touch', '/appdata_oci8jj...', Array, NULL)
    /var/www/nextcloud/lib/private/Files/Node/Folder.php - line 181: OC\Files\View->touch('/appdata_oci8jj...')
    /var/www/nextcloud/lib/private/Files/SimpleFS/SimpleFolder.php - line 83: OC\Files\Node\Folder->newFile('540-960-max.png')
    /var/www/nextcloud/lib/private/Preview/Generator.php - line 170: OC\Files\SimpleFS\SimpleFolder->newFile('540-960-max.png')
    /var/www/nextcloud/lib/private/Preview/Generator.php - line 110: OC\Preview\Generator->getMaxPreview(Object(OC\Files\SimpleFS\SimpleFolder), Object(OC\Files\Node\File), 'image/png')
    /var/www/nextcloud/lib/private/PreviewManager.php - line 201: OC\Preview\Generator->getPreview(Object(OC\Files\Node\File), 40, 40, true, 'fill', 'image/png')
    /var/www/nextcloud/core/Controller/PreviewController.php - line 113: OC\PreviewManager->getPreview(Object(OC\Files\Node\File), 40, 40, true, 'fill')
    [internal function] OC\Core\Controller\PreviewController->getPreview(Object(OC\Files\Node\File), 40, 40, false, false, 'fill')
    /var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php - line 160: call_user_func_array(Array, Array)
    /var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php - line 90: OC\AppFramework\Http\Dispatcher->executeController(Object(OC\Core\Controller\PreviewController), 'getPreview')
    /var/www/nextcloud/lib/private/AppFramework/App.php - line 114: OC\AppFramework\Http\Dispatcher->dispatch(Object(OC\Core\Controller\PreviewController), 'getPreview')
    /var/www/nextcloud/lib/private/AppFramework/Routing/RouteActionHandler.php - line 47: OC\AppFramework\App main('OC\\Core\\Control...', 'getPreview', Object(OC\AppFramework\DependencyInjection\DIContainer), Array)
    [internal function] OC\AppFramework\Routing\RouteActionHandler->__invoke(Array)
    /var/www/nextcloud/lib/private/Route/Router.php - line 299: call_user_func(Object(OC\AppFramework\Routing\RouteActionHandler), Array)
    /var/www/nextcloud/lib/base.php - line 1000: OC\Route\Router->match('/core/preview.p...')
    /var/www/nextcloud/index.php - line 40: OC handleRequest()
    {main}

Edit:
I did try to upload an other folder and I did get error on 4 files, some of the files did show the same error messages as above but now this messages did also accure, it’s regarding webdav.

Fatal	webdav	Doctrine\DBAL\Exception\DriverException: An exception occurred while executing 'INSERT INTO `oc_filecache` (`mimepart`,`mimetype`,`mtime`,`size`,`etag`,`storage_mtime`,`permissions`,`parent`,`checksum`,`path_hash`,`path`,`name`,`storage`) SELECT ?,?,?,?,?,?,?,?,?,?,?,?,? FROM `oc_filecache` WHERE `storage` = ? AND `path_hash` = ? HAVING COUNT(*) = 0' with params ["10", "11", 1499009531, 5356394, "b300dfb3832505108a75695269f5a075", 1499009531, 27, 6499, "", "b3cff09a74a8b52a44147b1c768181db", "files\/Bilder\/ExternHDD\/kaamera\/DSC_0140.JPG", "DSC_0140.JPG", 3, 3, "b3cff09a74a8b52a44147b1c768181db"]: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction

    /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php - line 128: Doctrine\DBAL\Driver\AbstractMySQLDriver->convertException('An exception oc...', Object(Doctrine\DBAL\Driver\PDOException))
    /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php - line 1015: Doctrine\DBAL\DBALException driverExceptionDuringQuery(Object(Doctrine\DBAL\Driver\PDOMySql\Driver), Object(Doctrine\DBAL\Driver\PDOException), 'INSERT INTO `oc...', Array)
    /var/www/nextcloud/lib/private/DB/Connection.php - line 213: Doctrine\DBAL\Connection->executeUpdate('INSERT INTO `oc...', Array, Array)
    /var/www/nextcloud/lib/private/DB/Adapter.php - line 114: OC\DB\Connection->executeUpdate('INSERT INTO `oc...', Array)
    /var/www/nextcloud/lib/private/DB/Connection.php - line 251: OC\DB\Adapter->insertIfNotExist('*PREFIX*filecac...', Array, Array)
    /var/www/nextcloud/lib/private/Files/Cache/Cache.php - line 273: OC\DB\Connection->insertIfNotExist('*PREFIX*filecac...', Array, Array)
    /var/www/nextcloud/lib/private/Files/Cache/Cache.php - line 230: OC\Files\Cache\Cache->insert('files/Bilder/Ex...', Array)
    /var/www/nextcloud/lib/private/Files/Cache/Scanner.php - line 279: OC\Files\Cache\Cache->put('files/Bilder/Ex...', Array)
    /var/www/nextcloud/lib/private/Files/Cache/Scanner.php - line 216: OC\Files\Cache\Scanner->addToCache('files/Bilder/Ex...', Array, -1)
    /var/www/nextcloud/lib/private/Files/Cache/Scanner.php - line 322: OC\Files\Cache\Scanner->scanFile('files/Bilder/Ex...', 3, 6499, false, false)
    /var/www/nextcloud/lib/private/Files/Cache/Updater.php - line 124: OC\Files\Cache\Scanner->scan('files/Bilder/Ex...', false, 3, false)
    /var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php - line 199: OC\Files\Cache\Updater->update('files/Bilder/Ex...')
    /var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php - line 151: OCA\DAV\Connector\Sabre\File->put(Resource id #20)
    /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 1095: OCA\DAV\Connector\Sabre\Directory->createFile('DSC_0140.JPG', Resource id #20)
    /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 525: Sabre\DAV\Server->createFile('Bilder/ExternHD...', Resource id #20, NULL)
    [internal function] Sabre\DAV\CorePlugin->httpPut(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
    /var/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105: call_user_func_array(Array, Array)
    /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 479: Sabre\Event\EventEmitter->emit('method PUT', Array)
    /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 254: Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
    /var/www/nextcloud/apps/dav/appinfo/v1/webdav.php - line 71: Sabre\DAV\Server->exec()
    /var/www/nextcloud/remote.php - line 162: require_once('/var/www/nextcl...')
    {main}

Have you configured redis as filelocking-cache? This should improve the speed on overall reliability of the filelocking feature.

No I have not done that, how do I do that?
And what setup do you recommend for it?

My server can handle it so I don’t know why its a issue sometimes.
My server is a:
Xeon e3-1265l v2
16GB ram
And nextcloud is on a sSd disk.

https://docs.nextcloud.com/server/12/admin_manual/configuration_server/caching_configuration.html

You need root permissions on your server to install and configure it. Your server is able to handle it and it should speed up the web-interface and file transactions.

Ok, I have root.

Can I just follow this guide but replace own cloud with nextcloud?

Do I need to open Reid’s ports in my firewall?

So the issue is because that I did upload to many files at once so it jumped over some files of some reason?

Do it still work if the server have only 8GB of ram?
It was pretty streight forward in the manual that you linked, thanks.

In theory yes, but only if you are using Ubuntu 14.04. In that case, I’d rather recommend you to upgrade to Ubuntu 16.04, php7.0 is much faster and you can use the caching packages directly from your distribution without compiling anything on your own.

If it is the same server, you can connect through sockets, there you probably don’t need anything to set in your firewall.

You can even use it on a Raspberry Pi 2/3 which has only 1GB RAM.

I have 16.04 LTS so I guess that I can’t use it.

I’m currently in my bed but and my wife’s phone are uploading around 6000 pics to my nextcloud trough the android app it have been for about 1h should bee finish soon but I did receive this email now:

The sender was www-data:

Subject:
Cron www-data@mu-domain php -f /var/www/nextcloud/occ preview:pre-generate >> /var/log/previewgenerator.log

Mail text:
/bin/sh: 1: cannot create /var/log/previewgenerator.log: Permission denied

touch /var/log/previewgenerator.log
chown www-data.www-data /var/log/previewgenerator.log

Thanks!
I’ll fix it tomorrow before I’m going to work.

I already have redis installed.
So any other suggestion to why this error messages does appear when I’m transferring many files at the same time?
It’s only sometimes also I did upload 11GB of pictures from my computer and it did work without any issue.

Redis is also configured in Nextcloud (in your config/config.php). In this case you can still improve your database settings and set reasonable cache sizes (mysqltuner or tuning-primer.sh script can help you to find suitable settings). This way, the database can become much faster.

Question, is this somehow bad for the server or is it just that if somefile (just a few) are stuck in this loop they 'll not bee uploaded?