Failure with multiple files upload (1 of 10 regularly fails) with S3 primary storage

ℹ️ Support
1

Hi all, here is the issue I saw many times in my environment. I described just one example but it happens regularly if I upload many large files simultaneously.

Thank you in advance!

Steps to reproduce

  1. Try to upload 10 large files with 5GB size using web interface.
  2. 9 files were uploaded successfully.
  3. 1 file was not uploaded.

Expected behaviour

All files uploaded.

Actual behaviour

One file was not uploaded.

Server configuration

OS: RedHat 7.7
Nextcloud: 16.0.4

Signing status:

No errors have been found.

Nextcloud configuration:

<?php
$CONFIG = array (
  'objectstore' =>
  array (
    'class' => '\\OC\\Files\\ObjectStore\\S3',
    'arguments' =>
    array (
      'bucket' => 'driveline-mgb-2019-10-21',
      'autocreate' => true,
      'key' => ‘xxx',
      'secret' => ‘xxx',
      'hostname' => ‘s3.company.com',
      'use_ssl' => true,
      'region' => 'nord1',
    ),
  ),
  'passwordsalt' => ‘xxx',
  'secret' => ‘xxx',
  'trusted_domains' =>
  array (
    0 => 'localhost',
    1 => 'disk.[company.com](http://company.com)',
  ),
  'datadirectory' => '/var/www/nextcloud/data',
  'dbtype' => 'pgsql',
  'version' => '16.0.4.1',
  'overwrite.cli.url' => 'http://localhost',
  'dbname' => 'nextcloud',
  'dbhost' => '127.0.0.1',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'nextcloud',
  'dbpassword' => ‘xxx',
  'installed' => true,
  'instanceid' => '',
  'memcache.local' => '\OC\Memcache\APCu',
  'maintenance' => false,
);

Are you using encryption: yes/no

No

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/…

No

Client configuration

Browser:
Chrome 78.0.3904.108
Operating system:
MacOS Catalina

Logs

Aws\S3\Exception\S3MultipartUploadException: An exception occurred while uploading parts to a multipart upload. The following parts had errors: - Part 6: Error executing "UploadPart" on "[https://driveline-ieng5ziej-2019-09-13.s3.company.com/urn%3Aoid%3A9981?partNumber=6&uploadId=MDA2ODkxMTUxNTc0OTcxNTY2NjY0";](https://driveline-ieng5ziej-2019-09-13.s3.company.com/urn%3Aoid%3A9981?partNumber=6&uploadId=MDA2ODkxMTUxNTc0OTcxNTY2NjY0) AWS HTTP error: cURL error 55: TCP connection reset by peer (see http://curl.haxx.se/libcurl/c/libcurl-errors.html) (server): 100 Continue - {"reqId":"GYOVa4RkFp5xIL6EYPbm","level":3,"time":"2019-11-28T20:09:55+00:00","remoteAddr":"46.39.51.247","user":"Sergey","app":"objectstore","method":"MOVE","url":"\/remote.php\/dav\/uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","message":{"Exception":"Aws\\S3\\Exception\\S3MultipartUploadException","Message":"An exception occurred while uploading parts to a multipart upload. The following parts had errors:\n- Part 6: Error executing \"UploadPart\" on \"https:\/\/[driveline-ieng5ziej-2019-09-13.s3.company.com](http://driveline-ieng5ziej-2019-09-13.s3.company.com)\/urn%3Aoid%3A9981?partNumber=6&uploadId=MDA2ODkxMTUxNTc0OTcxNTY2NjY0\"; AWS HTTP error: cURL error 55: TCP connection reset by peer (see http:\/\/curl.haxx.se\/libcurl\/c\/libcurl-errors.html) (server): 100 Continue - \n","Code":0,"Trace":[{"function":"Aws\\Multipart\\{closure}","class":"Aws\\Multipart\\AbstractUploadManager","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Coroutine.php","line":122,"function":"send","class":"Generator","type":"->","args":[null]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":203,"function":"_handleSuccess","class":"GuzzleHttp\\Promise\\Coroutine","type":"->","args":[null]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":156,"function":"callHandler","class":"GuzzleHttp\\Promise\\Promise","type":"::","args":[1,null,[{"__class__":"GuzzleHttp\\Promise\\Promise"},[{"__class__":"GuzzleHttp\\Promise\\Coroutine"},"_handleSuccess"],[{"__class__":"GuzzleHttp\\Promise\\Coroutine"},"_handleFailure"]]]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/TaskQueue.php","line":47,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\Promise","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/guzzle\/src\/Handler\/CurlMultiHandler.php","line":98,"function":"run","class":"GuzzleHttp\\Promise\\TaskQueue","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/guzzle\/src\/Handler\/CurlMultiHandler.php","line":125,"function":"tick","class":"GuzzleHttp\\Handler\\CurlMultiHandler","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":246,"function":"execute","class":"GuzzleHttp\\Handler\\CurlMultiHandler","type":"->","args":["*** sensitive parameter replaced ***"]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Coroutine.php","line":65,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":246,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\Coroutine","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/aws\/aws-sdk-php\/src\/Multipart\/AbstractUploadManager.php","line":83,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/lib\/private\/Files\/ObjectStore\/S3ObjectTrait.php","line":92,"function":"upload","class":"Aws\\Multipart\\AbstractUploadManager","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/lib\/private\/Files\/ObjectStore\/ObjectStoreStorage.php","line":452,"function":"writeObject","class":"OC\\Files\\ObjectStore\\S3","type":"->","args":["urn:oid:9981",null]},{"file":"\/var\/www\/nextcloud\/lib\/private\/Files\/Storage\/Wrapper\/Wrapper.php","line":630,"function":"writeStream","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["files\/2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov",null,null]},{"file":"\/var\/www\/nextcloud\/lib\/private\/Files\/Storage\/Wrapper\/Wrapper.php","line":630,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["files\/2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov",null,null]},{"file":"\/var\/www\/nextcloud\/apps\/dav\/lib\/Connector\/Sabre\/File.php","line":191,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["files\/2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov",null]},{"file":"\/var\/www\/nextcloud\/apps\/dav\/lib\/Connector\/Sabre\/Directory.php","line":156,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[null]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php","line":316,"function":"createFile","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->","args":["2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov",null]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php","line":130,"function":"copyNode","class":"Sabre\\DAV\\Tree","type":"->","args":[{"__class__":"OCA\\DAV\\Upload\\FutureFile"},{"__class__":"OCA\\DAV\\Files\\FilesHome"},"2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov"]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php","line":161,"function":"copy","class":"Sabre\\DAV\\Tree","type":"->","args":["uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","files\/Sergey\/2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov"]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/CorePlugin.php","line":642,"function":"move","class":"Sabre\\DAV\\Tree","type":"->","args":["uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","files\/Sergey\/2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov"]},{"function":"httpMove","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"absoluteUrl":"https:\/\/disk.mediabg.ru\/remote.php\/dav\/uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/event\/lib\/EventEmitterTrait.php","line":105,"function":"call_user_func_array","args":[[{"__class__":"Sabre\\DAV\\CorePlugin"},"httpMove"],[{"absoluteUrl":"https:\/\/disk.mediabg.ru\/remote.php\/dav\/uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":479,"function":"emit","class":"Sabre\\Event\\EventEmitter","type":"->","args":["method:MOVE",[{"absoluteUrl":"https:\/\/disk.mediabg.ru\/remote.php\/dav\/uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":254,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"absoluteUrl":"https:\/\/disk.mediabg.ru\/remote.php\/dav\/uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"\/var\/www\/nextcloud\/apps\/dav\/lib\/Server.php","line":316,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/apps\/dav\/appinfo\/v2\/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/remote.php","line":163,"args":["\/var\/www\/nextcloud\/apps\/dav\/appinfo\/v2\/remote.php"],"function":"require_once"}],"File":"\/var\/www\/nextcloud\/3rdparty\/aws\/aws-sdk-php\/src\/Multipart\/AbstractUploadManager.php","Line":134,"CustomMessage":"Could not create object urn:oid:9981 for files\/2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov"},"userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/78.0.3904.108 Safari\/537.36","version":"16.0.4.1"}

2

The content of the configuration file and from log files can better be read, if you insert lines with three backticks (```) before and after a block of information :wink:

3

Sorry, updated.

1 Like