S3 random storage problem on large files

Hi,
i am using the lastest docker image for nextcloud together with S3. I am running into random 400 BadRequest Errors when the files are large… If we are talking about files between 3GB and 5GB, i do have a success rate of maybe 20%. So i have to rerun the upload a few times but finaly the upload is successfull.

Anyways… in 80% the Log shows me an “BadRequest” 400 Timeout. I have researched due the whole internet and dont find any solution. Maybe this has to do something with a curl timeout or a S3 parameter. I also dont find anything in the code. Please help me with this… Thx

Environment:
Latest (03.03.20) Docker Nextcloud Image 18.0.1 with Apache

Setup:
<?php
$CONFIG = array (
‘htaccess.RewriteBase’ => ‘/’,
‘memcache.local’ => ‘\OC\Memcache\APCu’,
‘apps_paths’ =>
array (
0 =>
array (
‘path’ => ‘/var/www/html/apps’,
‘url’ => ‘/apps’,
‘writable’ => false,
),
1 =>
array (
‘path’ => ‘/var/www/html/custom_apps’,
‘url’ => ‘/custom_apps’,
‘writable’ => true,
),
),
‘instanceid’ => ‘REMOVED’,
‘passwordsalt’ => ‘REMOVED’,
‘secret’ => ‘REMOVED+O0zjhS8eG0pFbO2’,
‘trusted_domains’ =>
array (
0 => ‘REMOVED’
),
‘datadirectory’ => ‘/var/www/html/data’,
‘tempdirectory’ => ‘/tmp’,
‘dbtype’ => ‘mysql’,
‘version’ => ‘18.0.1.3’,
‘overwrite.cli.url’ => ‘REMOVED’,
‘dbname’ => ‘nextcloud’,
‘dbhost’ => ‘REMOVED’,
‘dbport’ => ‘’,
‘dbtableprefix’ => ‘oc_’,
‘mysql.utf8mb4’ => true,
‘dbuser’ => ‘REMOVED’,
‘dbpassword’ => ‘REMOVED’,
‘installed’ => true,
‘objectstore’ =>
array (
‘class’ => ‘\OC\Files\ObjectStore\S3’,
‘arguments’ =>
array (
‘bucket’ => ‘s3.ceetrox.de’,
‘autocreate’ => true,
‘key’ => ‘REMOVED’,
‘secret’ => ‘REMOVED’,
‘hostname’ => ‘s3.eu-central-1.amazonaws.com’,
‘port’ => 443,
‘use_ssl’ => true,
‘region’ => ‘eu-central-1’,
),
),
‘app_install_overwrite’ =>
array (
0 => ‘uploaddetails’,
),
‘ldapIgnoreNamingRules’ => false,
‘ldapProviderFactory’ => ‘OCA\User_LDAP\LDAPProviderFactory’,
);

Log
Aws\S3\Exception\S3MultipartUploadException: An exception occurred while uploading parts to a multipart upload. The following parts had errors: - Part 4: Error executing “UploadPart” on “https://s3.eu-central-1.amazonaws.com/s3.ceetrox.de/urn%3Aoid%3A112540?partNumber=4&uploadId=15q9YdrBOkelJcaifcPoS_k.3FTKQmfMSwUImQ.o6zo09h2jIeFxJRAhS_.SUo5hlRGJEkIN_j_81zS2wQKEEd5v8AcMhsVjpd6gVeI1Fdhk.gansp6VHgpYDUxdYCf1”; AWS HTTP error: Client error: PUT https://s3.eu-central-1.amazonaws.com/s3.ceetrox.de/urn%3Aoid%3A112540?partNumber=4&uploadId=15q9YdrBOkelJcaifcPoS_k.3FTKQmfMSwUImQ.o6zo09h2jIeFxJRAhS_.SUo5hlRGJEkIN_j_81zS2wQKEEd5v8AcMhsVjpd6gVeI1Fdhk.gansp6VHgpYDUxdYCf1 resulted in a 400 Bad Request response: <?xml version="1.0" encoding="UTF-8"?> RequestTimeoutYour socket connection to the server w (truncated…) RequestTimeout (client): Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed. - <?xml version="1.0" encoding="UTF-8"?> RequestTimeoutYour socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.A4E6E02FD8951BB6smDJwoIp/06ckg1ojYVuv5f5Rli+kuTzcBe7+kntLBGR506eaRhxGRijIIuJ7kT5d2zp3kk0ZIk= - Part 5: Error executing “UploadPart” on “https://s3.eu-central-1.amazonaws.com/s3.ceetrox.de/urn%3Aoid%3A112540?partNumber=5&uploadId=15q9YdrBOkelJcaifcPoS_k.3FTKQmfMSwUImQ.o6zo09h2jIeFxJRAhS_.SUo5hlRGJEkIN_j_81zS2wQKEEd5v8AcMhsVjpd6gVeI1Fdhk.gansp6VHgpYDUxdYCf1”; AWS HTTP error: Client error: PUT https://s3.eu-central-1.amazonaws.com/s3.ceetrox.de/urn%3Aoid%3A112540?partNumber=5&uploadId=15q9YdrBOkelJcaifcPoS_k.3FTKQmfMSwUImQ.o6zo09h2jIeFxJRAhS_.SUo5hlRGJEkIN_j_81zS2wQKEEd5v8AcMhsVjpd6gVeI1Fdhk.gansp6VHgpYDUxdYCf1 resulted in a 400 Bad Request response: <?xml version="1.0" encoding="UTF-8"?> RequestTimeoutYour socket connection to the server w (truncated…) RequestTimeout (client): Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed. - <?xml version="1.0" encoding="UTF-8"?> RequestTimeoutYour socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.62A6CA44A98FC9F2O0nKqsjdOtb5oiYPzO7OaNk5eFoLH1D89ZNhEifFfVdmSDk5/E52AFV7b/SSaBMWdLuAPfVYxqo=

Got the same error… fixed by modifying /lib/private/Files/ObjectStore/S3ObjectTrait.php and lowering S3_UPLOAD_PART_SIZE from 500MB to 250MB.

Give it a try…