S3 random storage problem on large files

ℹ️ Support
, ,
4

I had the same problem, but with my Nextcloud version (23) I had to change the uploadPartSize in a different file.

I modified:
/var/www/nextcloud/lib/private/Files/ObjectStore/S3ConnectionTrait.php

And changed the numerical value in this line:
$this->uploadPartSize = $params['uploadPartSize'] ?? 104857600;

Here I set the value to 104857600 which is 100MB. (250MB did not solve it for me)

I got the inspiration to set it to 100MB thanks to this comment: S3 default upload part_size set to 500MB · Issue #24390 · nextcloud/server · GitHub

In line with the last link, I also modified the file:
/var/www/nextcloud/3rdparty/aws/aws-sdk-php/src/S3/MultipartUploader.php

With the following settings:

    const PART_MIN_SIZE = 4294967296;
    const PART_MAX_SIZE = 1048576000000;
    const PART_MAX_NUM = 10000;

As you can see/calculate, PART_MIN_SIZE is set it to 4GB and PART_MAX_SIZE is set to roughly 1TB (0.95TB) which was determined by 100MB*10000.

2 Likes