Big file sync with external storage (Amazon S3) fails

Nextcloud version: 11.0.2
Operating system and version: Debian GNU/Linux 8
Apache or nginx version: nginx/1.6.2
PHP version: 5.6.29
Is this the first time you’ve seen this error and can you replicate it?: yes

The issue you are facing:
I am not able to sync big files from an external storage (Amazon S3) Share. There are serveral Files with file size from 1.9GB to 5 GB.

I am able to sync the Files with 1.9 GB, but not these with 5 GB.

I’ve also tried to save these 5 GB files directly on the nextcloud Server storage (not external storage). This seems to work, I am able to sync these files.

I’ve found and used this guide: https://github.com/owncloud/documentation/wiki/Uploading-files-up-to-16GB

Anyone have an Idea why this does not work via external Amazon Storage?

Output of Nextcloud log in Admin > Logging:
no errors found

Output of config.php file:

<?php $CONFIG = array ( 'instanceid' => 'REMOVED', 'passwordsalt' => 'REMOVED', 'secret' => 'REMOVED', 'trusted_domains' => array ( 0 => 'https://subdomain.domain.ch', ), 'datadirectory' => '/var/www/owncloud/data', 'overwrite.cli.url' => 'https://subdomain.domain.ch', 'dbtype' => 'mysql', 'version' => '11.0.2.7', 'dbname' => 'REMOVED', 'dbhost' => 'REMOVED', 'dbtableprefix' => 'REMOVED', 'dbuser' => 'REMOVED', 'dbpassword' => 'REMOVED', 'logtimezone' => 'Europe/Zurich', 'installed' => true, 'updatechecker' => 'false', 'ldapIgnoreNamingRules' => false, 'mail_smtpmode' => 'REMOVED', 'mail_from_address' => 'REMOVED', 'mail_domain' => 'REMOVED', 'mail_smtpauthtype' => 'REMOVED', 'mail_smtpauth' => 1, 'mail_smtphost' => 'REMOVED', 'mail_smtpport' => 'REMOVED', 'mail_smtpname' => 'REMOVED', 'mail_smtppassword' => 'REMOVED', 'maintenance' => false, 'default_language' => 'en', 'trashbin_retention_obligation' => 'auto, 30', 'versions_retention_obligation' => 'auto, 30', 'theme' => '', 'loglevel' => 2, 'singleuser' => false, 'htaccess.RewriteBase' => '/', 'log_rotate_size' => '104857600', 'ldapProviderFactory' => '\\OCA\\User_LDAP\\LDAPProviderFactory', 'memcache.distributed' => '\\OC\\Memcache\\Redis', 'memcache.locking' => '\\OC\\Memcache\\Redis', 'memcache.local' => '\OC\Memcache\APCu', 'redis' => array ( 'host' => 'redis', 'port' => 6379, ), ); Output of nginx log: 2017/04/20 09:56:46 [warn] 102#102: *959565 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/6/70/0000017706 while reading upstream, client: 10.64.45.1, server: subdomain.domain.ch, request: "GET /index.php/s/ZcI11dxliqhikda/download?path=%2F&files=ShowKeyPlus.exe HTTP/1.0", upstream: "http://10.10.64.44:32770/index.php/s/ZcI11dxliqhikda/download?path=%2F&files=ShowKeyPlus.exe", host: "subdomain.domain.ch" Output of nginx reverse proxy: 2017/04/20 09:56:56 [error] 3133#0: *27579527 upstream timed out (110: Connection timed out) while reading response header from upstream, client: xxx.xxx.xxx.xxx, server: *.domain.ch, request: "GET /remote.php/webdav/hiagshare/Test/de_windows_10_multiple_editions_version_1607_updated_jul_2016_x64_dvd_9053906%20-%20Kopie%20(2).iso HTTP/1.1", upstream: "http://10.10.64.43:80/remote.php/webdav/hiagshare/Test/de_windows_10_multiple_editions_version_1607_updated_jul_2016_x64_dvd_9053906%20-%20Kopie%20(2).iso", host: "subdomain.domain.ch" Output of Nextcloud Client log: 04-20 09:56:44:374 2202 unknown: QString::arg: Argument missing: "https://docs.nextcloud.com/desktop/2.2/" , 2 04-20 09:56:46:278 2202 unknown: QString::arg: Argument missing: "https://docs.nextcloud.com/desktop/2.2/" , 2 04-20 09:56:46:279 2202 unknown: QString::arg: Argument missing: "https://docs.nextcloud.com/desktop/2.2/" , 2 04-20 09:56:53:865 2202 unknown: QString::arg: Argument missing: "https://docs.nextcloud.com/desktop/2.2/" , 2

I’m having the same problem with large files and S3 external storage. The client seems to upload the file from the client to the nextcloud server… but it feels like when it starts to upload to s3 it blows up.

To debug, you need to figure out where the timeout is happening and why. Does it work on the NC server directly without the reverse proxy?

Problem has been fixed i guess. Does not appear again with version 13.

Thanks a lot Nextclouders! :slight_smile: