Client unable to upload files bigger than 2GB

Nextcloud version (eg, 12.0.2): 13.0.4
Operating system and version (eg, Ubuntu 17.04): ArchLinuxARM-odroid-xu3-latest
Apache or nginx version (eg, Apache 2.4.25): Apache/2.4.33
PHP version (eg, 7.1): 7.2.7

The issue you are facing:
I’m unable to synchronize or upload files bigger than 2Gb with nextcloud client nor with the web upload.

In the logs I see this:
Error while copying file to target location (copied bytes: 2147483647, expected filesize: -1 )

I’m currently using a Odroid HC1 (32 bits system), and I know there are some limitations regarding php and big files… but as far as I understood from the docs
or posts like this one, this limitation was bypassed and it should not affect the client sync, as it is uploading the file by chunks.

I’ve tried to trace the code and the problem seems coming from streamCopy call, fwrite would write exactly 2147483647 bytes (PHP_INT_MAX :smiley: ) so the loop stops and it returns with a false state.

I’ve spent several days trying to figure out what is the problem, but I can’t manage to make it work…

Am I missing something? is this how it is supposed to work for 32 bits systems?

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

  1. Upload a file bigger than 2Gb

The output of your Nextcloud log in Admin > Logging:

{"reqId":"g9dWaeloXO6eRyMatPMw","level":4,"time":"2018-07-01T18:04:35+00:00","remoteAddr":"192.168.1.74","user":"andreu","app":"webdav","method":"MOVE","url":"/remote.php/dav/uploads/andreu/2039344413/.file","message":"Exception: {"Exception":"SabreDAVException","Message":"Error while copying file to target location (copied bytes: 2147483647, expected filesize: -1 )","Code":0,"Trace":"#0 /usr/share/webapps/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php(176): OCADAVConnectorSabreFile->put(Resource id #25)
#1 /usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php(318): OCADAVConnectorSabreDirectory->createFile('disk1.iso', Resource id #25)
#2 /usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php(130): SabreDAVTree->copyNode(Object(OCADAVUploadFutureFile), Object(OCADAVConnectorSabreDirectory), 'disk1.iso')
#3 /usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php(164): SabreDAVTree->copy('uploads/andreu/...', 'files/andreu/Im...')
#4 /usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php(646): SabreDAVTree->move('uploads/andreu/...', 'files/andreu/Im...')
#5 [internal function]: SabreDAVCorePlugin->httpMove(Object(SabreHTTPRequest), Object(SabreHTTPResponse))
#6 /usr/share/webapps/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php(105): call_user_func_array(Array, Array)
#7 /usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php(479): SabreEventEventEmitter->emit('method:MOVE', Array)
#8 /usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php(254): SabreDAVServer->invokeMethod(Object(SabreHTTPRequest), Object(SabreHTTPResponse))
#9 /usr/share/webapps/nextcloud/apps/dav/lib/Server.php(287): SabreDAVServer->exec()
#10 /usr/share/webapps/nextcloud/apps/dav/appinfo/v2/remote.php(35): OCADAVServer->exec()
#11 /usr/share/webapps/nextcloud/remote.php(164): require_once('/usr/share/weba...')
#12 {main}","File":"/usr/share/webapps/nextcloud/apps/dav/lib/Connector/Sabre/File.php","Line":176}","userAgent":"Mozilla/5.0 (Linux) mirall/2.3.3 (Nextcloud)","version":"13.0.4.0"}

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'debug' => true,
  'instanceid' => 'XXXXXXXX',
  'passwordsalt' => 'XXXXXXXX',
  'secret' => 'XXXXXXXXXXX',
  'trusted_domains' => 
  array (
    0 => 'nextcloud',
  ),
  'datadirectory' => '/media/drive/nextcloud/data',
  'overwrite.cli.url' => 'https://nextcloud',
  'dbtype' => 'mysql',
  'version' => '13.0.4.0',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'nextcloud',
  'dbpassword' => 'XXXXXXXXX',
  'installed' => true,
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'maintenance' => false,
  'forcessl' => true,
  'forceSSLforSubdomains' => true,
  'loglevel' => 0,
);

btw, the nextcloud data is stored in a ext4 hard drive

Thanks in advance!
Andreu

32bit or 64bit?

it is 32bit

In past versions, it was completely impossible to handle files with 32-bit php. In NC 13 they fixed a few things around that but I still have seen issues (like the upload limits). You can check the bug tracker (github.com/nextcloud/server/issues) if there are/were requests for that, perhaps ask the developer if this could be possible or if this limits are due to the 32-bit version and workarounds are not possible.

I see. I will try to dig more into this issue and ask the developers.
Hopefully I will manage to bypass the limitation :slight_smile:

Cheers!
Andreu

To overcome file size limitations beyond 2 GB, use a 64 Bit OS and a 64 Bit PHP version. It’s sad, because sometimes you’re not choosing your environment, but given a correct configuration (see current manual) huge file sizes are possible with a complete 64 bit setup.