Problems with large file uploads: "Could either be a network problem on the sending side or a problem writing to the storage on the server side."

Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version (eg, 18.0.2): 20.0.3
Operating system and version (eg, Ubuntu 20.04): Ubuntu 20.04 LTS
Linux 5.4.0-60-generic x86_64
Apache or nginx version (eg, Apache 2.4.25): Apache/2.4.41
PHP version (eg, 7.1): 7.4.3

The issue you are facing:

Hello, I have recently installed Nextcloud VM from Hansson IT.
It seems to work well with downloads.
But there are issues with uploads.
While uploading large files (1 GB and more) sometimes it fails.

I have set loglevel 0, but the only usefull message from nextcloud.log seems to be this:

“Expected filesize of 7176596992 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 5592825856 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.”

I have already tried changing the following parameters in php.ini:

upload_max_filesize = 50G
post_max_size = 50G
memory_limit = 2048M

Also I disabled chunking in Nextcloud.
It did not help.

Is there anything else I should try?

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

  1. upload a file via web-interface
  2. after some time receive error message

The output of your Nextcloud log in Admin > Logging:

{"reqId":"jK21zbxDeeKne5Awu6dE","level":3,"time":"2021-01-13T12:33:37+03:00","remoteAddr":"1.2.3.4","user":"user1","app":"no app in context","method":"PUT","url":"/remote.php/webdav/Buh30_backup_2021_01_13_010602_6878268.bak","message":{"Exception":"Sabre\\DAV\\Exception\\BadRequest","Message":"Expected filesize of 7195471360 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 6110658560 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":155,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1104,"function":"createFile","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":527,"function":"createFile","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":474,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":251,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":319,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/appinfo/v1/webdav.php","line":84,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":167,"args":["/var/www/nextcloud/apps/dav/appinfo/v1/webdav.php"],"function":"require_once"}],"File":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","Line":246,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36","version":"20.0.3.2"}

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'passwordsalt' => '',
  'secret' => '',
  'trusted_domains' => 
  array (
    0 => 'localhost',
    1 => '5.6.7.8',
    2 => '4.3.2.1',
  ),
  'datadirectory' => '/mnt/ncdata',
  'skeletondirectory' => '',
  'dbtype' => 'pgsql',
  'version' => '20.0.3.2',
  'overwrite.cli.url' => 'http://localhost/',
  'dbname' => 'nextcloud_db',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'ncadmin',
  'dbpassword' => '',
  'installed' => true,
  'instanceid' => '',
  'upgrade.disable-web' => 'true',
  'log_type' => 'file',
  'logfile' => '/var/log/nextcloud/nextcloud.log',
  'loglevel' => '0',
  'log.condition' => 
  array (
    'apps' => 
    array (
      0 => 'admin_audit',
    ),
  ),
  'mail_smtpmode' => 'smtp',
  'remember_login_cookie_lifetime' => '1800',
  'log_rotate_size' => '504857600',
  'trashbin_retention_obligation' => 'auto, 180',
  'versions_retention_obligation' => 'auto, 365',
  'simpleSignUpLink.shown' => false,
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'filelocking.enabled' => true,
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => '/var/run/redis/redis-server.sock',
    'port' => 0,
    'timeout' => 0.5,
    'dbindex' => 0,
    'password' => '0rvLHtxQKxEtlEdg1MrEDrDZE4N4',
  ),
  'logtimezone' => 'Europe/Moscow',
  'htaccess.RewriteBase' => '/',
  'ldapIgnoreNamingRules' => false,
  'ldapProviderFactory' => 'OCA\\User_LDAP\\LDAPProviderFactory',
  'maintenance' => false,
);

The output of your Apache/nginx/system log in /var/log/____:

[Wed Jan 13 12:13:00.338716 2021] [mpm_event:notice] [pid 1036:tid 139622946147392] AH00489: Apache/2.4.41 (Ubuntu) OpenSSL/1.1.1f configured -- resuming normal operations
[Wed Jan 13 12:13:00.340680 2021] [core:notice] [pid 1036:tid 139622946147392] AH00094: Command line: '/usr/sbin/apache2'

Has no one had a similar problem?

I am having this same issue since the last 5 days. I am a user in one of the server providers and anything beyond 8MB/28MB/58MB is giving this error.
Expected filesize of 10485760 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 7774208 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.

The wrote size keeps changing but the expected number remains the same.

Same here…
Seems like Nextcloud is just full of bugs!

Generally this error is not an indication of a bug, but an environment matter. If you want to troubleshoot your own unique situation, I suggest creating a dedicated post so that we can try to help you out.

Honestly, I’m just tired of fighting against all sorts of errors.

Dedicated post? look here:

Nextcloud errors with external SMB - TrueNAS General - TrueNAS Community Forums

How to make TrueNAS home directory as Nextcloud users’ data directory - :information_source: Support - Nextcloud community

Auto Delete in Server and Phone and vice versa - :information_source: Support - Nextcloud community

not my posts but still actual problems:

Media folders may not contain an ampersand (&) in the folder name · Issue #2545 · nextcloud/photos (github.com)

Prompt to move to app folder on upload even though ‘Keep in source folder’ is selected · Issue #4465 · nextcloud/android (github.com)

Now I don’t have questions why cloud services are expensive. They just work.

Maybe this solution.

Well, they have employees that make them work, and that’s what you’re paying for, along with infrastructure costs.

The solution gave this error:

Maybe the hint does not work. For Debian Bookworm there is no php-smbclient and also there is php8.2-smbclient only for Debian Trixie and Debian Sid.

Can you search in /usr/lib/php/<version/ a file smbclient.so and if exists check:

dpkg -S /usr/lib/php/<version/smbclient.so

If you cannot solve the SMB problem. Can you perhaps mount the file system and then use the Local option?

there is no /usr folder in docker container
there is no /usr/lib/php/ folder in truenas host

this is an option, but just very inconvenient option. As in this case, I’d have to give www-data user read/write permissions to whole /home directory of users. And also create that local folder for each user.

by the way I can mount the local folder only when I set additional storage in the nextcloud installation configuration. I was unable to manually mount host directory in nextcloud docker container as it does not see host folders.

Then unfortunately I don’t know. If you really want to keep SMB, you will probably have to follow the tip to reinstall / first install??? “php-smbclient”. You will probably only be able to do this manually from another source. However, I don’t know whether this works under TrueNAS.

Interesting if there are some other people who also would like to have their home directory in the cloud?

Someone said that local mounts are faster and overall better than SMB mount.

But now I’m thinking if mounting a large directory in the cloud is a bad practice in the first place?

The integration of SMB/CIFS or Local is always a crutch in the end. Perhaps you can simply provide your users with more normal Nextcloud storage. Maybe pure Nextcloud is the future or your users use regular old CIFS/SMB in parallel to Nextcloud.

1 Like