cURL error 28: Operation timed out after 30000 milliseconds with 0 bytes received - Federated Nextcloud

Nextcloud version: 21.0.1.1
Operating system and version: Ubuntu 20.04
Apache or nginx version: Apache 2.4.41
PHP version: 7.4

The issue you are facing:
Sharing files over a federated folder. Bigger files (>500MB) result in an error (cURL error 28: Operation timed out after 30000 milliseconds with 0 bytes received).
I try to share files between my two NC. NC1 is a VPS, NC2 is running on a QNAP NAS (Virtual Machine Ubuntu 20.04). Smaller files are no problem at all.

Is this the first time you’ve seen this error? (Y/N):
Yes

Steps to replicate it:

  1. Federate Folder
  2. Share big files >500MB

The output of your Nextcloud log in Admin > Logging:

{"reqId":"iQZMXS0s4pW09UosJDaW","level":3,"time":"2021-04-09T11:10:45+00:00","remoteAddr":"X","user":"X.Y","app":"no app in context","method":"MOVE","url":"/remote.php/dav/uploads/X.Y/web-file-upload-56368bb80eec6bcd1cefb876660b8d34-1617966380606/.file","message":{"Exception":"GuzzleHttp\\Exception\\ConnectException","Message":"cURL error 28: Operation timed out after 30000 milliseconds with 0 bytes received (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://X.Y.net/public.php/webdav/XY.rar",
"Code":0,"Trace":[{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php","line":158,"function":"createRejection","class":"GuzzleHttp\\Handler\\CurlFactory","type":"::"},
{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php","line":110,"function":"finishError","class":"GuzzleHttp\\Handler\\CurlFactory","type":"::"},
{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlHandler.php","line":47,"function":"finish","class":"GuzzleHttp\\Handler\\CurlFactory","type":"::"},
{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/Proxy.php","line":28,"function":"__invoke","class":"GuzzleHttp\\Handler\\CurlHandler","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/Proxy.php","line":48,"function":"GuzzleHttp\\Handler\\{closure}","class":"GuzzleHttp\\Handler\\Proxy","type":"::","args":["*** sensitive parameters replaced ***"]},
{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php","line":64,"function":"GuzzleHttp\\Handler\\{closure}","class":"GuzzleHttp\\Handler\\Proxy","type":"::","args":["*** sensitive parameters replaced ***"]},
{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php","line":31,"function":"__invoke","class":"GuzzleHttp\\PrepareBodyMiddleware","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/RedirectMiddleware.php","line":71,"function":"GuzzleHttp\\{closure}","class":"GuzzleHttp\\Middleware","type":"::","args":["*** sensitive parameters replaced ***"]},
{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php","line":63,"function":"__invoke","class":"GuzzleHttp\\RedirectMiddleware","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/HandlerStack.php","line":75,"function":"GuzzleHttp\\{closure}","class":"GuzzleHttp\\Middleware","type":"::","args":["*** sensitive parameters replaced ***"]},
{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php","line":331,"function":"__invoke","class":"GuzzleHttp\\HandlerStack","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php","line":168,"function":"transfer","class":"GuzzleHttp\\Client","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php","line":187,"function":"requestAsync","class":"GuzzleHttp\\Client","type":"->"},
{"file":"/var/www/nextcloud/lib/private/Http/Client/Client.php","line":342,"function":"request","class":"GuzzleHttp\\Client","type":"->"},
{"file":"/var/www/nextcloud/lib/private/Files/Storage/DAV.php","line":516,"function":"put","class":"OC\\Http\\Client\\Client","type":"->"},
{"file":"/var/www/nextcloud/lib/private/Files/Storage/DAV.php","line":423,"function":"uploadFile","class":"OC\\Files\\Storage\\DAV","type":"->"},
{"file":"/var/www/nextcloud/lib/private/Files/Storage/DAV.php","line":414,"function":"writeBack","class":"OC\\Files\\Storage\\DAV","type":"->"},{"function":"OC\\Files\\Storage\\{closure}","class":"OC\\Files\\Storage\\DAV","type":"->","args":["*** sensitive parameters replaced ***"]},
{"file":"/var/www/nextcloud/apps/files_external/3rdparty/icewind/streams/src/CallbackWrapper.php","line":119,"function":"call_user_func"},
{"function":"stream_close","class":"Icewind\\Streams\\CallbackWrapper","type":"->"},
{"file":"/var/www/nextcloud/lib/private/Files/Storage/Common.php","line":871,"function":"fclose"},
{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":629,"function":"writeStream","class":"OC\\Files\\Storage\\Common","type":"->"},
{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":629,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},
{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":629,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},
{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","line":207,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},
{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":156,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":305,"function":"createFile","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":133,"function":"copyNode","class":"Sabre\\DAV\\Tree","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":163,"function":"copy","class":"Sabre\\DAV\\Tree","type":"->"},
{"file":"/var/www/nextcloud/apps/dav/lib/Upload/ChunkingPlugin.php","line":95,"function":"move","class":"Sabre\\DAV\\Tree","type":"->"},
{"file":"/var/www/nextcloud/apps/dav/lib/Upload/ChunkingPlugin.php","line":77,"function":"performMove","class":"OCA\\DAV\\Upload\\ChunkingPlugin","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"beforeMove","class":"OCA\\DAV\\Upload\\ChunkingPlugin","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":603,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpMove","class":"Sabre\\DAV\\CorePlugin","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},
{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},
{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":332,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},
{"file":"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},
{"file":"/var/www/nextcloud/remote.php","line":167,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php","Line":210,"CustomMessage":"--"},
"userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36 OPR/75.0.3969.149","version":"21.0.1.1","id":"6070388ab2533"}```

The output of your config.php file in /var/www/nextcloud/config/

<?php
$CONFIG = array (
  'instanceid' => 'oc7c1ii1uo4f',
  'passwordsalt' => 'XY',
  'secret' => 'XY',
  'trusted_domains' =>
  array (
    0 => 'www.X-Y.de',
    1 => 'X-Y.de',
    2 => 'X.Y.de',
  ),
  'datadirectory' => '/var/www/nextcloud/data',
  'dbtype' => 'mysql',
  'version' => '21.0.1.1',
  'overwrite.cli.url' => 'https://www.X-Y.de',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'oc_X.Y',
  'dbpassword' => 'XY',
  'installed' => true,
  'maintenance' => false,
  'loglevel' => 0,
  'theme' => '',
  'allow_local_remote_servers' => true,
  'default_phone_region' => 'DE',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'updater.release.channel' => 'stable',
  'app_install_overwrite' =>
  array (
    0 => 'files_external_ipfs',
    1 => 'calendar',
    2 => 'cookbook',
    3 => 'dicomviewer',
    4 => 'files_frommail',
  ),
  'mail_from_address' => 'nextcloud',
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'mail_domain' => 'X-Y.de',
  'mail_smtphost' => 'X.Y.net',
  'mail_smtpport' => '25',
  'mail_smtpauth' => 1,
  'mail_smtpname' => 'BLA@X-Y.de',
  'mail_smtppassword' => 'xyz',
  'mail_smtpsecure' => 'tls',
  'mail_smtpauthtype' => 'LOGIN',
  'onlyoffice' =>
  array (
    'verify_peer_off' => true,
  ),
  'updater.secret' => 'xyZ',
);

The output of your Apache/nginx/system log in /var/log/nextcloud.error:

[Fri Apr 09 13:10:45.830872 2021] [proxy_fcgi:error] [pid 841] (70008)Partial results are valid but processing is incomplete: [client X.X.X.X:X] AH01075: Error dispatching reques>

I had the same issue. There is a performance issue at nextcloud with big federation.
I overcome the issue by increasing the timeouts, but this has to be adjusted manually in the code.

May you try this.

  1. Increase the timeout to get the folder structure at storage.php in apps/files_sharing/lib/External

Line 329: ‘timeout’ => 60,

  1. Increase the timeout to get big file transfer working at client.php lib/private/Http/Client

Line 80: RequestOptions::TIMEOUT => 120

1 Like

This works. Thanks.

But it’s absolutely no fix for me. It will be reset after next app update.

Nextcloud, please fix your bugs instead of implementing new features…

I’m curious to see what will happen.

cURL time out by jlssmt · Pull Request #35461 · nextcloud/server · GitHub

Your internet connection / the server’s uplink is slow. How should we fix that? :wink:

1 Like

@kesselb Do you really think that this error is due to the user and not the application?

1 Like

Not the user, but the infrastructure.

For federation, we assume that the peers can exchange information within a reasonable timespan.

The request to list a remote directory or file should not need 30 seconds to complete. Such a slow response makes the whole feature unusable after all.

@kesselb
I have download: 50Mbit/s and upload 10Mbit/s. The same for my second (backup) server at another location. I know Germany is slow, but I think this speed should be fine. Otherwise, nextcloud should start adding gigabit connections to the minimum requirements :wink:
Like I said: IMO, this is a bug. I don’t know if this timeout is the right solution, but it is at least a solution. The error has been unresolved since mid-April 2021.

By the way: This error occurs only with big files. In my example in combination with backup app (huge backup files).

1 Like

FYI, set davstorage.request_timeout to something lerger and you’re set:

1 Like