Unable to copy or move large files from external storages to S3

Nextcloud version (eg, 20.0.5): 27.1.4
Operating system and version (eg, Ubuntu 20.04): Alpine Linux 3.19
Apache or nginx version (eg, Apache 2.4.25): nginx/1.24.0
PHP version (eg, 7.4): PHP 8.2.13 (cli) (built: Nov 21 2023 19:38:47) (NTS)

The issue you are facing:

I use an S3 backend for my primary storage with Wasabi configured as described in the docs (Configuring Object Storage as Primary Storage — Nextcloud latest Administration Manual latest documentation) however I’m unable to import Google Drive data. While the process starts, as soon as it gets to any large files (say above 50mb), it seems to stop with the only error showing in the logs as follows:

	Declaration of Aws\HashingStream::seek($offset, $whence = Aws\SEEK_SET) must be compatible with Psr\Http\Message\StreamInterface::seek(int $offset, int $whence = Psr\Http\Message\SEEK_SET): void at /usr/share/webapps/nextcloud/3rdparty/aws/aws-sdk-php/src/HashingStream.php#50

Just wondering if someone could check over my config, or provide any tips?

Is this the first time you’ve seen this error? (Y/N): No, this error has happened since setting up Nextcloud a few days ago.

Steps to replicate it:

  1. Configure Object Storage as primary backend with Wasabi as the host
  2. Add Google Drive app
  3. Sign in an start a migration
  4. Process should stall, and no progress made for a number of hours, going back into the data migration menu you will be prompted to start the import again.

OR

  1. Configure Object Storage as primary backend with Wasabi as the host
  2. Setup an external share of any description
  3. Copy or move a folder containing large files in the UI
  4. The file transfer should fail with an error displayed, going into the logs the above error is seen again

The output of your Nextcloud log in Admin > Logging:
https://cloud.hrvy.uk/s/D7oZxCZwMmjNzaC

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'datadirectory' => '/var/lib/nextcloud/data',
  'logfile' => '/var/log/nextcloud/nextcloud.log',
  'logdateformat' => 'F d, Y H:i:s',
  'log_rotate_size' => 104857600,
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/usr/share/webapps/nextcloud/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/lib/nextcloud/apps',
      'url' => '/apps-appstore',
      'writable' => true,
    ),
  ),
  'updatechecker' => false,
  'check_for_working_htaccess' => false,
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => 'localhost',
    'port' => 6379,
    'dbindex' => 0,
    'timeout' => 1.5,
  ),
  'installed' => true,
  'passwordsalt' => 'REDACTED',
  'secret' => 'REDACTED',
  'trusted_domains' => 
  array (
    0 => 'localhost',
    1 => '192.168.1.171',
    2 => 'cloud.hrvy.uk',
  ),
  'dbtype' => 'mysql',
  'version' => '27.1.4.1',
  'overwrite.cli.url' => 'http://localhost',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud',
  'dbpassword' => 'REDACTED',
  'instanceid' => 'REDACTED',
  'objectstore' => 
  array (
    'class' => '\\OC\\Files\\ObjectStore\\S3',
    'arguments' => 
    array (
      'bucket' => 'mysys-nextcloud-storage',
      'hostname' => 's3.eu-west-1.wasabisys.com',
      'key' => 'REDACTED',
      'secret' => 'REDACTED',
      'port' => 443,
      'use_path_style' => true,
    ),
  ),
  'app_install_overwrite' => 
  array (
    0 => 'transfer',
  ),
);

The output of your Apache/nginx/system log in /var/log/____:

2023/11/27 21:48:27 [warn] 1520#1520: duplicate extension "js", content type: "text/javascript", previous content type: "application/javascript" in /etc/nginx/http.d/nextcloud.conf:68
2023/11/27 21:48:27 [emerg] 1520#1520: no port in upstream "php-handler" in /etc/nginx/http.d/nextcloud.conf:139
2023/11/27 21:49:14 [warn] 1541#1541: duplicate extension "js", content type: "text/javascript", previous content type: "application/javascript" in /etc/nginx/http.d/nextcloud.conf:68
2023/11/27 21:49:14 [emerg] 1541#1541: no port in upstream "php-handler" in /etc/nginx/http.d/nextcloud.conf:139
2023/11/27 21:53:02 [warn] 1580#1580: duplicate extension "js", content type: "text/javascript", previous content type: "application/javascript" in /etc/nginx/http.d/nextcloud.conf:79
2023/11/27 21:53:02 [warn] 1583#1583: duplicate extension "js", content type: "text/javascript", previous content type: "application/javascript" in /etc/nginx/http.d/nextcloud.conf:79

Disable camerarawpreviews app and it’ll go away:

1 Like

Perfect, thank you. Disabled that app and the error has now gone away and my issues are mostly gone.

However, I’ve also got this other error now, after NC completes a batch of data transfers, it will cancel the migration with the following error in the logs on the next batch:

Google Drive import error: Unknown job failure. InvalidArgumentException: SSL CA bundle not found: /tmp/oc_tmp_dDooFe-.crt in /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php:363 Stack trace: #0 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php(59): GuzzleHttp\Handler\CurlFactory->applyHandlerOptions() #1 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlHandler.php(43): GuzzleHttp\Handler\CurlFactory->create() #2 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php(137): GuzzleHttp\Handler\CurlHandler->__invoke() #3 /usr/share/webapps/nextcloud/lib/private/Http/Client/DnsPinMiddleware.php(154): GuzzleHttp\Middleware::GuzzleHttp{closure}() #4 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php(35): OC\Http\Client\DnsPinMiddleware->OC\Http\Client{closure}() #5 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php(31): GuzzleHttp\PrepareBodyMiddleware->__invoke() #6 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/RedirectMiddleware.php(71): GuzzleHttp\Middleware::GuzzleHttp{closure}() #7 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php(63): GuzzleHttp\RedirectMiddleware->__invoke() #8 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/HandlerStack.php(75): GuzzleHttp\Middleware::GuzzleHttp{closure}() #9 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php(331): GuzzleHttp\HandlerStack->__invoke() #10 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php(168): GuzzleHttp\Client->transfer() #11 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php(187): GuzzleHttp\Client->requestAsync() #12 /usr/share/webapps/nextcloud/lib/private/Http/Client/Client.php(230): GuzzleHttp\Client->request() #13 /var/lib/nextcloud/apps/integration_google/lib/Service/GoogleAPIService.php(120): OC\Http\Client\Client->get() #14 /var/lib/nextcloud/apps/integration_google/lib/Service/GoogleDriveAPIService.php(253): OCA\Google\Service\GoogleAPIService->request() #15 /var/lib/nextcloud/apps/integration_google/lib/Service/GoogleDriveAPIService.php(179): OCA\Google\Service\GoogleDriveAPIService->importFiles() #16 /var/lib/nextcloud/apps/integration_google/lib/BackgroundJob/ImportDriveJob.php(36): OCA\Google\Service\GoogleDriveAPIService->importDriveJob() #17 /usr/share/webapps/nextcloud/lib/public/BackgroundJob/Job.php(81): OCA\Google\BackgroundJob\ImportDriveJob->run() #18 /usr/share/webapps/nextcloud/lib/public/BackgroundJob/QueuedJob.php(57): OCP\BackgroundJob\Job->start() #19 /usr/share/webapps/nextcloud/lib/public/BackgroundJob/QueuedJob.php(47): OCP\BackgroundJob\QueuedJob->start() #20 /usr/share/webapps/nextcloud/cron.php(152): OCP\BackgroundJob\QueuedJob->execute() #21 {main}

Do you know what might be causing this?

Can you place preformatted text wrap rather than blockquotes on that. It’s impossible to read the stack trace without it. :slight_smile:

No blockquotes for code

But preferformatted works well for it - paste code here!

Sure, sorry had to do some additional formatting as it still wasn’t showing very well.

Google Drive import error: Unknown job failure. InvalidArgumentException: SSL CA bundle not found: /tmp/oc_tmp_dDooFe-.crt in /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php:363
Stack trace:
#0 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php(59): GuzzleHttp\\Handler\\CurlFactory->applyHandlerOptions()
#1 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlHandler.php(43): GuzzleHttp\\Handler\\CurlFactory->create()
#2 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php(137): GuzzleHttp\\Handler\\CurlHandler->__invoke()
#3 /usr/share/webapps/nextcloud/lib/private/Http/Client/DnsPinMiddleware.php(154): GuzzleHttp\\Middleware::GuzzleHttp\\{closure}()
#4 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php(35): OC\\Http\\Client\\DnsPinMiddleware->OC\\Http\\Client\\{closure}()
#5 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php(31): GuzzleHttp\\PrepareBodyMiddleware->__invoke()
#6 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/RedirectMiddleware.php(71): GuzzleHttp\\Middleware::GuzzleHttp\\{closure}()
#7 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php(63): GuzzleHttp\\RedirectMiddleware->__invoke()
#8 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/HandlerStack.php(75): GuzzleHttp\\Middleware::GuzzleHttp\\{closure}()
#9 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php(331): GuzzleHttp\\HandlerStack->__invoke()
#10 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php(168): GuzzleHttp\\Client->transfer()\n#11 /usr/share/webapps/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php(187): GuzzleHttp\\Client->requestAsync()
#12 /usr/share/webapps/nextcloud/lib/private/Http/Client/Client.php(230): GuzzleHttp\\Client->request()
#13 /var/lib/nextcloud/apps/integration_google/lib/Service/GoogleAPIService.php(120): OC\\Http\\Client\\Client->get()
#14 /var/lib/nextcloud/apps/integration_google/lib/Service/GoogleDriveAPIService.php(253): OCA\\Google\\Service\\GoogleAPIService->request()
#15 /var/lib/nextcloud/apps/integration_google/lib/Service/GoogleDriveAPIService.php(179): OCA\\Google\\Service\\GoogleDriveAPIService->importFiles()
#16 /var/lib/nextcloud/apps/integration_google/lib/BackgroundJob/ImportDriveJob.php(36): OCA\\Google\\Service\\GoogleDriveAPIService->importDriveJob()
#17 /usr/share/webapps/nextcloud/lib/public/BackgroundJob/Job.php(81): OCA\\Google\\BackgroundJob\\ImportDriveJob->run()
#18 /usr/share/webapps/nextcloud/lib/public/BackgroundJob/QueuedJob.php(57): OCP\\BackgroundJob\\Job->start()
#19 /usr/share/webapps/nextcloud/lib/public/BackgroundJob/QueuedJob.php(47): OCP\\BackgroundJob\\QueuedJob->start()
#20 /usr/share/webapps/nextcloud/cron.php(152): OCP\\BackgroundJob\\QueuedJob->execute()
#21 {main}

This topic was automatically closed 8 days after the last reply. New replies are no longer allowed.