NC20.0.6, Preview Generator 3.1.1, S3 as primary storage -> cronjob failures

Nextcloud version (eg, 20.0.5): 20.0.6
Operating system and version (eg, Ubuntu 20.04): Uberspace 7

The issue you are facing: Preview generator cronjobs send error mail after a few minutes with a failure to create an already existing S3 bucket. This hasn’t happened before updating Preview Generator to 3.1.1, but happens at every cron run since the update to 3.1.1.

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

  1. Use S3 as primary storage
  2. Update Preview Generator to 3.1.1
  3. Run Preview Generator cronjob

The output of your Nextcloud log in Admin > Logging:

[objectstore] Error: Exception: Creation of bucket "<obfuscated>" failed. Error executing "CreateBucket" on "https://<obfuscated>.s3.eu-central-1.amazonaws.com/"; AWS HTTP error: cURL error 6: getaddrinfo() thread failed to start
 (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) at <<closure>>

 0. /var/www/virtual/mexell/html/lib/private/Files/ObjectStore/S3ObjectTrait.php line 90
    OC\Files\ObjectStore\S3->getConnection()
 1. /var/www/virtual/mexell/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php line 492
    OC\Files\ObjectStore\S3->writeObject("urn:oid:345750", null)
 2. /var/www/virtual/mexell/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php line 422
    OC\Files\ObjectStore\ObjectStoreStorage->writeStream("appdata_oc276gm ... g", null, 332463)
 3. /var/www/virtual/mexell/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php line 335
    OC\Files\ObjectStore\ObjectStoreStorage->writeBack("/tmp/oc_tmp_HzsRAL-.jpg", "appdata_oc276gm ... g")
 4. <<closure>>
    OC\Files\ObjectStore\ObjectStoreStorage->OC\Files\ObjectStore\{closure}("*** sensitive parameters replaced ***")
 5. /var/www/virtual/mexell/html/3rdparty/icewind/streams/src/CallbackWrapper.php line 121
    call_user_func(Closure {})
 6. <<closure>>
    Icewind\Streams\CallbackWrapper->stream_close()
 7. /var/www/virtual/mexell/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php line 443
    fclose(null)
 8. /var/www/virtual/mexell/html/lib/private/Files/Storage/Wrapper/Wrapper.php line 257
    OC\Files\ObjectStore\ObjectStoreStorage->file_put_contents("appdata_oc276gm ... g", null)
 9. /var/www/virtual/mexell/html/lib/private/Files/Storage/Wrapper/Availability.php line 275
    OC\Files\Storage\Wrapper\Wrapper->file_put_contents("appdata_oc276gm ... g", null)
10. /var/www/virtual/mexell/html/lib/private/Files/Storage/Wrapper/Wrapper.php line 257
    OC\Files\Storage\Wrapper\Availability->file_put_contents("appdata_oc276gm ... g", null)
11. /var/www/virtual/mexell/html/lib/private/Files/View.php line 1165
    OC\Files\Storage\Wrapper\Wrapper->file_put_contents("appdata_oc276gm ... g", null)
12. /var/www/virtual/mexell/html/lib/private/Files/View.php line 702
    OC\Files\View->basicOperation("file_put_contents", "/appdata_oc276g ... g", ["create","write"], null)
13. /var/www/virtual/mexell/html/lib/private/Files/Node/Folder.php line 189
    OC\Files\View->file_put_contents("/appdata_oc276g ... g", null)
14. /var/www/virtual/mexell/html/lib/private/Files/SimpleFS/NewSimpleFile.php line 133
    OC\Files\Node\Folder->newFile("1200-1600-max.jpg", null)
15. /var/www/virtual/mexell/html/lib/private/Preview/Generator.php line 261
    OC\Files\SimpleFS\NewSimpleFile->putContent(null)
16. /var/www/virtual/mexell/html/lib/private/Preview/Generator.php line 140
    OC\Preview\Generator->getMaxPreview(OC\Files\SimpleFS\SimpleFolder {}, OC\Files\SimpleFS\NewSimpleFile {}, "image/jpeg", "")
17. /var/www/virtual/mexell/html/lib/private/PreviewManager.php line 205
    OC\Preview\Generator->generatePreviews(OC\Files\Node\File {}, [{width: 32,heig ... }], "image/jpeg")
18. /var/www/virtual/mexell/html/apps/previewgenerator/lib/Command/PreGenerate.php line 217
    OC\PreviewManager->generatePreviews(OC\Files\Node\File {}, [{width: 32,heig ... }])
19. /var/www/virtual/mexell/html/apps/previewgenerator/lib/Command/PreGenerate.php line 195
    OCA\PreviewGenerator\Command\PreGenerate->processFile(OC\Files\Node\File {})
20. /var/www/virtual/mexell/html/apps/previewgenerator/lib/Command/PreGenerate.php line 163
    OCA\PreviewGenerator\Command\PreGenerate->processRow({id: "29713",uid ... "})
21. /var/www/virtual/mexell/html/apps/previewgenerator/lib/Command/PreGenerate.php line 131
    OCA\PreviewGenerator\Command\PreGenerate->startProcessing()
22. /var/www/virtual/mexell/html/3rdparty/symfony/console/Command/Command.php line 255
    OCA\PreviewGenerator\Command\PreGenerate->execute(Symfony\Componen ... {}, Symfony\Componen ... {})
23. /var/www/virtual/mexell/html/3rdparty/symfony/console/Application.php line 1000
    Symfony\Component\Console\Command\Command->run(Symfony\Componen ... {}, Symfony\Componen ... {})
24. /var/www/virtual/mexell/html/3rdparty/symfony/console/Application.php line 271
    Symfony\Component\Console\Application->doRunCommand(OCA\PreviewGener ... {}, Symfony\Componen ... {}, Symfony\Componen ... {})
25. /var/www/virtual/mexell/html/3rdparty/symfony/console/Application.php line 147
    Symfony\Component\Console\Application->doRun(Symfony\Componen ... {}, Symfony\Componen ... {})
26. /var/www/virtual/mexell/html/lib/private/Console/Application.php line 215
    Symfony\Component\Console\Application->run(Symfony\Componen ... {}, Symfony\Componen ... {})
27. /var/www/virtual/mexell/html/console.php line 100
    OC\Console\Application->run()
28. /var/www/virtual/mexell/html/occ line 11
    require_once("/var/www/virtua ... p")

at 2021-01-29T09:09:11+00:00

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'passwordsalt' => '<obfuscated>',
  'secret' => '<obfuscated>',
  'trusted_domains' =>
  array (
    0 => '<obfuscated>',
  ),
  'datadirectory' => '/home/mexell/nextcloud_data',
  'dbtype' => 'mysql',
  'version' => '20.0.6.1',
  'overwrite.cli.url' => '<obfuscated>',
  'dbname' => '<obfuscated>',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'mexell',
  'dbpassword' => '<obfuscated>',
  'installed' => true,
  'instanceid' => 'oc276gm0ohuv',
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'redis' =>
  array (
    'host' => '/home/mexell/.redis/sock',
    'port' => '0',
    'timeout' => '1.5',
  ),
  'filelocking.enabled' => 'true',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'objectstore' =>
  array (
    'class' => '\\OC\\Files\\ObjectStore\\S3',
    'arguments' =>
    array (
      'bucket' => '<obfuscated>',
      'autocreate' => true,
      'key' => '<obfuscated>',
      'secret' => '<obfuscated>',
      'use_ssl' => true,
      'region' => 'eu-central-1',
      'use_path_style' => false,
    ),
  ),
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'tls',
  'mail_sendmailmode' => 'smtp',
  'mail_from_address' => '<obfuscated>',
  'mail_domain' => '<obfuscated>',
  'mail_smtpauthtype' => 'PLAIN',
  'mail_smtphost' => '<obfuscated>',
  'mail_smtpport' => '587',
  'mail_smtpauth' => 1,
  'mail_smtpname' => '<obfuscated>',
  'mail_smtppassword' => '<obfuscated>',
  'preview_max_x' => '2048',
  'preview_max_y' => '2048',
  'jpeg_quality' => '60',
  'maintenance' => false,
);

One more piece of information: The preview process starts, runs for a bit, and crashes. This is what happens when I start it manually with -vvvv to see what’s going on:

An unhandled exception has been thrown:
Error: Class 'OC\Log\ExceptionSerializer' not found in /var/www/virtual/mexell/html/lib/private/Log.php:318
Stack trace:
#0 /var/www/virtual/mexell/html/lib/private/Log/ErrorHandler.php(92): OC\Log->logException(Object(Error), Array)
#1 /var/www/virtual/mexell/html/lib/composer/composer/ClassLoader.php(444): OC\Log\ErrorHandler::onError(2, 'include(/var/ww...', '/var/www/virtua...', 444, Array)
#2 /var/www/virtual/mexell/html/lib/composer/composer/ClassLoader.php(444): include()
#3 /var/www/virtual/mexell/html/lib/composer/composer/ClassLoader.php(322): Composer\Autoload\includeFile('/var/www/virtua...')
#4 [internal function]: Composer\Autoload\ClassLoader->loadClass('GuzzleHttp\\Exce...')
#5 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php(200): spl_autoload_call('GuzzleHttp\\Exce...')
#6 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php(155): GuzzleHttp\Handler\CurlFactory::createRejection(Object(GuzzleHttp\Handler\EasyHandle), Array)
#7 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php(105): GuzzleHttp\Handler\CurlFactory::finishError(Object(GuzzleHttp\Handler\CurlMultiHandler), Object(GuzzleHttp\Handler\EasyHandle), Object(GuzzleHttp\Handler\CurlFactory))
#8 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(201): GuzzleHttp\Handler\CurlFactory::finish(Object(GuzzleHttp\Handler\CurlMultiHandler), Object(GuzzleHttp\Handler\EasyHandle), Object(GuzzleHttp\Handler\CurlFactory))
#9 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(130): GuzzleHttp\Handler\CurlMultiHandler->processMessages()
#10 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(145): GuzzleHttp\Handler\CurlMultiHandler->tick()
#11 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/promises/src/Promise.php(246): GuzzleHttp\Handler\CurlMultiHandler->execute(true)
#12 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/promises/src/Promise.php(223): GuzzleHttp\Promise\Promise->invokeWaitFn()
#13 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/promises/src/Promise.php(267): GuzzleHttp\Promise\Promise->waitIfPending()
#14 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/promises/src/Promise.php(225): GuzzleHttp\Promise\Promise->invokeWaitList()
#15 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/promises/src/Promise.php(267): GuzzleHttp\Promise\Promise->waitIfPending()
#16 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/promises/src/Promise.php(225): GuzzleHttp\Promise\Promise->invokeWaitList()
#17 /var/www/virtual/mexell/html/3rdparty/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\Promise\Promise->waitIfPending()
#18 /var/www/virtual/mexell/html/3rdparty/aws/aws-sdk-php/src/AwsClientTrait.php(58): GuzzleHttp\Promise\Promise->wait()
#19 /var/www/virtual/mexell/html/3rdparty/aws/aws-sdk-php/src/S3/S3ClientTrait.php(287): Aws\AwsClient->execute(Object(Aws\Command))
#20 /var/www/virtual/mexell/html/3rdparty/aws/aws-sdk-php/src/S3/S3ClientTrait.php(259): Aws\S3\S3Client->checkExistenceWithCommand(Object(Aws\Command))
#21 /var/www/virtual/mexell/html/lib/private/Files/ObjectStore/S3ConnectionTrait.php(133): Aws\S3\S3Client->doesBucketExist('nc-laemke-de')
#22 /var/www/virtual/mexell/html/lib/private/Files/ObjectStore/S3ObjectTrait.php(90): OC\Files\ObjectStore\S3->getConnection()
#23 /var/www/virtual/mexell/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php(492): OC\Files\ObjectStore\S3->writeObject('urn:oid:362003', Resource id #28006)
#24 /var/www/virtual/mexell/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php(422): OC\Files\ObjectStore\ObjectStoreStorage->writeStream('appdata_oc276gm...', Resource id #28006, 177811)
#25 /var/www/virtual/mexell/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php(335): OC\Files\ObjectStore\ObjectStoreStorage->writeBack('/tmp/oc_tmp_JIF...', 'appdata_oc276gm...')
#26 [internal function]: OC\Files\ObjectStore\ObjectStoreStorage->OC\Files\ObjectStore\{closure}()
#27 /var/www/virtual/mexell/html/3rdparty/icewind/streams/src/CallbackWrapper.php(121): call_user_func(Object(Closure))
#28 [internal function]: Icewind\Streams\CallbackWrapper->stream_close()
#29 /var/www/virtual/mexell/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php(443): fclose(Resource id #28005)
#30 /var/www/virtual/mexell/html/lib/private/Files/Storage/Wrapper/Wrapper.php(257): OC\Files\ObjectStore\ObjectStoreStorage->file_put_contents('appdata_oc276gm...', '\xFF\xD8\xFF\xE0\x00\x10JFIF\x00\x01\x01\x01\x00...')
#31 /var/www/virtual/mexell/html/lib/private/Files/Storage/Wrapper/Availability.php(275): OC\Files\Storage\Wrapper\Wrapper->file_put_contents('appdata_oc276gm...', '\xFF\xD8\xFF\xE0\x00\x10JFIF\x00\x01\x01\x01\x00...')
#32 /var/www/virtual/mexell/html/lib/private/Files/Storage/Wrapper/Wrapper.php(257): OC\Files\Storage\Wrapper\Availability->file_put_contents('appdata_oc276gm...', '\xFF\xD8\xFF\xE0\x00\x10JFIF\x00\x01\x01\x01\x00...')
#33 /var/www/virtual/mexell/html/lib/private/Files/View.php(1165): OC\Files\Storage\Wrapper\Wrapper->file_put_contents('appdata_oc276gm...', '\xFF\xD8\xFF\xE0\x00\x10JFIF\x00\x01\x01\x01\x00...')
#34 /var/www/virtual/mexell/html/lib/private/Files/View.php(702): OC\Files\View->basicOperation('file_put_conten...', '/appdata_oc276g...', Array, '\xFF\xD8\xFF\xE0\x00\x10JFIF\x00\x01\x01\x01\x00...')
#35 /var/www/virtual/mexell/html/lib/private/Files/Node/Folder.php(189): OC\Files\View->file_put_contents('/appdata_oc276g...', '\xFF\xD8\xFF\xE0\x00\x10JFIF\x00\x01\x01\x01\x00...')
#36 /var/www/virtual/mexell/html/lib/private/Files/SimpleFS/NewSimpleFile.php(133): OC\Files\Node\Folder->newFile('1600-1068-max.j...', '\xFF\xD8\xFF\xE0\x00\x10JFIF\x00\x01\x01\x01\x00...')
#37 /var/www/virtual/mexell/html/lib/private/Preview/Generator.php(261): OC\Files\SimpleFS\NewSimpleFile->putContent('\xFF\xD8\xFF\xE0\x00\x10JFIF\x00\x01\x01\x01\x00...')
#38 /var/www/virtual/mexell/html/lib/private/Preview/Generator.php(140): OC\Preview\Generator->getMaxPreview(Object(OC\Files\SimpleFS\SimpleFolder), Object(OC\Files\SimpleFS\NewSimpleFile), 'image/jpeg', '')
#39 /var/www/virtual/mexell/html/lib/private/PreviewManager.php(205): OC\Preview\Generator->generatePreviews(Object(OC\Files\Node\File), Array, 'image/jpeg')
#40 /var/www/virtual/mexell/html/apps/previewgenerator/lib/Command/PreGenerate.php(217): OC\PreviewManager->generatePreviews(Object(OC\Files\Node\File), Array)
#41 /var/www/virtual/mexell/html/apps/previewgenerator/lib/Command/PreGenerate.php(195): OCA\PreviewGenerator\Command\PreGenerate->processFile(Object(OC\Files\Node\File))
#42 /var/www/virtual/mexell/html/apps/previewgenerator/lib/Command/PreGenerate.php(163): OCA\PreviewGenerator\Command\PreGenerate->processRow(Array)
#43 /var/www/virtual/mexell/html/apps/previewgenerator/lib/Command/PreGenerate.php(131): OCA\PreviewGenerator\Command\PreGenerate->startProcessing()
#44 /var/www/virtual/mexell/html/3rdparty/symfony/console/Command/Command.php(255): OCA\PreviewGenerator\Command\PreGenerate->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#45 /var/www/virtual/mexell/html/3rdparty/symfony/console/Application.php(1000): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#46 /var/www/virtual/mexell/html/3rdparty/symfony/console/Application.php(271): Symfony\Component\Console\Application->doRunCommand(Object(OCA\PreviewGenerator\Command\PreGenerate), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#47 /var/www/virtual/mexell/html/3rdparty/symfony/console/Application.php(147): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#48 /var/www/virtual/mexell/html/lib/private/Console/Application.php(215): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#49 /var/www/virtual/mexell/html/console.php(100): OC\Console\Application->run()
#50 /var/www/virtual/mexell/html/occ(11): require_once('/var/www/virtua...')
#51 {main}{"reqId":"Pre90N0SyQNvqQSEB7Hr","level":3,"time":"2021-01-29T15:36:01+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"include(): Failed opening '/var/www/virtual/mexell/html/lib/composer/composer/../../../lib/private/Log/ExceptionSerializer.php' for inclusion (include_path='/var/www/virtual/mexell/html/3rdparty/pear/archive_tar:/var/www/virtual/mexell/html/3rdparty/pear/console_getopt:/var/www/virtual/mexell/html/3rdparty/pear/pear-core-minimal/src:/var/www/virtual/mexell/html/3rdparty/pear/pear_exception:/var/www/virtual/mexell/html/apps') at /var/www/virtual/mexell/html/lib/composer/composer/ClassLoader.php#444","userAgent":"--","version":"20.0.6.1"}