Errors accessing s3 backend

ℹ️ Support
1

Hello,

I’m trying to solve a very annoying issue. I have a docker container of nextcloud using LinuxServer image. I’m using an s3 object storage at Scaleway which has been working great until know.

I have a tons of log error and console error when opening the network inspector in Firefox. The result is that files are listed, but I cannot have any preview of it, or open it.

Some logs

{"reqId":"Ja6eD9XgVwgC2oPcJuUZ","level":3,"time":"2020-04-20T13:44:00+00:00","remoteAddr":"172.18.0.1","user":"c4534a82-ebf6-1038-9fee-d975121bf7b5","app":"PHP","method":"GET","url":"\/apps\/gallery\/preview\/40946?width=400&height=200","message":"fread() expects parameter 1 to be resource, bool given at \/config\/www\/nextcloud\/lib\/private\/Files\/Storage\/Wrapper\/Encryption.php#858","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/80.0.3987.163 Safari\/537.36","version":"17.0.4.2"}
{"reqId":"Ja6eD9XgVwgC2oPcJuUZ","level":3,"time":"2020-04-20T13:44:00+00:00","remoteAddr":"172.18.0.1","user":"c4534a82-ebf6-1038-9fee-d975121bf7b5","app":"PHP","method":"GET","url":"\/apps\/gallery\/preview\/40946?width=400&height=200","message":"fclose() expects parameter 1 to be resource, bool given at \/config\/www\/nextcloud\/lib\/private\/Files\/Storage\/Wrapper\/Encryption.php#859","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/80.0.3987.163 Safari\/537.36","version":"17.0.4.2"}
{"reqId":"aUnja1I02Yagec0wv2Xb","level":3,"time":"2020-04-20T13:44:01+00:00","remoteAddr":"172.18.0.1","user":"c4534a82-ebf6-1038-9fee-d975121bf7b5","app":"PHP","method":"GET","url":"\/apps\/gallery\/preview\/40951?width=400&height=200","message":"fopen(https:\/\/s3-imp-nextcloud.s3.nl-ams.scw.cloud\/urn%3Aoid%3A42389): failed to open stream: HTTP request failed! HTTP\/1.1 500 Internal Server Error\r\n at \/config\/www\/nextcloud\/lib\/private\/Files\/ObjectStore\/S3ObjectTrait.php#70","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/80.0.3987.163 Safari\/537.36","version":"17.0.4.2"}
{"reqId":"aUnja1I02Yagec0wv2Xb","level":3,"time":"2020-04-20T13:44:01+00:00","remoteAddr":"172.18.0.1","user":"c4534a82-ebf6-1038-9fee-d975121bf7b5","app":"gallery","method":"GET","url":"\/apps\/gallery\/preview\/40951?width=400&height=200","message":{"Exception":"OCP\\Files\\NotFoundException","Message":"","Code":0,"Trace":[{"file":"\/config\/www\/nextcloud\/lib\/private\/PreviewManager.php","line":210,"function":"getPreview","class":"OC\\Preview\\Generator","type":"->","args":[{"__class__":"OC\\Files\\Node\\File"},410,256,false,"fill","image\/jpeg"]},{"file":"\/config\/www\/nextcloud\/apps\/gallery\/lib\/Service\/PreviewService.php","line":106,"function":"getPreview","class":"OC\\PreviewManager","type":"->","args":[{"__class__":"OC\\Files\\Node\\File"},400,200,false]},{"file":"\/config\/www\/nextcloud\/apps\/gallery\/lib\/Controller\/Preview.php","line":167,"function":"createPreview","class":"OCA\\Gallery\\Service\\PreviewService","type":"->","args":[{"__class__":"OC\\Files\\Node\\File"},400,200,true,false]},{"file":"\/config\/www\/nextcloud\/apps\/gallery\/lib\/Controller\/Preview.php","line":117,"function":"getPreviewData","class":"OCA\\Gallery\\Controller\\PreviewController","type":"->","args":[{"__class__":"OC\\Files\\Node\\File"},true,400,200,true,false]},{"file":"\/config\/www\/nextcloud\/apps\/gallery\/lib\/Controller\/PreviewController.php","line":134,"function":"getData","class":"OCA\\Gallery\\Controller\\PreviewController","type":"->","args":[40951,400,200]},{"file":"\/config\/www\/nextcloud\/lib\/private\/AppFramework\/Http\/Dispatcher.php","line":170,"function":"getPreview","class":"OCA\\Gallery\\Controller\\PreviewController","type":"->","args":[40951,400,200]},{"file":"\/config\/www\/nextcloud\/lib\/private\/AppFramework\/Http\/Dispatcher.php","line":99,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Gallery\\Controller\\PreviewController"},"getPreview"]},{"file":"\/config\/www\/nextcloud\/lib\/private\/AppFramework\/App.php","line":126,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Gallery\\Controller\\PreviewController"},"getPreview"]},{"file":"\/config\/www\/nextcloud\/lib\/private\/AppFramework\/Routing\/RouteActionHandler.php","line":47,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["PreviewController","getPreview",{"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},{"fileId":"40951","_route":"gallery.preview.get_preview"}]},{"function":"__invoke","class":"OC\\AppFramework\\Routing\\RouteActionHandler","type":"->","args":[{"fileId":"40951","_route":"gallery.preview.get_preview"}]},{"file":"\/config\/www\/nextcloud\/lib\/private\/Route\/Router.php","line":297,"function":"call_user_func","args":[{"__class__":"OC\\AppFramework\\Routing\\RouteActionHandler"},{"fileId":"40951","_route":"gallery.preview.get_preview"}]},{"file":"\/config\/www\/nextcloud\/lib\/base.php","line":997,"function":"match","class":"OC\\Route\\Router","type":"->","args":["\/apps\/gallery\/preview\/40951"]},{"file":"\/config\/www\/nextcloud\/index.php","line":42,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"\/config\/www\/nextcloud\/lib\/private\/Preview\/Generator.php","Line":156,"CustomMessage":"--"},"userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/80.0.3987.163 Safari\/537.36","version":"17.0.4.2"}
{"reqId":"aUnja1I02Yagec0wv2Xb","level":3,"time":"2020-04-20T13:44:01+00:00","remoteAddr":"172.18.0.1","user":"c4534a82-ebf6-1038-9fee-d975121bf7b5","app":"gallery","method":"GET","url":"\/apps\/gallery\/preview\/40951?width=400&height=200","message":"Exception: Preview generation has failed","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/80.0.3987.163 Safari\/537.36","version":"17.0.4.2"}
{"reqId":"Ja6eD9XgVwgC2oPcJuUZ","level":3,"time":"2020-04-20T13:44:01+00:00","remoteAddr":"172.18.0.1","user":"c4534a82-ebf6-1038-9fee-d975121bf7b5","app":"PHP","method":"GET","url":"\/apps\/gallery\/preview\/40946?width=400&height=200","message":"fopen(https:\/\/s3-imp-nextcloud.s3.nl-ams.scw.cloud\/urn%3Aoid%3A41625): failed to open stream: HTTP request failed! HTTP\/1.1 500 Internal Server Error\r\n at \/config\/www\/nextcloud\/lib\/private\/Files\/ObjectStore\/S3ObjectTrait.php#70","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/80.0.3987.163 Safari\/537.36","version":"17.0.4.2"}
{"reqId":"Ja6eD9XgVwgC2oPcJuUZ","level":3,"time":"2020-04-20T13:44:01+00:00","remoteAddr":"172.18.0.1","user":"c4534a82-ebf6-1038-9fee-d975121bf7b5","app":"index","method":"GET","url":"\/apps\/gallery\/preview\/40946?width=400&height=200","message":{"Exception":"TypeError","Message":"Argument 3 passed to OC\\AppFramework\\Middleware\\MiddlewareDispatcher::beforeOutput() must be of the type string, null given, called in \/config\/www\/nextcloud\/lib\/private\/AppFramework\/Http\/Dispatcher.php on line 123","Code":0,"Trace":[{"file":"\/config\/www\/nextcloud\/lib\/private\/AppFramework\/Http\/Dispatcher.php","line":123,"function":"beforeOutput","class":"OC\\AppFramework\\Middleware\\MiddlewareDispatcher","type":"->","args":[{"__class__":"OCA\\Gallery\\Controller\\PreviewController"},"getPreview",null]},{"file":"\/config\/www\/nextcloud\/lib\/private\/AppFramework\/App.php","line":126,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Gallery\\Controller\\PreviewController"},"getPreview"]},{"file":"\/config\/www\/nextcloud\/lib\/private\/AppFramework\/Routing\/RouteActionHandler.php","line":47,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["PreviewController","getPreview",{"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},{"fileId":"40946","_route":"gallery.preview.get_preview"}]},{"function":"__invoke","class":"OC\\AppFramework\\Routing\\RouteActionHandler","type":"->","args":[{"fileId":"40946","_route":"gallery.preview.get_preview"}]},{"file":"\/config\/www\/nextcloud\/lib\/private\/Route\/Router.php","line":297,"function":"call_user_func","args":[{"__class__":"OC\\AppFramework\\Routing\\RouteActionHandler"},{"fileId":"40946","_route":"gallery.preview.get_preview"}]},{"file":"\/config\/www\/nextcloud\/lib\/base.php","line":997,"function":"match","class":"OC\\Route\\Router","type":"->","args":["\/apps\/gallery\/preview\/40946"]},{"file":"\/config\/www\/nextcloud\/index.php","line":42,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"\/config\/www\/nextcloud\/lib\/private\/AppFramework\/Middleware\/MiddlewareDispatcher.php","Line":157,"CustomMessage":"--"},"userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/80.0.3987.163 Safari\/537.36","version":"17.0.4.2"}

Screenshot_20200420_155115
Screenshot_20200420_1551151280×551 86.7 KB

It can still be visible at https://nextcloud.media.imperium-gaming.fr

I was on 18.03 version (latest), but tried to rollback to 3 differents backups (nextcloud config files + database), including a nextcloud v17.04 and it was always the same issue.

I would have just started from fresh if I could but as nextcloud is storing files in some specific ways (creating files like “urn:oid:48802”) I sadly cannot get my files back…

I’m still conversion with my object storage provider to see if he has any backup but i’m not optimist…

My config.php but not sure it helps:
Also, encrytion is disabled.

<?php
$CONFIG = array (
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'instanceid' => 'xxx',
  'passwordsalt' => 'xxxx',
  'secret' => 'xxx',
  'trusted_domains' => 
  array (
    0 => 'nextcloud.media.imperium-gaming.fr',
  ),
  'dbtype' => 'mysql',
  'version' => '17.0.4.2',
  'overwrite.cli.url' => 'https://nextcloud.media.imperium-gaming.fr',
  'installed' => true,
  'ldapIgnoreNamingRules' => false,
  'ldapProviderFactory' => 'OCA\\User_LDAP\\LDAPProviderFactory',
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 2,
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'tls',
  'mail_sendmailmode' => 'smtp',
  'mail_from_address' => 'nextcloud.media',
  'mail_domain' => 'mydomain',
  'mail_smtphost' => 'my-smtp',
  'mail_smtpport' => '587',
  'app_install_overwrite' => 
  array (
    0 => 'impersonate',
  ),
  'mysql.utf8mb4' => true,
  'dbname' => 'nextcloud',
  'dbhost' => 'mariadb',
  'dbuser' => 'nextcloud',
  'dbpassword' => 'xxxx',
  'datadirectory' => '/data',
  'objectstore' => 
  array (
    'class' => '\\OC\\Files\\ObjectStore\\S3',
    'arguments' => 
    array (
      'bucket' => 'xxxx',
      'autocreate' => true,
      'key' => 'xxxx',
      'secret' => 'xxxx',
      'hostname' => 's3.nl-ams.scw.cloud',
      'port' => 443,
      'use_ssl' => true,
      'region' => 'nl-ams',
    ),
  ),
);

Any one with the ability to help will be appreciated.

2

Why can you not open the following Javascript with Firefox?
Please post error of you firefox networking analysis.

https://nextcloud.media.imperium-gaming.fr/core/js/dist/main.js

3

Hello @devnull,

Well I dont know why, I figure it’s all linked to the same issue, I tried from both computers from the same place… Dont you have theses errors ?

I can accede the link you provided, but firefox is still displaying errors in the network inspector when i go to the root url

4

You tried to revert multiple upgrades while using S3?

All of those files are lost. The database stores the location of each file and you’ve reverted to an older database. You will need to copy the entire S3 directory to your PC and sort the files one by one by hand to get the data back.

You would need to revert to a backup you took JUST BEFORE trying to change your installation to an older version.

Regarding your errors - those are understandable. The files are no longer on the backend and/or not where nextcloud expects them to be.

Why did you downgrade originally?

5

Hello @Paradox,

I got a backup for before the reverts. I had fall back to the original files (prior to any revert/restore on my side) one hour ago.

I had theses issues from the start before any reverts/restores.

The downgrade was just a side effect of my oldest backup restore.

1 Like
6

I just got news from my object storage provider, the problem was well on their side…

I will be waiting for it to conclude, this issue can be closed.

7

Because of the error in your firefox networking analysis. It was not HTTP 200.