Cannot list files or read file contents over object storage (Aliyun OSS using S3 connector)

When storing files on OSS, it appears as though nextcloud is able to write to, but is unable to read from, the object store. Created files will appear in the storage control panel itself, but will be invisible to nextcloud.

Encryption will fail with a key not found message, I believe this is because encryption can’t find a key for an inaccessible file.

I have verified this by turning off encryption and attempting to create a file. The file appears on the server and can even be written to sporadically, but does not appear in the listings on nextcloud.

Nextcloud will also present a “File already exists” error if I try to create a file with the same name.

{“reqId”:"[redacted]",“level”:0,“time”:“2018-04-13T10:18:02+00:00”,“remoteAddr”:"[redacted]",“user”:"[redacted]",“app”:“PHP”,“method”:“PUT”,“url”:"/remote.php/webdav/Aliyun/sdfasdf.txt",“message”:“Invalid argument supplied for foreach() at /var/www/nextcloud/apps/files_external/lib/Lib/Storage/AmazonS3.php#295”,“userAgent”:“Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36”,“version”:“13.0.1.1”}
{“reqId”:"[redacted]",“level”:0,“time”:“2018-04-13T10:18:02+00:00”,“remoteAddr”:"[redacted]",“user”:"[redacted]",“app”:“PHP”,“method”:“PUT”,“url”:"/remote.php/webdav/Aliyun/sdfasdf.txt",“message”:“rewinddir(): 62 is not a valid Directory resource at /var/www/nextcloud/apps/files_external/3rdparty/icewind/streams/src/Wrapper.php#31”,“userAgent”:“Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36”,“version”:“13.0.1.1”}
{“reqId”:"[redacted]",“level”:0,“time”:“2018-04-13T10:18:02+00:00”,“remoteAddr”:"[redacted]",“user”:"[redacted]",“app”:“PHP”,“method”:“PUT”,“url”:"/remote.php/webdav/Aliyun/sdfasdf.txt",“message”:“rewinddir(): 66 is not a valid Directory resource at /var/www/nextcloud/lib/private/Files/Stream/Encryption.php#198”,“userAgent”:“Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36”,“version”:“13.0.1.1”}
{“reqId”:"[redacted]",“level”:0,“time”:“2018-04-13T10:18:11+00:00”,“remoteAddr”:"[redacted]",“user”:"[redacted]",“app”:“PHP”,“method”:“GET”,“url”:"/apps/files_texteditor/ajax/loadfile?filename=sdfasdf.txt&dir=%2FAliyun",“message”:“rewinddir(): 75 is not a valid Directory resource at /var/www/nextcloud/lib/private/Files/Stream/Encryption.php#198”,“userAgent”:“Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36”,“version”:“13.0.1.1”}
{“reqId”:"[redacted]",“level”:0,“time”:“2018-04-13T10:18:11+00:00”,“remoteAddr”:"[redacted]",“user”:"[redacted]",“app”:“PHP”,“method”:“GET”,“url”:"/core/preview.png?file=%2FAliyun%2Fsdfasdf.txt&c=5ad083db4b4b6&x=32&y=32&forceIcon=0",“message”:“rewinddir(): 72 is not a valid Directory resource at /var/www/nextcloud/lib/private/Files/Stream/Encryption.php#198”,“userAgent”:“Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36”,“version”:“13.0.1.1”}
{“reqId”:"[redacted]",“level”:0,“time”:“2018-04-13T10:18:11+00:00”,“remoteAddr”:"[redacted]",“user”:"[redacted]",“app”:“PHP”,“method”:“GET”,“url”:"/core/preview.png?file=%2FAliyun%2Fsdfasdf.txt&c=5ad083db4b4b6&x=32&y=32&forceIcon=0",“message”:“rewinddir(): 80 is not a valid Directory resource at /var/www/nextcloud/apps/files_external/3rdparty/icewind/streams/src/Wrapper.php#31”,“userAgent”:“Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36”,“version”:“13.0.1.1”}
{“reqId”:"[redacted]",“level”:0,“time”:“2018-04-13T10:18:12+00:00”,“remoteAddr”:"[redacted]",“user”:"[redacted]",“app”:“PHP”,“method”:“GET”,“url”:"/apps/files_texteditor/ajax/loadfile?filename=sdfasdf.txt&dir=Aliyun",“message”:“rewinddir(): 75 is not a valid Directory resource at /var/www/nextcloud/lib/private/Files/Stream/Encryption.php#198”,“userAgent”:“Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36”,“version”:“13.0.1.1”}
{“reqId”:"[redacted]",“level”:3,“time”:“2018-04-13T10:18:13+00:00”,“remoteAddr”:"[redacted]",“user”:"[redacted]",“app”:“index”,“method”:“GET”,“url”:"/core/preview.png?file=%2FAliyun%2Fsdfasdf.txt&c=225&x=32&y=32&forceIcon=0",“message”:“Exception: {“Exception”:“OCP\\Lock\\LockedException”,“Message”:”\“Aliyun\/sdfasdf.txt\” is locked",“Code”:0,“Trace”:"#0 \/var\/www\/nextcloud\/lib\/private\/Files\/View.php(2036): OC\\Files\\View->lockPath(’\/files\/Aliyun\/s…’, 1, false)\n#1 \/var\/www\/nextcloud\/lib\/private\/Files\/View.php(1134): OC\\Files\\View->lockFile(’\/files\/Aliyun\/s…’, 1)\n#2 \/var\/www\/nextcloud\/lib\/private\/Files\/View.php(987): OC\\Files\\View->basicOperation(‘fopen’, ‘\/files\/Aliyun\/s…’, Array, ‘r’)\n#3 \/var\/www\/nextcloud\/lib\/private\/Preview\/TXT.php(48): OC\\Files\\View->fopen(’\/files\/Aliyun\/s…’, ‘r’)\n#4 \/var\/www\/nextcloud\/lib\/private\/Preview\/GeneratorHelper.php(54): OC\\Preview\\TXT->getThumbnail(’\/files\/Aliyun\/s…’, 4096, 4096, false, Object(OC\\Files\\View))\n#5 \/var\/www\/nextcloud\/lib\/private\/Preview\/Generator.php(184): OC\\Preview\\GeneratorHelper->getThumbnail(Object(OC\\Preview\\TXT), Object(OC\\Files\\Node\\File), 4096, 4096)\n#6 \/var\/www\/nextcloud\/lib\/private\/Preview\/Generator.php(112): OC\\Preview\\Generator->getMaxPreview(Object(OC\\Files\\SimpleFS\\SimpleFolder), Object(OC\\Files\\Node\\File), ‘text\/plain’)\n#7 \/var\/www\/nextcloud\/lib\/private\/PreviewManager.php(202): OC\\Preview\\Generator->getPreview(Object(OC\\Files\\Node\\File), 32, 32, true, ‘fill’, ‘text\/plain’)\n#8 \/var\/www\/nextcloud\/core\/Controller\/PreviewController.php(114): OC\\PreviewManager->getPreview(Object(OC\\Files\\Node\\File), 32, 32, true, ‘fill’)\n#9 [internal function]: OC\\Core\\Controller\\PreviewController->getPreview(Object(OC\\Files\\Node\\File), 32, 32, false, false, ‘fill’)\n#10 \/var\/www\/nextcloud\/lib\/private\/AppFramework\/Http\/Dispatcher.php(161): call_user_func_array(Array, Array)\n#11 \/var\/www\/nextcloud\/lib\/private\/AppFramework\/Http\/Dispatcher.php(91): OC\\AppFramework\\Http\\Dispatcher->executeController(Object(OC\\Core\\Controller\\PreviewController), ‘getPreview’)\n#12 \/var\/www\/nextcloud\/lib\/private\/AppFramework\/App.php(115): OC\\AppFramework\\Http\\Dispatcher->dispatch(Object(OC\\Core\\Controller\\PreviewController), ‘getPreview’)\n#13 \/var\/www\/nextcloud\/lib\/private\/AppFramework\/Routing\/RouteActionHandler.php(47): OC\\AppFramework\\App::main(‘OC\\\\Core\\\\Control…’, ‘getPreview’, Object(OC\\AppFramework\\DependencyInjection\\DIContainer), Array)\n#14 [internal function]: OC\\AppFramework\\Routing\\RouteActionHandler->__invoke(Array)\n#15 \/var\/www\/nextcloud\/lib\/private\/Route\/Router.php(297): call_user_func(Object(OC\\AppFramework\\Routing\\RouteActionHandler), Array)\n#16 \/var\/www\/nextcloud\/lib\/base.php(998): OC\\Route\\Router->match(’\/core\/preview.p…’)\n#17 \/var\/www\/nextcloud\/index.php(37): OC::handleRequest()\n#18 {main}",“File”:"\/var\/www\/nextcloud\/lib\/private\/Files\/View.php",“Line”:1931}",“userAgent”:“Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36”,“version”:“13.0.1.1”}
{“reqId”:"[redacted]",“level”:3,“time”:“2018-04-13T10:18:14+00:00”,“remoteAddr”:"[redacted]",“user”:"[redacted]",“app”:“no app in context”,“method”:“GET”,“url”:"/apps/files_texteditor/ajax/loadfile?filename=sdfasdf.txt&dir=Aliyun",“message”:"$absolutePath must be relative to “files”, value is “%s”",“userAgent”:“Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36”,“version”:“13.0.1.1”}
{“reqId”:"[redacted]",“level”:0,“time”:“2018-04-13T10:18:17+00:00”,“remoteAddr”:"[redacted]",“user”:"[redacted]",“app”:“PHP”,“method”:“GET”,“url”:"/core/preview.png?file=%2FAliyun%2Fsdfasdf.txt&x=75&y=75&c=225&forceIcon=0",“message”:“rewinddir(): 71 is not a valid Directory resource at /var/www/nextcloud/lib/private/Files/Stream/Encryption.php#198”,“userAgent”:“Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36”,“version”:“13.0.1.1”}
{“reqId”:"[redacted]",“level”:0,“time”:“2018-04-13T10:18:17+00:00”,“remoteAddr”:"[redacted]",“user”:"[redacted]",“app”:“PHP”,“method”:“GET”,“url”:"/core/preview.png?file=%2FAliyun%2Fsdfasdf.txt&x=75&y=75&c=225&forceIcon=0",“message”:“rewinddir(): 79 is not a valid Directory resource at /var/www/nextcloud/apps/files_external/3rdparty/icewind/streams/src/Wrapper.php#31”,“userAgent”:“Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36”,“version”:“13.0.1.1”}

This is because Aliyun API is not 100% compatible with S3.

There are two solutions.

1: Minio. I believe Minio has the capacity to connect to Aliyun, and form there it is possible to connect to Minio using the S3 adapter on Nextcloud

2: Develop an Aliyun connector for nextcloud.

1 Like