Nextcloud version (eg, 18.0.2): 19.0.1snap2
Operating system and version (eg, Ubuntu 20.04): Fedora 32
The issue you are facing:
I have configured Nextcloud to use S3 as its primary storage, and activated server-side encryption. This seemed to work fine - I uploaded a number of images, created a couple user accounts, everything was working.
Then I used the desktop sync client (running on another Fedora machine) to upload ~35000 document files, about 25GB of content. This also worked correctly, no problems.
The problem came up when I tried to access a few PDF documents through the web interface after the upload. When I clicked a PDF file, the PDF viewer came up and showed its progress bar at the top. The progress bar ran part of the way - sometimes 40% or so, sometimes more like 80% - and then nothing more happened. I waited a long time, nothing.
I tried several different files and also uploaded extra ones manually. After a while I found that I can mostly access a given file correctly on the second attempt. Sometimes it takes three or four attempts - but eventually the file loads in the previewer.
I checked nextcloud.log and I found something interesting: there are loads of messages with the error “Bad Signature”. The messages look like this (just a part - the trace is very long - also, I redacted bits in a few places - see output from Admin/Logging below):
{
"reqId":"8E6IeVlTxZJP53z85IIW",
"level":3,
"time":"2020-08-05T18:27:40+00:00",
"remoteAddr":"***",
"user": "***",
"app":"no app in context",
"method":"GET",
"url":"/remote.php/webdav/documents/somedocument.pdf",
"message": {
"Exception":"OCP\\Encryption\\Exceptions\\GenericEncryptionExce
ption",
"Message":"Bad Signature",
"Code":0,
"Trace":[{
"file":"/snap/nextcloud/22400/htdocs/apps/encryption/lib/Crypt
o/Crypt.php",
"line":463,
"function":"checkSignature",
"class":"OCA\\Encryption\\Crypto\\Crypt",
"type":"->",
"args": ...
I tried to tail the log to see exactly which messages referred to which requests - this is a bit harder than it should be because apparently the log is not flushed after each message. However, I am quite sure that the Bad Signature errors appear even when the PDF file is displayed correctly in the browser.
One final thing I did: I downloaded one of the objects from the S3 storage, and I could clearly tell that it was an encrypted file (there’s an “OC …” header, readable in text format). I just did this to be sure that the encryption was taking place as intended.
The output of your Nextcloud log in Admin > Logging:
This is one of the messages I described above:
[no app in context] Error: OCP\Encryption\Exceptions\GenericEncryptionException: Bad Signature at <<closure>>
0. /snap/nextcloud/22400/htdocs/apps/encryption/lib/Crypto/Crypt.php line 463
OCA\Encryption\Crypto\Crypt->checkSignature("mMLhve4x7rLFlpt ... A", null, "be897356d39cc8b ... 1")
1. /snap/nextcloud/22400/htdocs/apps/encryption/lib/Crypto/Encryption.php line 375
OCA\Encryption\Crypto\Crypt->symmetricDecryptFileContent("*** sensitive parameter replaced ***", null, "AES-256-CTR", 1, "*** sensitive parameter replaced ***")
2. /snap/nextcloud/22400/htdocs/lib/private/Files/Storage/Wrapper/Encryption.php line 585
OCA\Encryption\Crypto\Encryption->decrypt("*** sensitive parameters replaced ***")
3. /snap/nextcloud/22400/htdocs/lib/private/Files/Storage/Wrapper/Encryption.php line 505
OC\Files\Storage\Wrapper\Encryption->fixUnencryptedSize("*** sensitive parameters replaced ***")
4. /snap/nextcloud/22400/htdocs/lib/private/Files/Storage/Wrapper/Encryption.php line 166
OC\Files\Storage\Wrapper\Encryption->verifyUnencryptedSize("*** sensitive parameters replaced ***")
5. /snap/nextcloud/22400/htdocs/lib/private/Files/Storage/Wrapper/Encryption.php line 409
OC\Files\Storage\Wrapper\Encryption->filesize("*** sensitive parameter replaced ***")
6. /snap/nextcloud/22400/htdocs/lib/private/Files/Storage/Wrapper/Wrapper.php line 300
OC\Files\Storage\Wrapper\Encryption->fopen("*** sensitive parameter replaced ***", "r")
7. /snap/nextcloud/22400/htdocs/lib/private/Files/View.php line 1159
OC\Files\Storage\Wrapper\Wrapper->fopen("*** sensitive parameter replaced ***", "r")
8. /snap/nextcloud/22400/htdocs/lib/private/Files/View.php line 995
OC\Files\View->basicOperation("fopen", "/documents/2017 ... f", ["read"], "r")
9. /snap/nextcloud/22400/htdocs/apps/dav/lib/Connector/Sabre/File.php line 423
OC\Files\View->fopen("documents/somedoc ... f", "r")
10. /snap/nextcloud/22400/htdocs/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 90
OCA\DAV\Connector\Sabre\File->get()
11. /snap/nextcloud/22400/htdocs/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
Sabre\DAV\CorePlugin->httpGet(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
12. /snap/nextcloud/22400/htdocs/3rdparty/sabre/dav/lib/DAV/Server.php line 474
Sabre\DAV\Server->emit("method:GET", [Sabre\HTTP\Requ ... }])
13. /snap/nextcloud/22400/htdocs/3rdparty/sabre/dav/lib/DAV/Server.php line 251
Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
14. /snap/nextcloud/22400/htdocs/3rdparty/sabre/dav/lib/DAV/Server.php line 319
Sabre\DAV\Server->start()
15. /snap/nextcloud/22400/htdocs/apps/dav/appinfo/v1/webdav.php line 82
Sabre\DAV\Server->exec()
16. /snap/nextcloud/22400/htdocs/remote.php line 167
require_once("/snap/nextcloud ... p")
GET /remote.php/webdav/documents/somedocument.pdf
from *** by *** at 2020-08-05T18:27:40+00:00
I don’t know if they are connected, but I notice a few other types of error messages in Admin/Logging (they occur many times):
[no app in context] Error: Couldn't re-calculate unencrypted size for files/documents/somedocument.pdf
GET /remote.php/webdav/documents/somedocuments.pdf
from *** by *** at 2020-08-05T18:27:40+00:00
[PHP] Error: Cannot modify header information - headers already sent by (output started at /snap/nextcloud/22400/htdocs/3rdparty/sabre/http/lib/Sapi.php:112) at /snap/nextcloud/22400/htdocs/3rdparty/sabre/http/lib/Sapi.php#70
GET /remote.php/webdav/documents/somedocument.pdf
from *** by *** at 2020-08-05T18:25:59+00:00
[webdav] Fatal: Sabre\DAV\Exception: An exception occurred while completing a multipart upload: Error executing "CompleteMultipartUpload" on "https://***.s3.eu-west-2.amazonaws.com/urn***?uploadId=***"; AWS HTTP error: Client error: `POST https://***.s3.eu-west-2.amazonaws.com/urn***?uploadId=***` resulted in a `400 Bad Request` response:
<Error><Code>MalformedXML</Code><Message>The XML you provided was not well-formed or did not validate against our publis (truncated...)
MalformedXML (client): The XML you provided was not well-formed or did not validate against our published schema - <Error><Code>MalformedXML</Code><Message>The XML you provided was not well-formed or did not validate against our published schema</Message><RequestId>0410727350B5CD69</RequestId><HostId>***</HostId></Error> at <<closure>>
0. /snap/nextcloud/22400/htdocs/apps/dav/lib/Connector/Sabre/File.php line 252
OCA\DAV\Connector\Sabre\File->convertToSabreException(Aws\S3\Exception ... {})
1. /snap/nextcloud/22400/htdocs/apps/dav/lib/Connector/Sabre/Directory.php line 154
OCA\DAV\Connector\Sabre\File->put(null)
2. /snap/nextcloud/22400/htdocs/apps/dav/lib/Upload/UploadFolder.php line 46
OCA\DAV\Connector\Sabre\Directory->createFile("00000014", null)
3. /snap/nextcloud/22400/htdocs/3rdparty/sabre/dav/lib/DAV/Server.php line 1104
OCA\DAV\Upload\UploadFolder->createFile("00000014", null)
4. /snap/nextcloud/22400/htdocs/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 527
Sabre\DAV\Server->createFile("uploads/***/3015040066/00000014", null, null)
5. /snap/nextcloud/22400/htdocs/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
Sabre\DAV\CorePlugin->httpPut(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
6. /snap/nextcloud/22400/htdocs/3rdparty/sabre/dav/lib/DAV/Server.php line 474
Sabre\DAV\Server->emit("method:PUT", [Sabre\HTTP\Requ ... }])
7. /snap/nextcloud/22400/htdocs/3rdparty/sabre/dav/lib/DAV/Server.php line 251
Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
8. /snap/nextcloud/22400/htdocs/3rdparty/sabre/dav/lib/DAV/Server.php line 319
Sabre\DAV\Server->start()
9. /snap/nextcloud/22400/htdocs/apps/dav/lib/Server.php line 320
Sabre\DAV\Server->exec()
10. /snap/nextcloud/22400/htdocs/apps/dav/appinfo/v2/remote.php line 35
OCA\DAV\Server->exec()
11. /snap/nextcloud/22400/htdocs/remote.php line 167
require_once("/snap/nextcloud ... p")
PUT /remote.php/dav/uploads/***/3015040066/00000014
from *** by *** at 2020-08-05T17:34:27+00:00
The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):
<?php
$CONFIG = array (
'apps_paths' =>
array (
0 =>
array (
'path' => '/snap/nextcloud/current/htdocs/apps',
'url' => '/apps',
'writable' => false,
),
1 =>
array (
'path' => '/var/snap/nextcloud/current/nextcloud/extra-apps',
'url' => '/extra-apps',
'writable' => true,
),
),
'supportedDatabases' =>
array (
0 => 'mysql',
),
'memcache.locking' => '\\OC\\Memcache\\Redis',
'memcache.local' => '\\OC\\Memcache\\Redis',
'redis' =>
array (
'host' => '/tmp/sockets/redis.sock',
'port' => 0,
),
'objectstore' =>
array (
'class' => '\\OC\\Files\\ObjectStore\\S3',
'arguments' =>
array (
'bucket' => '***',
'region' => 'eu-west-2',
'autocreate' => false,
'key' => '***',
'secret' => '***',
'use_ssl' => true,
),
),
'passwordsalt' => '***',
'secret' => '***',
'trusted_domains' =>
array (
0 => 'localhost',
1 => '***',
),
'datadirectory' => '/var/snap/nextcloud/common/nextcloud/data',
'dbtype' => 'mysql',
'version' => '19.0.1.1',
'overwrite.cli.url' => 'http://localhost',
'dbname' => 'nextcloud',
'dbhost' => 'localhost:/tmp/sockets/mysql.sock',
'dbport' => '',
'dbtableprefix' => 'oc_',
'mysql.utf8mb4' => true,
'dbuser' => 'nextcloud',
'dbpassword' => '***',
'installed' => true,
'instanceid' => 'oc939xnfnvvp',
'twofactor_enforced' => 'true',
'twofactor_enforced_groups' =>
array (
),
'twofactor_enforced_excluded_groups' =>
array (
),
);