Synology Cloud Sync Issue with Nextcloud

FreeMinded · September 5, 2017, 12:12pm

We are using Synology Diskstations with the Cloud Sync App (WebDAV client) as backup for our nextcloud instances. This has been working fine for several instances while using unidirectional sync from nextcloud to Synology only.

Now, in one case, we are using a bidirectional sync and the Synology Cloud Sync throws the error “Your public cloud quota limit reached”.

After a a while the error goes away a the Cloud Sync says “Synchronisation halted” (not sure about the translation)

Manually restarting syncs a few files and then the error apears again.

We have no quotas neither on the Synology or on the Nextcloud!

There is a thread in the Synology forum: Cloud Sync: Your public cloud quota limit reached
Unfortunately no solution

Has anyone seen and solved this before? Any help is appreciated.

tflidd · September 5, 2017, 10:17pm

Hard to say. Because it is not only Nextcloud, it’s more likely a Synology problem (not authenticating again, misinterpreting errors, …). What you can do with Nextcloud is to see what URL the Synology is requesting and if it gets an error back (not authenticated, not found, …). Worst case you dump the traffic and see the whole communication.

On Synology side, you perhaps have also more logging options. What quota is shown in your Nextcloud? If you haven’t set a quota, there is still the hard drive limit.

FreeMinded · September 6, 2017, 5:27am

I can see the following error in the nextcloud log showing up with every failure

Fatal	webdav	Sabre\DAV\Exception\BadRequest: HTTP/1.1 400 expected filesize 69738518 got 8159232	2017-09-05 16:12:20
Fatal	webdav	Sabre\DAV\Exception\BadRequest: HTTP/1.1 400 expected filesize 39936405 got 2359296	2017-09-05 16:12:20
Fatal	webdav	Sabre\DAV\Exception\InsufficientStorage: HTTP/1.1 507 Insufficient Storage

Storage on the nextcloud side should not be an issue. We still have plenty.

On the Synology side I havn’t found a way to increase logging yet.

FreeMinded · September 6, 2017, 5:39am

The problem seems to come from a few files… don’t know yet why and which files exactly. Removing this folder from Sync makes it work again. But know I see the follwoing errors in Nextcloud

Fatal	webdav	Sabre\DAV\Exception\Conflict: HTTP/1.1 409 The destination node is not found	2017-09-06 07:31:36
Error	PHP	stream_get_contents() expects parameter 2 to be integer, string given at /nextcloud/3rdparty/sabre/http/lib/Message.php#81	2017-09-06 07:31:35
Error	PHP	stream_get_contents() expects parameter 2 to be integer, string given at /nextcloud/3rdparty/sabre/http/lib/Message.php#81

Any idea what this means?

tflidd · September 6, 2017, 6:38am

Can you tell us more about the system you are running Nextcloud? 32-bit php?

If you push files from your Synology, can you try with smaller amount of data if this is happening as well? Is this a local setup, or could you try to use a test server to upload?
There is a test server (Demo instances of 9, 10, 11 and 12 now live) however I’m not sure if there is enough time to reproduce this error until the server is reset. In this case, I could give you perhaps some credential of one of my test instances.

FreeMinded · September 6, 2017, 10:36am

@tflidd thanks for your reply. We are currently using the the Wonderfall Docker Image for version 11.0.4.
Test setup is not a problem, we can spin-up a instance on our infrastructure. But thanks for the offer.
I’ll post any findings or further questions here.

FreeMinded · September 6, 2017, 12:10pm

I found the “cause” of the problem. It seems to be an unzipped Joomla folder (Joomla_3.7.3-Stable-Full_Package).
Any idea why this refuses to sync?

Now we still have the following error persistent and showing up about 3 times per second.
Error PHP stream_get_contents() expects parameter 2 to be integer, string given at /nextcloud/3rdparty/sabre/http/lib/Message.php#81

Any idea on this?

tflidd · September 6, 2017, 2:10pm

I haven’t used such an image, so perhaps there is an issue with that.

Can you try to sync this folder with the linked test setup above?

FreeMinded · September 6, 2017, 3:18pm

@tflidd Sync is setup to your NC11. Sync problem just happend again…

Error log from your Nextcloud

Fatal	webdav	Sabre\DAV\Exception\InsufficientStorage: HTTP/1.1 507 Insufficient Storage	2017-09-06T17:15:37+0200
Fatal	webdav	Sabre\DAV\Exception\InsufficientStorage: HTTP/1.1 507 Insufficient Storage	2017-09-06T17:14:12+0200
Fatal	webdav	Sabre\DAV\Exception\InsufficientStorage: HTTP/1.1 507 Insufficient Storage	2017-09-06T17:13:38+0200
Fatal	webdav	Sabre\DAV\Exception\MethodNotAllowed: HTTP/1.1 405 The resource you tried to create already exists	2017-09-06T17:08:29+0200

tflidd · September 6, 2017, 3:27pm

A related issue I found:

github.com/nextcloud/server

Uploading a blacklisted files via WebUI results in no/not helpful error message

opened 04:07AM - 19 Apr 17 UTC

closed 09:06AM - 21 May 21 UTC

individual-it

bug 1. to develop feature: sharing papercut

### Steps to reproduce 1. blacklist some files in config.php e.g `'blacklisted_…files' => array('.htaccess','c.png'),` 2. upload a blacklisted file via normal WebUI 3. upload a blacklisted file via upload only share ### Expected behaviour - Error message saying that this file cannot be uploaded - the blacklisted files checked by JS before uploading. `fileIsBlacklisted(file)` seems not to do it correctly https://github.com/nextcloud/server/blob/master/core/js/js.js#L231 `oc_config.blacklist_files_regex` contains only `"\.(part|filepart)$"` - server should not reply `InsufficientStorage` but something more useful ### Actual behaviour 2. not enough free space error message 3. no message at all, just spinning wheel ### Server configuration **Operating system**: Debian 4.8.15-2~bpo8+2 **Web server:** Apache 2.0 **Database:** sqlite **PHP version:** 7.0.7-5 **Nextcloud version:** (see Nextcloud admin page) 12.0 alpha (git master) **Updated from an older Nextcloud/ownCloud or fresh install:** fresh install **Where did you install Nextcloud from:** git **List of activated apps:** <details> <summary>App list</summary> ``` Enabled: - comments: 1.2.0 - dav: 1.2.0 - federatedfilesharing: 1.2.0 - federation: 1.2.0 - files: 1.7.2 - files_sharing: 1.4.0 - files_trashbin: 1.2.0 - files_versions: 1.5.0 - lookup_server_connector: 1.0.0 - provisioning_api: 1.2.0 - sharebymail: 1.2.0 - systemtags: 1.2.0 - theming: 1.3.0 - twofactor_backupcodes: 1.1.0 - updatenotification: 1.2.0 - workflowengine: 1.2.0 Disabled: - admin_audit - contacts - encryption - files_external - testing - user_ldap ``` </details> **Nextcloud configuration:** <details> <summary>Config report</summary> ``` { "system": { "instanceid": "ocigom9x0hj5", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "localhost" ], "datadirectory": "\/home\/artur\/www\/nextcloud-server\/data", "overwrite.cli.url": "http:\/\/localhost\/nextcloud-server", "dbtype": "sqlite3", "version": "12.0.0.14", "installed": true, "blacklisted_files": [ ".htaccess", "c.png" ], "theme": "", "loglevel": 2, "maintenance": false } } ``` </details> **Are you using external storage, if yes which one:** local/smb/sftp/... no **Are you using encryption:** yes/no no **Are you using an external user-backend, if yes which one:** LDAP/ActiveDirectory/Webdav/... no </details> ### Client configuration **Browser:** FF 52.0.2 **Operating system:** Debian ### Logs #### Nextcloud log (data/nextcloud.log) <details> <summary>Nextcloud log</summary> a. normal WebUI: ``` {"reqId":"DqXAw2CrveyRjCOxTXZF","level":4,"time":"2017-04-19T03:15:37+00:00","remoteAddr":"::1","user":"admin","app":"webdav","method":"PUT","url":"\/nextcloud-server\/remote.php\/webdav\/c.png","message":"Exception: {\"Message\":\"HTTP\\\/1.1 507 Insufficient Storage\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\\\\InsufficientStorage\",\"Code\":0,\"Trace\":\"#0 [internal function]: OCA\\\\DAV\\\\Connector\\\\Sabre\\\\QuotaPlugin->checkQuota('\\\/c.png', Resource id #47, Object(OCA\\\\DAV\\\\Connector\\\\Sabre\\\\Directory), false)\\n#1 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#2 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1093): Sabre\\\\Event\\\\EventEmitter->emit('beforeCreateFil...', Array)\\n#3 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(525): Sabre\\\\DAV\\\\Server->createFile('c.png', Resource id #47, NULL)\\n#4 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPut(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#5 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#6 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(479): Sabre\\\\Event\\\\EventEmitter->emit('method:PUT', Array)\\n#7 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#8 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/webdav.php(63): Sabre\\\\DAV\\\\Server->exec()\\n#9 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/remote.php(162): require_once('\\\/home\\\/artur\\\/www...')\\n#10 {main}\",\"File\":\"\\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/QuotaPlugin.php\",\"Line\":113,\"User\":\"admin\"}","userAgent":"Mozilla\/5.0 (X11; Linux x86_64; rv:52.0) Gecko\/20100101 Firefox\/52.0","version":"12.0.0.14"} ``` b. upload only share: ``` {"reqId":"PUYPd2ifTspA5ecV5eOG","level":4,"time":"2017-04-19T03:16:18+00:00","remoteAddr":"::1","user":"--","app":"webdav","method":"PUT","url":"\/nextcloud-server\/public.php\/webdav\/c.png","message":"Exception: {\"Message\":\"HTTP\\\/1.1 507 Insufficient Storage\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\\\\InsufficientStorage\",\"Code\":0,\"Trace\":\"#0 [internal function]: OCA\\\\DAV\\\\Connector\\\\Sabre\\\\QuotaPlugin->checkQuota('\\\/c.png', Resource id #48, Object(OCA\\\\DAV\\\\Connector\\\\Sabre\\\\Directory), false)\\n#1 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#2 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1093): Sabre\\\\Event\\\\EventEmitter->emit('beforeCreateFil...', Array)\\n#3 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(525): Sabre\\\\DAV\\\\Server->createFile('c.png', Resource id #48, NULL)\\n#4 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPut(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#5 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#6 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(479): Sabre\\\\Event\\\\EventEmitter->emit('method:PUT', Array)\\n#7 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#8 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/publicwebdav.php(104): Sabre\\\\DAV\\\\Server->exec()\\n#9 \\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/public.php(76): require_once('\\\/home\\\/artur\\\/www...')\\n#10 {main}\",\"File\":\"\\\/home\\\/artur\\\/www\\\/nextcloud-server\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/QuotaPlugin.php\",\"Line\":113,\"User\":false}","userAgent":"Mozilla\/5.0 (X11; Linux x86_64; rv:52.0) Gecko\/20100101 Firefox\/52.0","version":"12.0.0.14"} ``` </details> #### Browser log <details> <summary>Browser log</summary> a. normal WebUI: ``` PUT XHR http://localhost/nextcloud-server/remote.php/webdav/c.png [HTTP/1.1 507 Insufficient Storage 128ms] Response: <?xml version="1.0" encoding="utf-8"?> <d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns"> <s:exception>Sabre\DAV\Exception\InsufficientStorage</s:exception> <s:message/> </d:error> GET XHR http://localhost/nextcloud-server/index.php/apps/files/ajax/getstoragestats.php [HTTP/1.1 200 OK 75ms] Response: {"data":{"uploadMaxFilesize":535822336,"maxHumanFilesize":"Upload (max. 511 MB)","freeSpace":116587892736,"usedSpacePercent":0,"owner":"admin","ownerDisplayName":"admin"},"status":"success"} ``` b. upload-only share: ``` PUT XHR http://localhost/nextcloud-server/public.php/webdav/c.png [HTTP/1.1 507 Insufficient Storage 171ms] Response: <?xml version="1.0" encoding="utf-8"?> <d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns"> <s:exception>Sabre\DAV\Exception\InsufficientStorage</s:exception> <s:message/> </d:error> ``` </details>

Does the folder contain a .htaccess file or *.part or *.filepart?

FreeMinded · September 7, 2017, 4:30pm

There is a htaccess.txt but not .htaccess. Couldn’t find any *.part or *.filepart files.

There is a .DS_Store file

FreeMinded · September 22, 2017, 7:06am

Update: I found the file causing the problem. It’s a .js file from the Joomla Package. So in case you want to reproduce this problem. Try uploading an unziped Joomla Package.