Can't delete folder on local external storage

I host a Nextcloud server (11.0.1) on my Raspberry Pi at home with an added local external storage. I have trouble debugging the following behaviour. I am:

  1. Unable to delete folders in the external storage directory;
  2. Able to delete files in the external storage directory;
  3. Able to delete files and folders in my main user directory;
  4. Able to create files and folders in my main user directory and in the external storage directory.

When deleting a folder (empty or has content) from (any subdirectory of) the external storage through the web interface or iOS app, the loading takes a long time before: “Error deleting file: connection to server lost.” While loading, the server shows a large CPU usage by mysql, and repeated attempts have resulted in crashing of mysql, crashing of the whole system (needing restart), or just nothing. The subsequent Nextcloud server log shows nothing related to the failure to delete.

The data structure is as follows.

  • External hard disk mounted on /home/pi/Documents/WD_2TB/ with ownership pi:pi and permission 755 (pi is the default non-root user)
  • Nextcloud installed in /var/www/nextcloud/
  • Nextcloud data directory at /home/pi/Documents/WD_2TB/nextcloud/data/ with ownership www-data:www-data and permission 755
  • External storage linked from Nextcloud: /home/pi/Documents/WD_2TB/Storage/ with ownership www-data:www-data and permission 755

Some more info:

  • Nextcloud 11.0.1 installed on Raspberry Pi 3 Model B with 1.2GHz 64-bit quad-core ARMv8 CPU and 1GB RAM
  • PHP: Version 5.6.29, Memory Limit 512MB, Max Execution Time 3600
  • MySQL Database: Version 10.0.28, Size 303.4MB

I would appreciate some guidance on how to solve this problem.

Did you have a look at your other logfiles (webserver, database, php)? You say that you can’t delete a folder on external storage, can you delete files on your external storage?

Files (non-directories) can be deleted normally in the external storage. This problem affects only directories and subdirectories in the external storage. Directories can also be created without a problem in the external storage.

Last time I tried to delete a folder, mysql had huge CPU usage for 20 minutes, so apache2 log shows “Failed to connect to the database”. Mysql error log is empty during this time. Nextcloud log shows File deleted: “/Archive/Test2” three times, with errors in between because mysql server “has gone away”. Mysql general log has the following relevant part, which repeated three times during the 20 mins:

170123 23:08:37    59 Connect   oc_winstonyin@localhost as anonymous on nextcloud
           59 Query SET NAMES UTF8
           59 Query SET SESSION AUTOCOMMIT=1
           59 Query SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED
           59 Query SELECT * FROM `oc_appconfig`
           59 Query SELECT `uid`, `displayname` FROM `oc_users` WHERE LOWER(`uid`) = LOWER('winstonyin')
           59 Query SELECT `appid`, `configkey`, `configvalue` FROM `oc_preferences` WHERE `userid` = 'winstonyin'
           59 Query SELECT `id`, `uid`, `login_name`, `password`, `name`, `type`, `remember`, `token`, `last_activity`, `last_check`, `scope` FROM `oc_authtoken` WHERE `token` = '54f2104fd611b7b6e30e905326423f6287325035b95915fd67c655bd5b92c7ad55c2a1618ba3e93734b6685b1b48d3b5d96850118af534f9912e3eb757d2e385'
           59 Query SELECT m.`mount_id`, `mount_point`, `storage_backend`, `auth_backend`, `priority`, m.`type` FROM `oc_external_mounts` m INNER JOIN `oc_external_applicable` a ON m.`mount_id` = a.`mount_id` WHERE (a.`type` = 3) AND (a.`value` = 'winstonyin') AND (m.`type` = '1')
           59 Query SELECT m.`mount_id`, `mount_point`, `storage_backend`, `auth_backend`, `priority`, m.`type` FROM `oc_external_mounts` m INNER JOIN `oc_external_applicable` a ON m.`mount_id` = a.`mount_id` WHERE (a.`type` = 1) AND (a.`value` IS NULL) AND (m.`type` = '1')
           59 Query SELECT `gid` FROM `oc_group_user` WHERE `uid` = 'winstonyin'
           59 Query SELECT m.`mount_id`, `mount_point`, `storage_backend`, `auth_backend`, `priority`, m.`type` FROM `oc_external_mounts` m INNER JOIN `oc_external_applicable` a ON m.`mount_id` = a.`mount_id` WHERE (a.`type` = 2) AND (a.`value` IN ('admin', 'family')) AND (m.`type` = '1')
           59 Query SELECT `type`, `value`, `mount_id` FROM `oc_external_applicable` WHERE `mount_id` IN ('2')
           59 Query SELECT `key`, `value`, `mount_id` FROM `oc_external_config` WHERE `mount_id` IN ('2')
           59 Query SELECT `key`, `value`, `mount_id` FROM `oc_external_options` WHERE `mount_id` IN ('2')
           59 Query SELECT `id`, `numeric_id`, `available`, `last_checked` FROM `oc_storages` WHERE `id` IN ('local::/home/pi/Documents/WD_2TB/nextcloud/data/winstonyin/')
           59 Query SELECT m.`mount_id`, `mount_point`, `storage_backend`, `auth_backend`, `priority`, m.`type` FROM `oc_external_mounts` m INNER JOIN `oc_external_applicable` a ON m.`mount_id` = a.`mount_id` WHERE ((a.`type` = 1) AND (a.`value` IS NULL)) OR ((a.`type` = 3) AND (a.`value` = 'winstonyin')) OR ((a.`type` = 2) AND (a.`value` IN ('admin', 'family')))
           59 Query SELECT `type`, `value`, `mount_id` FROM `oc_external_applicable` WHERE `mount_id` IN ('2')
           59 Query SELECT `key`, `value`, `mount_id` FROM `oc_external_config` WHERE `mount_id` IN ('2')
           59 Query SELECT `key`, `value`, `mount_id` FROM `oc_external_options` WHERE `mount_id` IN ('2')
           59 Query SELECT `id`, `numeric_id`, `available`, `last_checked` FROM `oc_storages` WHERE `id` IN ('local::/home/pi/Documents/WD_2TB/Storage/')
           59 Query SELECT s.*, f.`fileid`, f.`path`, f.`permissions` as `f_permissions`, f.`storage`, f.`path_hash`, f.`parent` as `f_parent`, f.`name`, f.`mimetype`, f.`mimepart`, f.`size`, f.`mtime`, f.`storage_mtime`, f.`encrypted`, f.`unencrypted_size`, f.`etag`, f.`checksum`, st.`id` AS `storage_string_id` FROM `oc_share` s LEFT JOIN `oc_filecache` f ON s.`file_source` = f.`fileid` LEFT JOIN `oc_storages` st ON f.`storage` = st.`numeric_id` WHERE (`share_type` = '0') AND (`share_with` = 'winstonyin') AND ((`item_type` = 'file') OR (`item_type` = 'folder')) ORDER BY s.`id` ASC LIMIT 18446744073709551615 OFFSET 0
           59 Query SELECT s.*, f.`fileid`, f.`path`, f.`permissions` as `f_permissions`, f.`storage`, f.`path_hash`, f.`parent` as `f_parent`, f.`name`, f.`mimetype`, f.`mimepart`, f.`size`, f.`mtime`, f.`storage_mtime`, f.`encrypted`, f.`unencrypted_size`, f.`etag`, f.`checksum`, st.`id` AS `storage_string_id` FROM `oc_share` s LEFT JOIN `oc_filecache` f ON s.`file_source` = f.`fileid` LEFT JOIN `oc_storages` st ON f.`storage` = st.`numeric_id` WHERE (`share_type` = '1') AND (`share_with` IN ('admin', 'family')) AND ((`item_type` = 'file') OR (`item_type` = 'folder')) ORDER BY s.`id` ASC LIMIT 18446744073709551615 OFFSET 0
           59 Query SELECT `id`, `mimetype` FROM `oc_mimetypes`
           59 Query SELECT * FROM `oc_share` WHERE (`parent` IN (9)) AND (`share_with` = 'winstonyin') AND ((`item_type` = 'file') OR (`item_type` = 'folder'))
           59 Query SELECT `remote`, `share_token`, `password`, `mountpoint`, `owner`
                FROM `oc_share_external`
                WHERE `user` = 'winstonyin' AND `accepted` = '1'
           59 Query SELECT `id`, `numeric_id`, `available`, `last_checked` FROM `oc_storages` WHERE `id` IN ('home::winstonyin')
           59 Query SELECT `fileid` FROM `oc_filecache` WHERE `storage` = '9' AND `path_hash` = 'd41d8cd98f00b204e9800998ecf8427e'
           59 Query SELECT `fileid` FROM `oc_filecache` WHERE `storage` = '1' AND `path_hash` = 'd41d8cd98f00b204e9800998ecf8427e'
           59 Query SELECT `storage_id`, `root_id`, `user_id`, `mount_point`, `mount_id`, f.`path` FROM `oc_mounts` m INNER JOIN `oc_filecache` f ON m.`root_id` = f.`fileid` WHERE `user_id` = 'winstonyin'
           59 Query UPDATE `oc_mounts` SET `storage_id` = '9', `mount_point` = '/winstonyin/files/Archive/', `mount_id` = 2 WHERE (`user_id` = 'winstonyin') AND (`root_id` = 58598)
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                       `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                FROM `oc_filecache` WHERE `storage` = '1' AND `path_hash` = '45b963397aa40d4a0063e0d85e4fe7a1'
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                       `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                FROM `oc_filecache` WHERE `storage` = '9' AND `path_hash` = 'c454552d52d55d3ef56408742887362b'
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                       `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                FROM `oc_filecache` WHERE `storage` = '9' AND `path_hash` = 'c454552d52d55d3ef56408742887362b'
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                       `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                FROM `oc_filecache` WHERE `storage` = '9' AND `path_hash` = 'c454552d52d55d3ef56408742887362b'
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                           `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                    FROM `oc_filecache` WHERE `parent` = '668916' ORDER BY `name` ASC
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                       `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                FROM `oc_filecache` WHERE `storage` = '9' AND `path_hash` = 'c454552d52d55d3ef56408742887362b'
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                       `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                FROM `oc_filecache` WHERE `storage` = '9' AND `path_hash` = 'c454552d52d55d3ef56408742887362b'
           59 Query SELECT `path` FROM `oc_filecache` WHERE `fileid` = '668916' AND `storage` = '1'
           59 Query SELECT `path` FROM `oc_filecache` WHERE `fileid` = '668916' AND `storage` = '9'
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                       `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                FROM `oc_filecache` WHERE `storage` = '9' AND `path_hash` = 'c454552d52d55d3ef56408742887362b'
           59 Query SELECT `id`, `numeric_id`, `available`, `last_checked` FROM `oc_storages` WHERE `id` IN ('9')
           59 Query SELECT `share_with`, `file_source`, `file_target`
                FROM
                `oc_share`
                WHERE
                `item_source` = '668916' AND `share_type` IN ('0', '2') AND `item_type` IN ('file', 'folder')
           59 Query SELECT `share_with`, `file_source`, `file_target`
                FROM
                `oc_share`
                WHERE
                `item_source` = '668916' AND `share_type` = '1' AND `item_type` IN ('file', 'folder')
           59 Query SELECT `share_with`
                    FROM `oc_share`
                    WHERE `item_source` = '668916' AND `share_type` = '3' AND `item_type` IN ('file', 'folder') LIMIT 1
           59 Query SELECT `share_with`
                    FROM `oc_share`
                    WHERE `item_source` = '668916' AND `share_type` = '6' AND `item_type` IN ('file', 'folder') LIMIT 1
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                       `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                FROM `oc_filecache` WHERE `fileid` = '668916'
           59 Query SELECT `share_with`, `file_source`, `file_target`
                FROM
                `oc_share`
                WHERE
                `item_source` = '58598' AND `share_type` IN ('0', '2') AND `item_type` IN ('file', 'folder')
           59 Query SELECT `share_with`, `file_source`, `file_target`
                FROM
                `oc_share`
                WHERE
                `item_source` = '58598' AND `share_type` = '1' AND `item_type` IN ('file', 'folder')
           59 Query SELECT `share_with`
                    FROM `oc_share`
                    WHERE `item_source` = '58598' AND `share_type` = '3' AND `item_type` IN ('file', 'folder') LIMIT 1
           59 Query SELECT `share_with`
                    FROM `oc_share`
                    WHERE `item_source` = '58598' AND `share_type` = '6' AND `item_type` IN ('file', 'folder') LIMIT 1
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                       `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                FROM `oc_filecache` WHERE `fileid` = '58598'
           59 Query SELECT `userid`, `configvalue` FROM `oc_preferences` WHERE `appid` = 'activity' AND `configkey` = 'notify_stream_file_deleted' AND `userid` IN ('winstonyin')
           59 Query SELECT `userid`, `configvalue` FROM `oc_preferences` WHERE `appid` = 'activity' AND `configkey` = 'notify_email_file_deleted' AND `userid` IN ('winstonyin')
           59 Query INSERT INTO `oc_activity` (`app`, `subject`, `subjectparams`, `message`, `messageparams`, `file`, `link`, `user`, `affecteduser`, `timestamp`, `priority`, `type`, `object_type`, `object_id`) VALUES('files', 'deleted_self', '[{\"668916\":\"\\/Archive\\/Test2\"}]', '', '[]', '/Archive/Test2', 'https://nextcloud.winstonyin.com/index.php/apps/files/?dir=/Archive', 'winstonyin', 'winstonyin', '1485212917', '30', 'file_deleted', 'files', '668916')
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                       `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                FROM `oc_filecache` WHERE `storage` = '9' AND `path_hash` = 'c454552d52d55d3ef56408742887362b'
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                       `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                FROM `oc_filecache` WHERE `storage` = '9' AND `path_hash` = 'c454552d52d55d3ef56408742887362b'
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`,
                       `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`
                FROM `oc_filecache` WHERE `storage` = '9' AND `path_hash` = 'c454552d52d55d3ef56408742887362b'
           59 Query SELECT `fileid`, `storage`, `path`, `parent`, `name`,
                `mimetype`, `mimepart`, `size`, `mtime`, `encrypted`,
                `etag`, `permissions`, `checksum`
            FROM `oc_filecache`
            WHERE `storage` = '9' AND `name` COLLATE UTF8_general_ci LIKE '%%'

Can you delete a folder on the external storage via terminal? You could as well try to reindex all your files & folders with occ files:scan (https://docs.nextcloud.com/server/11/admin_manual/configuration_server/occ_command.html#file-operations).

I Have a ditto problem where I cannot delete folders[error deleting files] from external storage. No problem making new folders and deleting from local drive… No problem with files. Can make new folders but cant delete. Can OP tell us if he found a solution to this problem. Have checked all permissions and seems all good. It only started after upgrading to 11.0.0 and now 11.0.1. Anyone pls help.

This is possible and leads to no problem, but is not my intended workflow. The rescanning of external file system doesn’t cause particular stress to the machine, but somehow the deletion from a client app causes a lot of swap, either through mysql or PHP. Would somebody please look at the log files to see if there is any unreasonable mysql calls?

Glad (and sorry) it’s not just me. I haven’t yet found a solution. I’m not sure if the problem existed before upgrading to NC11. When you attempt to delete a folder on your external drive, does your server become unresponsive for a while? Does your machine start using memory swap? For me this is the case, and during the 20 or so minutes, neither apache nor ssh service is responsive.

I have same problem.

Nextcloud 11.0.1 installed on Raspberry Pi 3 Model B (migrated from OwncCloud, OwnCloud had not similar problem)
PHP Version 7.0.15
MySQL 5.5.54

External storage filesystem: ext4
External storage mountpoint: /var/html/www/html
NextCloud directory: /var/www/html/cloud
NextCloud storage path: /var/www/html/STORAGE/CLOUD/data

Log: Allowed memory size of 536870912 bytes exhausted (tried to allocate 4096 bytes) at /var/www/html/cloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php#104

I can delete folders via terminal or FTP. I tried occ files:scan, but no result.

I found fix that works for me.

3 Likes

The fix worked for me, too. You’re a saviour! Note that the symptoms (log files, etc) are not exactly the same for us. I did not see the “memory exhausted” message.

-SOLVED-

Trick works

Thanks Guys - You are life savers


update: seems like 11.0.2 doesn’t have this issue

bug report to get it fixed:

Hello, i’m either trying to use an hard disk for external storage with nextcloud. The hard disk is automatically mounted on /media/pi, but i can’t access to the folders & files through the nextcloud “external storage” menu.
I configured the external storage by choosing “local” & "media/pi/SAMSUNG. The button on the left is turning to green, but when i click on “external storage”, i’m going back to the main menu…
any idea?
thanks,

Eric

You may want to check the permissions of all files and directories in the mounted external storage. They have to be read/write enabled for the www-data user (or whichever www user you’ve got). You may even need to set the parent directory of the mount point to be accessible to www-data. However, I think your concern is more appropriate for a new thread in the forum, where more experienced users may offer some help.