Unable to remove deleted files with strange characters

ℹ️ Support
1
Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version (eg, 20.0.5): 27.1.3
Operating system and version (eg, Ubuntu 20.04): Centos v7
Apache or nginx version (eg, Apache 2.4.25): nginx 1.20.1
PHP version (eg, 7.4): 8.1.24

The issue you are facing: unable to remove files from trash that contain unusual character sequence.

Is this the first time you’ve seen this error? (Y/N): Yes

Steps to replicate it:

  1. Open deleted files
  2. Try and remove the files

image
image944×463 16.4 KB

I’ve found the files in the file system and removed them but they still appear in the Nextcloud web interface. I’ve tried occ file:scan and occ groupfolder:scan but no difference.

I only spotted occ files:delete after doing the above. Do these scan operations work on trash and revisions?

2

Did I understand that correctly: You deleted the file at file system level, so the file is no longer on the hard drive?

Your screenshot shows the trashbin folder. If those files do not exists, those are zombie entries.
Those zombie entries normaly disapear as soon as you do a “Restore”. Only an error message in logfile “unable to rename, file does not exists … (path/to/trashbin/filename)” appears but it restores the file-entry in the webview. After it is restored, it is easy to remove that entry (without underlying file in the filesystem) by runing
occ files:scan <user_id>
or
occ groupfolders:scan <groupfolder_id>

btw:

it must be
occ files:scan
and
occ groupfolders:scan
(you missed the “s” before the colon)

If my suggestion doesn’t help, then the file should perhaps be renamed.

  • &#039;

is the html entity of a single apostrophe: (')

Is it possible that the file did not get into the file system via the web frontend but was added directly at the file system level? Then it’s possible that it wasn’t properly UTF8 encoded. Or there was an encoding problem with the name on the client that added it?

I hope you get it solved, if not, don’t hesitate to come back here.

Much luck,
ernolf

3

This went on the backburner but the ticket escalated so looking at it again. The trashed files don’t exist - I found them in file system and deleted them on the assumption that Nextcloud would refresh it’s tables at some point.

The idea of restoring them doesn’t work - get the error “Restore action failed” and they stay in the list of deleted files.

I’ve run the two scan commands but no difference. occ files:scan --all did remove some files but groupfolders:scan --all just seems to list the groups, folders and file counts but not much else.

I assume these entries are in the database somewhere? Quite happy to start digging into there - in a test environment first though :slight_smile:

Later…

Spotted groupfolders:trashbin:cleanup - might give that a go… bit drastic but might help.

4

Where did you delete those files in the file system exactly?

Are those files from within a group folder?

If yes, then you could run

occ groupfolders:trashbin:cleanup <groupfolder_id>

not somewhere, they are stored in oc_group_folders_trash

MariaDB [nextcloud]> SELECT * FROM oc_group_folders_trash;

will list all the entries in that table.

Much luck,
ernolf

5

Where did you delete those files in the file system exactly?

I logged onto the Linux console and used find to locate them. I first tried renaming them in the file system (which I was able to do) but they didn’t rename in Nextcloud. Next I deleted them using rm :slight_smile:

Are those files from within a group folder?

Annoyingly I didn’t make a note of the folders but I think so…

occ groupfolders:trashbin:cleanup <groupfolder_id>

I’ll find out where they are exactly via the database tables and then have a go with this!

6

A quick SELECT reveals the four bad files in that table. Just backing up the entire VM before I turn it into a DELETE command…

Later…

Now this is weird. Run the delete command:

MariaDB [nextcloud]> DELETE From oc_group_folders_trash WHERE original_location like ‘%BBC%’;
Query OK, 4 rows affected (0.003 sec)

So four rows deleted as expected. Except they’re still visible in deleted items! Maybe they’re also somewhere else…

7

So in what (folder_id) groupfolder trash bin they are located?

Then you should take a look in that folder first, before starting to DELETE entries from the database:

__groupfolders/trash/<folder_id>/<name>.d<deleted_time>

(<folder_id>, <name> and <deleted_time> are as in the database table)

Yes, they are!
In your first query (SELECT * FROM oc_group_folders_trash;), you got not only folder_id, name and deleted_time but (beside others) file_id as well.

Now simply query

SELECT path FROM oc_filecache WHERE fileid = <file_id>;

you must replace <file_id> with the value obtained with the previous query of courese)

You will see, that the file is still stored in the filecache as located in your trashbin. So that entry must be removed as well or can be removed by

occ groupfolders:scan <folder_id>

Much luck,
ernolf

8

Ah ha thanks - I recorded the output from the SELECT command so I’ve got the fileid as well.

Later… they’ve finally gone! :slight_smile: Thanks for your help.

9

Is there no occ command that scans through all the files, group folders, versions and trash tables and deletes from the database if the target file doesn’t exist?

Aside from all the above, there is a bug in Nextcloud that it’s possible to upload files to it with weird file names somehow that the web interface is then unable to handle. I’ll put money on it being the HTML character codes:

BBC News - Surge in interest in &#039;wonder-material&#039; Graphene.flv

How many websites have you come across where “&” in a text field doesn’t render correctly :wink: