Missing Folders after running files:scan --all

Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version : 18.0.2
Operating system and version (eg, Ubuntu 20.04): Ubuntu Server 18.04LTS
Apache or nginx version (eg, Apache 2.4.25):
PHP version (eg, 7.1): 7.4

The issue you are facing: After running a files:scan, there are multiple folders missing under our shared group folders (40GB or so).

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

  1. Log in
  2. Try to access data
  3. Data only shows 25GB when it should be 65GB

The output of your Nextcloud log in Admin > Logging: There are no instances of DELETE or MOVE that would account for this loss.

PASTE HERE

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'oczcsoczpog8',
  'passwordsalt' => 'eHDoTO5Nxyyuf1ho87p2E3nxkGsZq/',
  'secret' => 'yCrjXVn+PjucExeag/zu/RVtC1dMxF4V6HOBakpHXtGhu2Ek',
  'trusted_domains' => 
  array (
    0 => 'nextcloud',
  ),
  'datadirectory' => '/dev/sdb1/nextcloudfiles/',
  'dbtype' => 'mysql',
  'version' => '18.0.2.2',
  'overwrite.cli.url' => 'https://',
  'dbname' => 'nextcloud_db',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'ncadmin',
  'dbpassword' => '',
  'installed' => true,
  'theme' => '',
  'loglevel' => 2,
  'maintenance' => false,
  'ldapProviderFactory' => 'OCA\\User_LDAP\\LDAPProviderFactory',
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'tls',
  'mail_sendmailmode' => 'smtp',
  'mail_from_address' => 'nextcloud',
  'mail_domain' => '.com',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'smtp.office365.com',
  'mail_smtpport' => '587',
  'mail_smtpname' => 'nextcloud.com',
  'mail_smtppassword' => '',
);

The output of your Apache/nginx/system log in /var/log/____:

PASTE HERE

Is this a mount point or device?

Post
mount | grep sdb1

Mount point. Everything was working wonderfully for the last two months until files:scan was run.

/dev/sdb on /dev/sdb1 type ext4 (rw,nosuid,nodev,relatime,stripe=256,data=ordered,x-gvfs-show)

Here is the only thing i’ve been able to track down:

73.229.151.160 - - [10/Jul/2020:12:33:47 -0600] “GET /apps/files_trashbin/preview?fileId=25274423&file=BoD%2FBOD+06-27-2019%2F~%24BoD+6-27-19+v1.xlsx%2F~%24BoD+6-27-19+v1.xlsx.d1593458543&c=1593458543 HTTP/2.0” 404 0 “-” “Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0”

There are a lot of these right around the time that we noticed things missing.

I think there is something missing in trashbin. Search your missing files in file system and post paths.

Here is the path to one of the folders that we lost.

admin:///dev/sdb1/nextcloudfiles/__groupfolders/trash/3/BoD.d1594059033

I think the files are in the trash. Undelete them. I think there is a folder “3” in your nextcloud-gui-trahbin.

Forgive me, but how do I undelete them? They are not in my “Deleted Files” section. Where should I be able to find them in the GUI?

Perhaps a problem with groups. Sorry i only tested groups a long time ago. Perhaps another user finds the files in “deleted files”.

Firstly, thank you so much for your support thus far. I appreciate it. This is a lot of sensitive data that just disappeared on us.

I’ve tried the three admin users that we have and none of them see the deleted files in the Deleted Files area. Do you have any other methods I might be able to try? Thanks again.

You can copy all files back in other folders and use “occ files:scan --all”. Or restore an old backup.

But files:scan is how they ended up in the trash to begin with.

No other options? That’s a lot of files with obtuse names.

Does anyone have idea why files:scan might have moved these files to the group folders trash and how I can go about getting them out?

Anyone? Seems like this is a rare event.

I’m just shocked that

  1. There is no clear method to recover items from the groupfolder trash
  2. There is no clear understanding how files:scan can move 45GB of data to the trashbin.

It just occurs to me that occ does all its stuff through the database, and a database should have a transactional log of sorts.

I’m not good with databases, but I think it should be possible to see from the database what was done, and use that log to revert the changes. E.g. if all files moved to trash are named in a log of sorts, then that log could be parsed and used in a script to revert the changes?

1 Like

Excellent idea, let me give that a go. Thank you so much for taking the time to respond!