SMB - Lots of missing files

Version: 25.0.0 RC5

I have used SMB via External Storage for well over a year without issue. I recently did a lot of sorting and so moved things around and deleted the share and remounted via External Storage tab.

The share is visible, and certain folders are there, but some show as empty, there are lots and lots of missing files, maybe 40GB worth.

I’ve see occ files:scan and various options for files_external: such as notify, but it seems whatever I try nextcloud refuses to pick up on these files.

This is getting pretty urgent as I use this daily and currently don’t have access to any of my files, any ideas on what to try?

Copy paste this into your post and please fill it out.

Support intro

Nextcloud version (eg, 20.0.5): 25 RC5
Operating system and version (eg, Ubuntu 20.04): Ubuntu 22.04
Apache or nginx version (eg, Apache 2.4.25): 2.4.25
PHP version (eg, 7.4): 8.1

The issue you are facing: above

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it: above

Hi @shaung,
You are missing the required support template. Please fill this form out and edit into your post.

This will give us the technical info and logs needed to better help you! Thanks.

Can you not roll back to a backup? Restore your Nextcloud on a stable build you used before upgrading.

Why are you upgrading to an experimental rc on a production server (more of a rhetorical question, because that is never a good idea fwiw).

Try to scan a dedicated path and increase the verbosity of the scan by e.g. adding “-vvv”. Sometimes a path is not correctly scanned because of wrong access rights or specific characters in file names etc.

@j-ed Yea I was trying that flag but where the hell is the SMB path? I have no idea where OC mounts this.

Please fill out the support form linked above.

Nothing in the template apart from OS is relevant to this issue. I will fill it out but want it to be known that you’re posts are just cluttering the thread and anyone who may reply like @j-ed didn’t need that info.

@j-ed Seems like --path flag doesn’t actually get taken correctly?

user@nextcloud:/mnt/nextcloud$ sudo -u www-data php /var/www/occ files:scan --path=/SMB/ -vvv

Unknown user 1 SMB

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 0       | 0     | 00:00:00     |
+---------+-------+--------------+

Please use “occ files:scan --help” to display all available command line options. You have to provide a user id or prefix the path with the user name.

@j-ed Thanks.

For a folder with files (showing in nextcloud UI), the below command results in files being scanned.

For this folder and many others showing 0kB in nextcloud UI, the result is below, it’s not seeing any files.

Before even posting here I have checked that permissions are exactly the same across all files and folders under this share, so I have no idea what’s happening here…

user@nextcloud:/mnt/nextcloud$ sudo -u www-data php /var/www/occ files:scan --path=/user/files/SMB/Documents/Photos -vvv
Starting scan for user 1 out of 1 (user)
	Folder	/user/files/SMB/Documents/Photos

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 1       | 0     | 00:00:00     |
+---------+-------+--------------+
user@nextcloud:/mnt/nextcloud$ 

Could anyone figure out how to solve this?
I have the same problem and couldn’t find a solution.

If needed, I can give more information about my installation.

I have the same problem several times. Just never never never use file:scan command. It will stop tracking some of the folders in the smb external storage. I notice there is a smb fseek() error, which, I guess, jeopardize the external storage crawl process. And it may or may not be related to some corrupted files or directories. But the only way I can get it to work again, is to delete the external storage and re-add it.
1.Delete the external storage from settings.
2.Stop the web server and cron.
3.Delete every thing in /nextcloud/data/app_xxxxxxxx/preview
4.Run occ maintenance:repair
5.Run occ file:scan-app-data
6.Run occ files:cleanup
7.Run occ files:repair-tree
****6&7 are optional.
And wait about 10min or longer all files will be listed. And be sure not to use file:scan again.

I get these errors in the log after I delete and add my SMB storage. But the folder mentioned in the Error log still shows up in the nextcloud and I can successfully download files from it. But if I update some file info it may or may not get updated in nextcoud photo app depending on the folder the file is located.

[files] Warning: User eli still has unscanned files after running background scan, background scan might be stopped prematurely

at 2023-01-28T21:06:20+00:00
[files] Error: Icewind\SMB\Exception\Exception: Malformed state response from server at <<closure>>

 0. /var/www/html/apps/files_external/3rdparty/icewind/smb/src/Wrapped/Share.php line 217
    Icewind\SMB\Wrapped\Parser->parseStat([])
 1. /var/www/html/apps/files_external/lib/Lib/Storage/SMB.php line 191
    Icewind\SMB\Wrapped\Share->stat("/nextcloud/PhonePic/Cam Video/志愿者")
 2. /var/www/html/apps/files_external/lib/Lib/Storage/SMB.php line 567
    OCA\Files_External\Lib\Storage\SMB->getFileInfo("/nextcloud/PhonePic/Cam Video/志愿者")
 3. /var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php line 599
    OCA\Files_External\Lib\Storage\SMB->getMetaData("PhonePic/Cam Video/志愿者")
 4. /var/www/html/lib/private/Files/Storage/Wrapper/Availability.php line 447
    OC\Files\Storage\Wrapper\Wrapper->getMetaData("PhonePic/Cam Video/志愿者")
 5. /var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php line 599
    OC\Files\Storage\Wrapper\Availability->getMetaData("PhonePic/Cam Video/志愿者")
 6. /var/www/html/lib/private/Files/Cache/Scanner.php line 116
    OC\Files\Storage\Wrapper\Wrapper->getMetaData("PhonePic/Cam Video/志愿者")
 7. /var/www/html/lib/private/Files/Cache/Scanner.php line 154
    OC\Files\Cache\Scanner->getData("PhonePic/Cam Video/志愿者")
 8. /var/www/html/lib/private/Files/Cache/Scanner.php line 340
    OC\Files\Cache\Scanner->scanFile("PhonePic/Cam Video/志愿者", 3, -1, null, true)
 9. /var/www/html/lib/private/Files/Cache/Scanner.php line 538
    OC\Files\Cache\Scanner->scan("PhonePic/Cam Video/志愿者", 2, 3)
10. /var/www/html/lib/private/Files/Cache/Scanner.php line 550
    OC\Files\Cache\Scanner->OC\Files\Cache\{closure}("*** sensitive parameters replaced ***")
11. /var/www/html/lib/private/Files/Cache/Scanner.php line 537
    OC\Files\Cache\Scanner->runBackgroundScanJob(Closure {}, "PhonePic/Cam Video/志愿者")
12. /var/www/html/lib/private/Files/Utils/Scanner.php line 182
    OC\Files\Cache\Scanner->backgroundScan()
13. /var/www/html/apps/files/lib/BackgroundJob/ScanFiles.php line 76
    OC\Files\Utils\Scanner->backgroundScan("")
14. /var/www/html/apps/files/lib/BackgroundJob/ScanFiles.php line 113
    OCA\Files\BackgroundJob\ScanFiles->runScanner("eli")
15. /var/www/html/lib/public/BackgroundJob/Job.php line 78
    OCA\Files\BackgroundJob\ScanFiles->run(null)
16. /var/www/html/lib/public/BackgroundJob/TimedJob.php line 103
    OCP\BackgroundJob\Job->start(OC\BackgroundJob\JobList {})
17. /var/www/html/lib/public/BackgroundJob/TimedJob.php line 93
    OCP\BackgroundJob\TimedJob->start(OC\BackgroundJob\JobList {})
18. /var/www/html/cron.php line 152
    OCP\BackgroundJob\TimedJob->execute(OC\BackgroundJob\JobList {}, OC\Log {})

at 2023-01-28T21:06:20+00:00