Occ files:scan crashes when external storage is not available

Support intro

Sorry to hear you’re facing problems. :slightly_frowning_face:

The community help forum (help.nextcloud.com) is for home and non-enterprise users. Support is provided by other community members on a best effort / “as available” basis. All of those responding are volunteering their time to help you.

If you’re using Nextcloud in a business/critical setting, paid and SLA-based support services can be accessed via portal.nextcloud.com where Nextcloud engineers can help ensure your business keeps running smoothly.

Getting help

In order to help you as efficiently (and quickly!) as possible, please fill in as much of the below requested information as you can.

Before clicking submit: Please check if your query is already addressed via the following resources:

(Utilizing these existing resources is typically faster. It also helps reduce the load on our generous volunteers while elevating the signal to noise ratio of the forums otherwise arising from the same queries being posted repeatedly).

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can. :heart:

The Basics

  • Nextcloud Server version (e.g., 29.x.x):
    • 32.0.3 AIO
  • Operating system and version (e.g., Ubuntu 24.04):
    • debian trixie
  • Web server and version (e.g, Apache 2.4.25):
    • ngxinx and caddy
  • Reverse proxy and version _(e.g. nginx 1.27.2)
    • replace me
  • PHP version (e.g, 8.3):
    • replace me
  • Is this the first time you’ve seen this error? (Yes / No):
    • no
  • When did this problem seem to first start?
    • i think long time ago
  • Installation method (e.g. AlO, NCP, Bare Metal/Archive, etc.)
    • AIO
  • Are you using CloudfIare, mod_security, or similar? (Yes / No)
    • no

Summary of the issue you are facing:

I configured 2 external storages. One is another nextcloud, and the other is via sftp configured in nc external storage.

The computer, which holds the filesystem mounted via sftp is most time offline. I turn this computer only on, when i need some data from there.

I put both external storages into a folder names “SHARED”, and in this folder is also a file named “.noindex” “.nomemories” and “.nomedia” to prevent indexing this folder and the mounted subfolders.

But when i run `occ files:scan –all` this folder and the mounted external storages get indexed too.

And when the sftp-mounte computer is turned off, occ files:scan crashes with an error… But it should ignore this not active external storage silently.

Steps to replicate it (hint: details matter!):

  1. add external storage type sftp

  2. put it into a subfolder “SHARED”

  3. add dotfiles “.noindex”, “.nomemories” and “.nomedia”

  4. run `occ files:scan –all -v`

Log entries

Nextcloud

Please provide the log entries from your Nextcloud log that are generated during the time of problem (via the Copy raw option from Administration settings->Logging screen or from your nextcloud.log located in your data directory). Feel free to use a pastebin/gist service if necessary.

Error during scan: fsockopen(): Unable to connect to 192.168.10.2:22 (Host is unreachable)                                                                                                    
Error during scan: Cannot connect to 192.168.10.2:22. Error 113. Host is unreachable                                                                                                          
Exception during scan: Login failed                                                                                                                                                           
#0 /var/www/html/apps/files_external/lib/Lib/Storage/SFTP.php(498): OCA\Files_External\Lib\Storage\SFTP->getConnection()                                                                      
#1 /var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php(294): OCA\Files_External\Lib\Storage\SFTP->getMetaData('')                                                                     
#2 /var/www/html/lib/private/Files/Storage/Wrapper/KnownMtime.php(47): OC\Files\Storage\Wrapper\Wrapper->getMetaData('')                                                                      
#3 /var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php(294): OC\Files\Storage\Wrapper\KnownMtime->getMetaData('')                                                                     
#4 /var/www/html/lib/private/Files/Storage/Wrapper/PermissionsMask.php(114): OC\Files\Storage\Wrapper\Wrapper->getMetaData('')                                                                
#5 /var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php(294): OC\Files\Storage\Wrapper\PermissionsMask->getMetaData('')                                                                
#6 /var/www/html/lib/private/Files/Storage/Wrapper/Availability.php(242): OC\Files\Storage\Wrapper\Wrapper->getMetaData('')                                                                   
#7 /var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php(294): OC\Files\Storage\Wrapper\Availability->getMetaData('')                                                                   
#8 /var/www/html/lib/private/Files/Storage/Wrapper/PermissionsMask.php(114): OC\Files\Storage\Wrapper\Wrapper->getMetaData('')                                                                
#9 /var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php(294): OC\Files\Storage\Wrapper\PermissionsMask->getMetaData('')                                                                
#10 /var/www/html/lib/private/Files/Cache/Scanner.php(98): OC\Files\Storage\Wrapper\Wrapper->getMetaData('')                                                                                  
#11 /var/www/html/lib/private/Files/Cache/Scanner.php(139): OC\Files\Cache\Scanner->getData('')                                                                                               
#12 /var/www/html/lib/private/Files/Cache/Scanner.php(320): OC\Files\Cache\Scanner->scanFile('', 3, -1, NULL, true)                                                                           
#13 /var/www/html/lib/private/Files/Utils/Scanner.php(265): OC\Files\Cache\Scanner->scan('', true, 3)                                                                                         
#14 /var/www/html/apps/files/lib/Command/Scan.php(165): OC\Files\Utils\Scanner->scan('/jakobus.schuer...', true, Object(Closure))                                                             
#15 /var/www/html/apps/files/lib/Command/Scan.php(255): OCA\Files\Command\Scan->scanFiles('jakobus.schuerz', '/jakobus.schuer...', NULL, Object(Symfony\Component\Console\Output\ConsoleOutput), Object(Closure), false, true)                                                                                                                                                              
#16 /var/www/html/3rdparty/symfony/console/Command/Command.php(326): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))                                                                                                                                                                             
#17 /var/www/html/core/Command/Base.php(218): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))                                                                                                                                                                                     
#18 /var/www/html/3rdparty/symfony/console/Application.php(1078): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))                                                                                                                                                                                      
#19 /var/www/html/3rdparty/symfony/console/Application.php(324): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))                                                                                                                             
#20 /var/www/html/3rdparty/symfony/console/Application.php(175): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))                                                                                                                                                                    
#21 /var/www/html/lib/private/Console/Application.php(187): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))                                                                                                                                                                           
#22 /var/www/html/console.php(90): OC\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput))                                                                             
#23 /var/www/html/occ(33): require_once('/var/www/html/c...')                                                                                                                                 
#24 {main}                                                                                                                                                                                    

Web Browser

If the problem is related to the Web interface, open your browser inspector Console and Network tabs while refreshing (reloading) and reproducing the problem. Provide any relevant output/errors here that appear.

PASTE

Web server / Reverse Proxy

The output of your Apache/nginx/system log in /var/log/____:

PASTE HERE

Configuration

Nextcloud

The output of occ config:list system or similar is best, but, if not possible, the contents of your config.php file from /path/to/nextcloud is fine (make sure to remove any identifiable information!):

            "quota": "ownCloudQuota",
            "home": "homeDirectory",
            "ldap_uid": "uid",
            "groups": "ownCloudRoles",
            "login_filter": "realm_access_roles",
            "photoURL": "picture",
            "is_admin": "ownCloudAdmin"
        },
        "oidc_login_default_group": "oidc",
        "oidc_login_use_external_storage": false,
        "oidc_login_scope": "openid profile",
        "oidc_login_proxy_ldap": false,
        "oidc_login_disable_registration": true,
        "oidc_login_redir_fallback": false,
        "oidc_login_alt_login_page": false,
        "oidc_login_tls_verify": true,
        "oidc_create_groups": false,
        "oidc_login_webdav_enabled": false,
        "oidc_login_password_authentication": false,
        "oidc_login_public_key_caching_time": 86400,
        "oidc_login_min_time_between_jwks_requests": 10,
        "oidc_login_well_known_caching_time": 86400,
        "oidc_login_update_avatar": false,
        "oidc_login_skip_proxy": false,
        "oidc_login_code_challenge_method": "S256",
        "loglevel": 2,
        "config_preset": 2,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_smtpauth": true,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "DOMAIN": "nc.schuerz.at"
    }
}

Apps

The output of occ app:list (if possible).

~: # ncphp occ app:list                                                                                                                                                                       
Enabled:                                                                                                                                                                                      
  - activity: 5.0.0-dev.0                                                                                                                                                                     
  - admin_audit: 1.22.0                                                                                                                                                                       
  - assistant: 2.12.0                                                                                                                                                                         
  - bookmarks: 16.0.1                                                                                                                                                                         
  - bruteforcesettings: 5.0.0-dev.0                                                                                                                                                           
  - calendar: 6.1.2                                                                                                                                                                           
  - circles: 32.0.0                                                                                                                                                                           
  - cloud_federation_api: 1.16.0                                                                                                                                                              
  - collectives: 3.4.0                                                                                                                                                                        
  - comments: 1.22.0                                                                                                                                                                          
  - contacts: 8.1.2                                                                                                                                                                           
  - contactsinteraction: 1.13.1                                                                                                                                                               
  - cookbook: 0.11.5                                                                                                                                                                          
  - cospend: 3.1.6                                                                                                                                                                            
  - dav: 1.34.2                                                                                                                                                                               
  - deck: 1.16.3                                                                                                                                                                              
  - epubviewer: 1.8.1                                                                                                                                                                         
  - external: 7.0.0                                                                                                                                                                           
  - federatedfilesharing: 1.22.0                                                                                                                                                              
  - federation: 1.22.0                                                                                                                                                                        
  - files: 2.4.0                                                                                                                                                                              
  - files_accesscontrol: 3.0.2                                                                                                                                                                
  - files_automatedtagging: 3.0.2                                                                                                                                                             
  - files_confidential: 4.0.0                                                                                                                                                                 
  - files_downloadactivity: 1.18.1                                                                                                                                                            
  - files_downloadlimit: 5.0.0-dev.0                                                                                                                                                          
  - files_external: 1.24.0                                                                                                                                                                    
  - files_pdfviewer: 5.0.0-dev.0                                                                                                                                                              
  - files_reminders: 1.5.0                                                                                                                                                                    
  - files_retention: 3.0.0                                                                                                                                                                    
  - files_sharing: 1.24.1                                                                                                                                                                     
  - files_trashbin: 1.22.0                                                                                                                                                                    
  - files_versions: 1.25.0                                                                                                                                                                    
  - fileslibreofficeedit: 2.0.1                                                                                                                                                               
  - forms: 5.2.3                                                                                                                                                                              
  - gpoddersync: 3.13.3                                                                                                                                                                       
  - groupfolders: 20.1.6                                                                                                                                                                      
  - guests: 4.6.0                                                                                                                                                                             
  - impersonate: 3.0.0                                                                                                                                                                        
  - integration_oidc: 0.1.6
  - integration_openai: 3.9.1
  - journeys: 0.19.5
  - logreader: 5.0.0-dev.0
  - lookup_server_connector: 1.20.0
  - maps: 1.6.0
  - memories: 7.7.0
  - music: 2.4.1
  - nextcloud-aio: 0.8.0
  - nextcloud_announcements: 4.0.0-dev.0
  - nextpod: 0.7.8
  - notes: 4.12.4
  - notifications: 5.0.0-dev.0
  - notify_push: 1.2.1
  - oauth2: 1.20.0
  - oidc_login: 3.2.2
  - ownershiptransfer: 1.3.0
  - photos: 5.0.0-dev.1
  - polls: 8.6.2
  - previewgenerator: 5.11.0
  - privacy: 4.0.0-dev.0
  - profile: 1.1.0
  - provisioning_api: 1.22.0
  - related_resources: 3.0.0-dev.0
  - richdocuments: 9.0.2
  - serverinfo: 4.0.0-dev.0
  - settings: 1.15.1
  - sharebymail: 1.22.0
  - spreed: 22.0.7
  - support: 4.0.0-dev.0
  - suspicious_login: 10.0.0-dev.0
  - systemtags: 1.22.0
  - tables: 1.0.2
  - tasks: 0.17.1
  - text: 6.0.1
  - theming: 2.7.0
  - twofactor_backupcodes: 1.21.0
  - twofactor_nextcloud_notification: 6.0.0-dev.0
  - twofactor_totp: 14.0.0
  - updatenotification: 1.22.0
  - user_oidc: 8.2.2
  - user_status: 1.12.0
  - viewer: 5.0.0-dev.0
  - webhook_listeners: 1.3.0
  - whiteboard: 1.5.0
  - workflow_pdf_converter: 3.0.0
  - workflow_script: 3.0.0
  - workflowengine: 2.14.0
Disabled:
  - app_api: 32.0.0 (installed 32.0.0)
  - dashboard: 7.12.0 (installed 7.11.0)
  - emlviewer: 2.0.1 (installed 2.0.1)
  - encryption: 2.20.0 (installed 2.19.0)
  - facerecognition: 0.9.70 (installed 0.9.70)
  - firstrunwizard: 5.0.0-dev.0 (installed 4.0.0)
  - intros: 1.2.1 (installed 1.2.1)
  - libresign: 12.1.0 (installed 12.1.0)
  - mediadc: 0.4.0 (installed 0.4.0)
  - password_policy: 4.0.0-dev.0 (installed 3.0.0)
  - recommendations: 5.0.0-dev.0 (installed 4.0.0)
  - survey_client: 4.0.0-dev.0 (installed 3.0.0)
  - user_ldap: 1.23.0
  - weather_status: 1.12.0 (installed 1.11.0)

Tips for increasing the likelihood of a response

  • Use the preformatted text formatting option in the editor for all log entries and configuration output.
  • If screenshots are useful, feel free to include them.
    • If possible, also include key error output in text form so it can be searched for.
  • Try to edit log output only minimally (if at all) so that it can be ran through analyzers / formatters by those trying to help you.

While these have an impact on certain apps, they do not have on core. AFAIK there’s no mechanismen to check for these files.

Do you run files:scan regularly ?

I wrote a mechanism which uses inotifywait on the filesystem and checks for manual changes in filesystem and triggers a rescan of changed files and directories.

This mechanism collects such changes and sorts and uniqs the entries and scans directories and files every 2 minutes in a batch job, which is locked, so that only one rescan works at a time.

But yesterday i run a full rescan on another reason, and i recognized, that files:scan failed with the named error on a external storage, which was not available at this time.