Folders Missing in Web UI After occ files:scan – "Too many open files in system" Error

ℹ️ Support
, , , , ,
1
Support intro

Sorry to hear you’re facing problems. :slightly_frowning_face:

The community help forum (help.nextcloud.com) is for home and non-enterprise users. Support is provided by other community members on a best effort / “as available” basis. All of those responding are volunteering their time to help you.

If you’re using Nextcloud in a business/critical setting, paid and SLA-based support services can be accessed via portal.nextcloud.com where Nextcloud engineers can help ensure your business keeps running smoothly.

Getting help

In order to help you as efficiently (and quickly!) as possible, please fill in as much of the below requested information as you can.

Before clicking submit: Please check if your query is already addressed via the following resources:

(Utilizing these existing resources is typically faster. It also helps reduce the load on our generous volunteers while elevating the signal to noise ratio of the forums otherwise arising from the same queries being posted repeatedly).

The Basics

  • Nextcloud Server version (e.g., 29.x.x):
  • Operating system and version (e.g., Ubuntu 24.04):
    • Mac Mini M4 Seqouia 15.5
  • Reverse proxy and version _(e.g. nginx 1.27.2)
    • nginx proxy manager (2.12.6)
  • Is this the first time you’ve seen this error? (Yes / No):
    • yes
  • When did this problem seem to first start?
    • after manually adding files to the DATADIR and then running the scan
  • Installation method (e.g. AlO, NCP, Bare Metal/Archive, etc.)
    • AIO on docker
  • Are you using CloudfIare, mod_security, or similar? (Yes / No)
    • no

Summary of the issue you are facing:

Hi all,

I’m running Nextcloud on a Mac Mini using Docker Compose (specifically, the AIO setup). I’m trying to migrate a large number of folders (around 3,500 in total, roughly 12GB) directly into my Nextcloud DATA_DIR.

Here’s what I’m doing:

  1. I move the folders directly into my DATA_DIR - /System/Volumes/Data/mnt/truenas/nas/data/nextcloud/admin/files/Documents/.
  2. I then run the file scan command:
    docker exec -u www-data nextcloud-aio-nextcloud php occ files:scan --path="admin/files/Documents"

The scan completes without errors, but when I check the web interface, many of the folders appear empty or are completely missing.

To debug, I ran:
docker exec nextcloud-aio-nextcloud ls -la /mnt/ncdata/admin/files/Documents/Family

And I get this error for every subfolder inside Family:
ls: cannot access '/mnt/ncdata/admin/files/Documents/Family/Holidays': Too many open files in system

It seems like the system is hitting a file descriptor limit during the scan or access process?

My Questions:

  • Is this a known issue with files:scan handling large folder structures?
  • Do I need to increase the open file limit for the Nextcloud-AIO container during setup?
  • Would breaking the scan into smaller parts help (e.g., scanning per subdirectory)?
  • Any tips to handle large folder imports more reliably going forward?

Configuration

Nextcloud

The output of occ config:list system or similar is best, but, if not possible, the contents of your config.php file from /path/to/nextcloud is fine (make sure to remove any identifiable information!):

{
    "system": {
        "one-click-instance": true,
        "one-click-instance.user-limit": 100,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "check_data_directory_permissions": false,
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "overwritehost": "nextcloud.homelab.local",
        "overwriteprotocol": "https",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "nextcloud.homelab.local"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "31.0.6.2",
        "overwrite.cli.url": "https:\/\/nextcloud.homelab.local\/",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "updatechecker": false,
        "loglevel": 2,
        "log_type": "file",
        "logfile": "\/var\/www\/html\/data\/nextcloud.log",
        "log_rotate_size": 10485760,
        "log.condition": {
            "apps": [
                "admin_audit"
            ]
        },
        "preview_max_x": 2048,
        "preview_max_y": 2048,
        "jpeg_quality": 60,
        "enabledPreviewProviders": {
            "1": "OC\\Preview\\Image",
            "2": "OC\\Preview\\MarkDown",
            "3": "OC\\Preview\\MP3",
            "4": "OC\\Preview\\TXT",
            "5": "OC\\Preview\\OpenDocument",
            "6": "OC\\Preview\\Movie",
            "7": "OC\\Preview\\Krita",
            "0": "OC\\Preview\\Imaginary",
            "23": "OC\\Preview\\ImaginaryPDF"
        },
        "enable_previews": true,
        "upgrade.disable-web": true,
        "mail_smtpmode": "smtp",
        "trashbin_retention_obligation": "auto, 30",
        "versions_retention_obligation": "auto, 30",
        "activity_expire_days": 30,
        "simpleSignUpLink.shown": false,
        "share_folder": "\/Shared",
        "one-click-instance.link": "https:\/\/nextcloud.com\/all-in-one\/",
        "upgrade.cli-upgrade-link": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/2726",
        "updatedirectory": "\/nc-updater",
        "maintenance_window_start": 100,
        "allow_local_remote_servers": true,
        "davstorage.request_timeout": 3600,
        "documentation_url.server_logs": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/5425",
        "htaccess.RewriteBase": "\/",
        "dbpersistent": false,
        "auth.bruteforce.protection.enabled": true,
        "ratelimit.protection.enabled": true,
        "files_external_allow_create_new_local": false,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",
        "preview_imaginary_key": "***REMOVED SENSITIVE VALUE***",
        "DOMAIN": "nextcloud.homelab.local"
    }
}```


<?php
$CONFIG = array (
  'one-click-instance' => true,
  'one-click-instance.user-limit' => 100,
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' =>
  array (
    0 =>
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 =>
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'check_data_directory_permissions' => false,
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' =>
  array (
    'host' => 'nextcloud-aio-redis',
    'password' => 'fffffff',
    'port' => 6379,
  ),
  'overwritehost' => 'nextcloud.homelab.local',
  'overwriteprotocol' => 'https',
  'passwordsalt' => 'ffffff',
  'secret' => 'fffff',
  'trusted_domains' =>
  array (
    0 => 'localhost',
    1 => 'nextcloud.homelab.local',
  ),
  'datadirectory' => '/mnt/ncdata',
  'dbtype' => 'pgsql',
  'version' => '31.0.6.2',
  'overwrite.cli.url' => 'https://nextcloud.homelab.local/',
  'dbname' => 'nextcloud_database',
  'dbhost' => 'nextcloud-aio-database:5432',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'oc_nextcloud',
  'dbpassword' => 'fffffff',
  'installed' => true,
  'instanceid' => 'ocdzqkdl3nt2',
  'maintenance' => false,
  'updatechecker' => false,
  'loglevel' => 2,
  'log_type' => 'file',
  'logfile' => '/var/www/html/data/nextcloud.log',
  'log_rotate_size' => 10485760,
  'log.condition' =>
  array (
    'apps' =>
    array (
      0 => 'admin_audit',
    ),
  ),
  'preview_max_x' => 2048,
  'preview_max_y' => 2048,
  'jpeg_quality' => 60,
  'enabledPreviewProviders' =>
  array (
    1 => 'OC\\Preview\\Image',
    2 => 'OC\\Preview\\MarkDown',
    3 => 'OC\\Preview\\MP3',
    4 => 'OC\\Preview\\TXT',
    5 => 'OC\\Preview\\OpenDocument',
    6 => 'OC\\Preview\\Movie',
    7 => 'OC\\Preview\\Krita',
    0 => 'OC\\Preview\\Imaginary',
    23 => 'OC\\Preview\\ImaginaryPDF',
  ),
  'enable_previews' => true,
  'upgrade.disable-web' => true,
  'mail_smtpmode' => 'smtp',
  'trashbin_retention_obligation' => 'auto, 30',
  'versions_retention_obligation' => 'auto, 30',
  'activity_expire_days' => 30,
  'simpleSignUpLink.shown' => false,
  'share_folder' => '/Shared',
  'one-click-instance.link' => 'https://nextcloud.com/all-in-one/',
  'upgrade.cli-upgrade-link' => 'https://github.com/nextcloud/all-in-one/discussions/2726',
  'updatedirectory' => '/nc-updater',
  'maintenance_window_start' => 100,
  'allow_local_remote_servers' => true,
  'davstorage.request_timeout' => 3600,
  'documentation_url.server_logs' => 'https://github.com/nextcloud/all-in-one/discussions/5425',
  'htaccess.RewriteBase' => '/',
  'dbpersistent' => false,
  'auth.bruteforce.protection.enabled' => true,
  'ratelimit.protection.enabled' => true,
  'files_external_allow_create_new_local' => false,
  'trusted_proxies' =>
  array (
    0 => '127.0.0.1',
    1 => '::1',
    10 => '192.168.107.0/16',
  ),
  'preview_imaginary_url' => 'http://nextcloud-aio-imaginary:9000',
  'preview_imaginary_key' => 'ffffffff',
);

 nextcloud-aio-mastercontainer:
    image: nextcloud/all-in-one:${NEXTCLOUD_AIO_VERSION}
    restart: unless-stopped
    container_name: nextcloud-aio-mastercontainer
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config
      - /var/run/docker.sock:/var/run/docker.sock:ro
    ports:
      - 1010:8080 
    environment:
      # Store Nextcloud data on NAS which is automounted
      - NEXTCLOUD_DATADIR=${DOCKER_MOUNT_STORAGE_DIR}/nextcloud
      - SKIP_DOMAIN_VALIDATION=true
      - APACHE_PORT=11000
      - APACHE_IP_BINDING=0.0.0.0
2

ulimit -n

if output is less than 3500 then increase it to bigger value

I am not much sure about manmini commands but try like below

sudo launchctl limit maxfiles 65536 65536

sudo nano /etc/sysctl.conf

kern.maxfiles=65536
kern.maxfilesperproc=65536

Save and exit. Reboot your Mac Mini to apply the changes.

After reboot check the values changed?
ulimit -n

Now try to if the values changed inside of container too?

docker exec -it nextcloud-aio-nextcloud bash
ulimit -n (It should return you same values as we set in your mac system)
exit

now again run scan

docker exec -u www-data nextcloud-aio-nextcloud php occ files:scan --path="admin/files/Documents

3

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.