The preview feature consumes too many inodes

ℹ️ Support
1

An astronomical amount of directories are being created in the appdata_*/preview folder, which results in consuming too many inodes, eventually blocking the entire system (ext4 partition).

screenshot

I tried to stop Apache, deleted the contents of the preview folder, disabled the preview function with this configuration, and restarted the server, but the folders are recreated.

  'enable_previews' => false,
  'preview_max_x' => 0,
  'preview_max_y' => 0,
  'preview_max_scale_factor' => 1,

Version: Nextcloud Hub 8 (29.0.1)

2

You should understand why and how they are created. It is organized in a “hex-tree”, thats all OK.

read my explanation → here ←

1 Like
3

I agree with your explanation, but why continue generating this tree structure when the function is disabled ?

4

Hi @devsamt

the question should be, why you run out of inodes before the storage space is exhausted.
This may be because the inode density was set too low when the partition was formatted.

What is the output of

dumpe2fs -h /dev/sda1

ernolf

5

The output of dumpe2fs -h /dev/sda1 is:

dumpe2fs 1.46.2 (28-Feb-2021)
Filesystem volume name:   <none>
Last mounted on:          /
Filesystem UUID:          07208e15-37b1-40bc-b66a-1860d68b61ea
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
Filesystem flags:         signed_directory_hash 
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              6496256
Block count:              25964032
Reserved block count:     1298201
Overhead clusters:        554244
Free blocks:              17689826
Free inodes:              6225730
First block:              0
Block size:               4096
Fragment size:            4096
Group descriptor size:    64
Reserved GDT blocks:      1024
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Flex block group size:    16
Filesystem created:       Tue Nov 28 10:15:22 2023
Last mount time:          Sat Jun  1 09:09:37 2024
Last write time:          Sat Jun  1 09:09:37 2024
Mount count:              187
Maximum mount count:      -1
Last checked:             Tue Nov 28 10:15:22 2023
Check interval:           0 (<none>)
Lifetime writes:          4163 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:	              256
Required extra isize:     32
Desired extra isize:      32
Journal inode:            8
First orphan inode:       786512
Default directory hash:   half_md4
Directory Hash Seed:      d483f535-f388-4a40-be1e-4c6c7afd0438
Journal backup:           inode blocks
Checksum type:            crc32c
Checksum:                 0xb0f6b721
Journal features:         journal_incompat_revoke journal_64bit journal_checksum_v3
Total journal size:       512M
Total journal blocks:     131072
Max transaction length:   131072
Fast commit length:       0
Journal sequence:         0x03923e5b
Journal start:            84289
Journal checksum type:    crc32c
Journal checksum:         0xb2a739b0

Your response implies that my question is not legitimate, but I must insist: if no preview file should be written, why create a structure for it anyway?

Thank you for your help.

6

Did you rerun occ files:scan-app-data again after you deleted things?

7 
~# sudo systemctl stop apache2
~# rm -rf /var/www/nextcloud/data/appdata_xxx/preview/*
~# sudo -u www-data php /var/www/nextcloud/occ files:scan-app-data
Scanning AppData for files

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 173     | 525   | 00:00:00     |
+---------+-------+--------------+
~# df -i /dev/sda1
Filesystem      Inodes  IUsed   IFree IUse% Mounted on
/dev/sda1      6496256 270567 6225689    5% /
~# sudo systemctl start apache2

After these operations, if I navigate to a folder containing images, a directory structure is recreated, with no file inside.

~# find //var/www/nexcloud/data/appdata_xxx/preview | wc -l
307
8

Yeah we understand your attempt at content-addressable storage, but my server just ground to a halt because NextCloud preview is using up 22 million inodes. This does not appear to be a sensible way to do things.

9

Hi, any way to prevent preview generator from consuming all the inodes by simply writing one preview file per original file instead of countless subfolders for each file?