Large Thumbnails size in the files app

Nextcloud version (eg, 18.0.2): 18.0.4
Operating system and version (eg, Ubuntu 20.04): Ubuntu 20.04
Apache or nginx version (eg, Apache 2.4.25): Nginx 1.18.0
PHP version (eg, 7.1): php-fpm 7.4.5

The issue you are facing:
The file size of images in the files app is extremely big. I have a lot of folders with a Hugh amount of pictures inside. In slow networks its impossible to use because of the thumbnails need a lot of time to load. So I checked the source code…
I found out that the files app uses pictures with a resolution of 1024x1024 and a file size of 1,2MB with for the small thumbnails. You can imagine that this couldn’t be a well experience with a slow network.

In the picture app everything is fine and its extremely fast. here the pictures are only 256x256 pixel with a size of 30 to 40KB.

I want that nextcloud is using the small previews in the files app

Is this the first time you’ve seen this error? (Y/N): no

Steps to replicate it:

  1. open files app
  2. open folder with a lot of pictures

The output of your Nextcloud log in Admin > Logging:
no error

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'xxx',
  'passwordsalt' => 'xxx',
  'secret' => 'xxx',
  'trusted_domains' => 
  array (
    0 => 'cloud.domain.tld',
  ),
  'datadirectory' => '/var/www/nextcloud/data',
  'dbtype' => 'mysql',
  'version' => '18.0.4.2',
  'overwrite.cli.url' => 'https://cloud.domain.tld',
  'dbname' => 'nextcloud_db',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextclouduser',
  'dbpassword' => 'xxx',
  'installed' => true,
  'updater.release.channel' => 'stable',
  'mail_smtpmode' => 'sendmail',
  'mail_sendmailmode' => 'smtp',
  'mail_smtpsecure' => 'ssl',
  'mail_from_address' => 'info',
  'mail_domain' => 'domain.tld',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'xxx',
  'mail_smtpport' => '465',
  'mail_smtpname' => 'xxx',
  'mail_smtppassword' => 'xxx',
  'mail_smtpauthtype' => 'LOGIN',
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 2,
  'enabledPreviewProviders' => 
  array (
    0 => 'OC\\Preview\\PNG',
    1 => 'OC\\Preview\\JPEG',
    2 => 'OC\\Preview\\GIF',
    3 => 'OC\\Preview\\BMP',
    4 => 'OC\\Preview\\XBitmap',
    5 => 'OC\\Preview\\Movie',
    6 => 'OC\\Preview\\TXT',
    7 => 'OC\\Preview\\MarkDown',
    8 => 'OC\\Preview\\MSOffice2003',
    9 => 'OC\\Preview\\MSOffice2007',
    10 => 'OC\\Preview\\MSOfficeDoc',
    11 => 'OC\\Preview\\OpenDocument',
    12 => 'OC\\Preview\\PDF',
  ),
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'filelocking.enabled' => true,
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => '/var/run/redis/redis-server.sock',
    'port' => 0,
    'timeout' => 0.0,
  ),
);

The output of your Apache/nginx/system log in /var/log/____:

PASTE HERE

You can generate smaller previews to safe time and bandwidth. Here I wrote small script to use in cron job How to know if Preview Generator is working?. And then:

After that you will have nice small previews:

But somehow you are right, I checked and there are still previews around 1 and 2 MB (not all of them), that is strange…

Seems that some previews were not pregenerated, then access to the file generates new file, but in png instead of jpg and it is quite big:

# du -ah preview/ | grep -E "png|jpg" | sort -h -r | head -n 20
2.9M	preview/806331/1536-1024-max.png
2.8M	preview/2823/1528-1044-max.png
2.7M	preview/2899/1527-1080-max.png
2.5M	preview/944452/1920-1080-max.png
2.4M	preview/2506474/1920-1080-max.png
2.4M	preview/1593491/1620-1080-max.png
2.3M	preview/333955/1660-1080-max.png
2.3M	preview/2503143/1190-893-max.png
2.1M	preview/1593493/1620-1080-max.png
2.0M	preview/2952315/1920-1080-max.png
2.0M	preview/2503144/1190-893-max.png
1.9M	preview/2790359/810-1080-max.png
1.9M	preview/1593492/1557-1080-max.png
1.8M	preview/2503151/1920-825-max.png
1.7M	preview/944586/1500-1000-max.png
1.7M	preview/942970/1500-1000-max.png
1.7M	preview/802162/1536-1024-max.png
1.7M	preview/330791/1500-1000-max.png
1.7M	preview/2503150/1190-893-max.png
1.7M	preview/1593629/1854-1080-max.png

and pre generated previews are in JPG, they are pretty ok:

# du -ah preview/ | grep "jpg" | sort -h -r | head -n 20
552K	preview/4588842/1919-1080-max.jpg
544K	preview/4587449/1734-1080-max.jpg
536K	preview/4588844/1919-1080-max.jpg
528K	preview/4588738/1919-1080-max.jpg
516K	preview/782965/1920-1080-max.jpg
512K	preview/72740/1920-1080-max.jpg
512K	preview/6190/1920-1080-max.jpg
512K	preview/4588762/1919-1080-max.jpg
512K	preview/4588693/1919-1080-max.jpg
500K	preview/4588735/1919-1080-max.jpg
500K	preview/2788724/1440-1080-max.jpg
500K	preview/1057309/1920-1080-max.jpg
496K	preview/4588731/1919-1080-max.jpg
496K	preview/4535176/1919-1080-max.jpg
496K	preview/2788709/1440-1080-max.jpg
488K	preview/12135/1920-1080-max.jpg
484K	preview/4588843/1919-1080-max.jpg
484K	preview/4586368/1919-1080-max.jpg
484K	preview/2782961/1440-1080-max.jpg
480K	preview/77330/1920-1080-max.jpg

Thank you very much. Its seems to working. I followed the instructions and pregenerate the previews. The size of my preview folder decreased from 14GB to 2,8GB

But now nextcloud don’t create more previews of PDF files…

ImagickException: attempt to perform an operation not allowed by the security policy `PDF' @ error/constitute.c/IsCoderAuthorized/408

This solved my PDF Problem

# Open the file 
sudo nano /etc/ImageMagick-6/policy.xml

# find and edit the line
<policy domain="coder" rights="none" pattern="PDF" />
# to :
<policy domain="coder" rights="read|write" pattern="PDF" />

Glad that it helps you!

You will meet the same problem with XPS, EPI and PS. Old link to how-to does not work any more, so please use new one from my wiki:
https://gas85.ddns.net/wiki/doku.php?id=howto:nextcloud&s[]=pdf#solutions_for_imagickexceptions_in_nextcloudlog

Solutions for ImagickExceptions in nextcloud.log

You are in trouble with previews and/or with thumbnails within your Nextcloud? You found errors like “ImagickException: not authorized…” in your Nextcloud logfile (/var/nc_data/nextcloud.log)? Just follow these instructions in your terminal session (e.g. ssh) and solve these “Imagick” issues yourself: Error messages e.g.:

tail -f /var/nc_data/nextcloud.log | grep -i imagickexception
 
...
...ImagickException: not authorized `/upload_tmp/oc_tmp_zCg8T6-.d1539116135' @ error/constitute.c/ReadImage/412
...ImagickException: not authorized `/upload_tmp/oc_tmp_BiLdF7-.pdf' @ error/constitute.c/ReadImage/412
...

Solution 1:

sudo -s (or su -)
apt install ghostscript -y
ln -s /usr/local/bin/gs /usr/bin/gs
cp /etc/ImageMagick-6/policy.xml /etc/ImageMagick-6/policy.xml.bak
sed -i "s/rights\=\"none\" pattern\=\"PS\"/rights\=\"read\|write\" pattern\=\"PS\"/" /etc/ImageMagick-6/policy.xml
sed -i "s/rights\=\"none\" pattern\=\"EPI\"/rights\=\"read\|write\" pattern\=\"EPI\"/" /etc/ImageMagick-6/policy.xml
sed -i "s/rights\=\"none\" pattern\=\"PDF\"/rights\=\"read\|write\" pattern\=\"PDF\"/" /etc/ImageMagick-6/policy.xml
sed -i "s/rights\=\"none\" pattern\=\"XPS\"/rights\=\"read\|write\" pattern\=\"XPS\"/" /etc/ImageMagick-6/policy.xml
service php7.3-fpm restart && service nginx restart

Thanks, but I recognize that Nextcloud still isn’t using the 32x32 pictures for the preview in the files list. I don’t know why…