Thumbnails / Previews loading slow on files and gallery app

Phil333 · August 23, 2019, 12:28pm

Hello,

I have a Nextcloud 16.0.4 running on a VM that uses dietpi as OS.
The System has 12GB RAM and a i5 3570S - so its not really on the low side (like a Raspberry or similar).
PHP is on version 7.3.7
and Apache2 is on Server version: Apache/2.4.25 (Debian) and Server built: 2019-04-02T19:05:13
The server is running locally on gigabit ethernet - so that isnt a bottleneck here.

So everything is up to date.

I am really happy with Nextcloud but the only thing that was always lacking was the slow speed on showing images.

I already did this changes to speed things up:

occ config:app:set previewgenerator squareSizes --value=“32 256”
occ config:app:set previewgenerator widthSizes --value=“256 384”
occ config:app:set previewgenerator heightSizes --value=“256”
occ config:system:set preview_max_x --value 2048
occ config:system:set preview_max_y --value 2048
occ config:system:set jpeg_quality --value 60
occ config:app:set preview jpeg_quality --value=“60”

source: ownyourbits.com

I installed previewgenerator and it created all the preview images (took ~4 hours for 14GB of previews cron is working and the app does it job as far as I can tell).

When I now open Nextcloud in a browser and go into a folder with pictures (jpeg) for the first time, then the thumbnails of the images show up too slow to be already generated.

So they might not be correcty generated or not used (they are located at nextcloud_data/appdata_***/preview).

Its the same for the “gallery” app.
On the first time I open a folder, the previews/thumbnails are generated - that can take several minutes for large folders.

After that is finished, the loading time is much faster, but I think the previewgenerator-app is made to overcome this problem.

I read something about imagick, previewgenerator and other tweaks, but before I try more that wont work, I think its the best to ask you guys for help and your thoughs on this.

If you need more (information, video that shows the problem as reference etc.), please let me know.

Thanks a lot!
Phil

Sanook · August 23, 2019, 7:14pm

I would install Redis

https://docs.nextcloud.com/server/16/admin_manual/configuration_server/caching_configuration.html#id2

Phil333 · August 26, 2019, 7:03pm

Thanks @Sanook for your reply.

I think that using Redis will increase overall performance, but it may not overcome the fact, that the preview images/thumbnails are only generated on fist opening of the containing folder.

After this, the thumbnails get generated and are then loaded pretty fast…
but there must be a way to generate these thumbnails in background…

Sanook · August 29, 2019, 1:23am

Here: https://www.allerstorfer.at/nextcloud-install-preview-generator/

Rychu · September 1, 2019, 4:42pm

@Sanook he wrote he already have previewgenerator installed.

@Phil333 I observe the same thing on my server. Also have preview generator installed and cron running fine but when I enter a folder for the first time from Android App first thing I see are grey boxes and need to wait a while to get them populated with actual preview. But I don’t know if this is just because of an overall speed of nextcloud (I have J5005 embedded Intel CPU) or is there really something wrong with previewgenerator’s thumbnails.

I guess the android app caches the thumbnails so this is why any next time the thumbnails are there immediately.

enoch85 · September 3, 2019, 1:32am

I added APCu in the Nextcloud VM recently. That made things ALOT faster regarding thumbnails and I don’t even have PreviewGenerator installed by default.

Check it out:

github.com

nextcloud/vm/blob/master/nextcloud_install_production.sh#L435-L483


# Enable igbinary for PHP
# https://github.com/igbinary/igbinary
if is_this_installed "php$PHPVER"-dev
then
    if ! yes no | pecl install -Z igbinary
    then
        msg_box "igbinary PHP module installation failed"
        exit
    else
        print_text_in_color "$IGreen" "igbinary PHP module installation OK!"
    fi
{
echo "# igbinary for PHP"
echo "extension=igbinary.so"
echo "session.serialize_handler=igbinary"
echo "igbinary.compact_strings=On"
} >> $PHP_INI
restart_webserver
fi

This file has been truncated. show original

For PreviewGenerator there is this script: https://github.com/nextcloud/vm/blob/master/apps/previewgenerator.sh

MichaIng · January 19, 2020, 3:59pm

Just saw the DietPi tag. When using dietpi-software to install Nextcloud, Redis and APCu are set up automatically. For image loading times it is indeed the preview generation that is quite slow on Nextcloud. Preview generator app can help with this to pre-generate them regularly in background instead ondemand when viewing in web UI the first time. There is much discussion about this topic on GitHub, about new methods and PHP modules for preview generation, having this done multi-threaded on multi core CPUs (@nachoparker) etc, so enhancements can be expected somewhere on the future. Until then tuning the settings to generate only preview sizes that are really required/used by the installed apps can help.

cracksilver · March 4, 2020, 12:09pm

Hi there.

I had the the same issue with preview. Enabled MemChache APCu. Now it is much better.

See here: https://docs.nextcloud.com/server/18/admin_manual/configuration_server/caching_configuration.html?highlight=memcache

Next step I’ll try preview generator here:https://www.allerstorfer.at/nextcloud-install-preview-generator/

cracksilver

cracksilver · March 26, 2020, 7:54pm

Hi again

I have installed today redis on my server. Now I get the double of speed than just with memcache.

My nextcloud runs on Unraid so I can speak just for that. But I think it should not be that hard to get Redis server working on any other linux os.

Installation manual I used

cracksilver

gas85 · March 27, 2020, 8:05am

How about this? In tutorial APCu and Redis being used parallel:
https://docs.nextcloud.com/server/17/admin_manual/configuration_server/caching_configuration.html#id2
My own config is also looks like this:

  'memcache.local' => '\\OC\\Memcache\\APCu',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',

It is basically “Organizations with single-server and clustered setups” https://docs.nextcloud.com/server/17/admin_manual/configuration_server/caching_configuration.html#organizations-with-single-server-and-clustered-setups

And additional notes for Redis vs. APCu and Memcache on memory caching:

APCu is faster at local caching than Redis. If you have enough memory, use APCu for Memory Caching and Redis for File Locking. If you are low on memory, use Redis for both.

Memcached is a reliable oldtimer for shared caching on distributed servers, and performs well with Nextcloud with one exception: it is not suitable to use with Transactional File Locking because it does not store locks, and data can disappear from the cache at any time (Redis is the best memcache for this).

KarlF12 · March 27, 2020, 12:24pm

Yes, you should set it up exactly how the documentation says unless you have a specific reason not to. I set mine up with APCu and Redis and have never had to go back and change it.