coredraw thumbnail files

The Basics

  • Nextcloud Server version (e.g., 29.x.x):
    • 30.0.4
  • Operating system and version (e.g., Ubuntu 24.04):
    • Ubuntu 24.04
  • Web server and version (e.g, Apache 2.4.25):
    • Apache/2.4.58
  • PHP version (e.g, 8.3):
    • PHP 8.3.6
  • Is this the first time you’ve seen this error? (Yes / No):
    • Yes
  • When did this problem seem to first start?
    • When accessing files
  • Installation method (e.g. AlO, NCP, Bare Metal/Archive, etc.)
    -Example installation on Ubuntu 22.04 LTS — Nextcloud latest Administration Manual latest documentation
  • Are you using CloudfIare, mod_security, or similar? (Yes / No)
    • no

Summary of the issue you are facing:

I have a server with 10tb of files running normally, where most of the files are from coreldraw (.cdr). I need a preview for these types of files, I couldn’t get it anywhere, I decided to develop my own way of displaying the thumbnails, in my test environment my solution worked perfectly, however when I put the quantities of files and their sizes into production It makes the server very slow to the point of being almost impossible to use. I would like help to optimize this task.

Steps to replicate it (hint: details matter!):

  1. apt-get install inkscape
  2. add in file nextcloud\lib\privatePreviewManager.php
		$this->registerCoreProvider(Preview\OpenDocument::class, '/application\/vnd.oasis.opendocument.*/');
		$this->registerCoreProvider(Preview\CorelDraw::class, '/application\/coreldraw/');
		$this->registerCoreProvider(Preview\Imaginary::class, Preview\Imaginary::supportedMimeTypes());
  1. add file nextcloud\lib\private\Preview\CorelDraw.php
<?php

/**
 * SPDX-FileCopyrightText: 2024 Nextcloud GmbH and Nextcloud contributors
 * SPDX-License-Identifier: AGPL-3.0-only
 */

namespace OC\Preview;

use OCP\Files\File;
use OCP\IImage;
use Psr\Log\LoggerInterface;

/**
 * Creates thumbnails for CorelDraw (.cdr) files using Inkscape
 */
class CorelDraw extends ProviderV2 {
    /**
     * {@inheritDoc}
     */
    public function getMimeType(): string {
        return '/application\/coreldraw/';
    }

    /**
     * Path to Inkscape executable
     */
    private string $inkscapePath = '/usr/bin/inkscape';

    /**
     * {@inheritDoc}
     */
    public function getThumbnail(File $file, int $maxX, int $maxY): ?IImage {
        
        $tmpPath = $this->getLocalFile($file);
        
        if ($tmpPath === false) {
            \OC::$server->get(LoggerInterface::class)->error(
                'Failed to get local file for thumbnail: ' . $file->getPath(),
                ['app' => 'core']
            );
            return null;
        }

        // Create temporary output file for PNG
        $outputPath = \OC::$server->getTempManager()->getTemporaryFile('.png');

        try {
            // Prepare Inkscape command
            $command = escapeshellcmd($this->inkscapePath . 
                ' "' . $tmpPath . '"' .
                ' --export-filename="' . $outputPath . '"' .
                ' --export-dpi=150');
            

            // Execute command
            exec($command . ' 2>&1', $output, $returnCode);

            if ($returnCode !== 0) {
                throw new \Exception('Inkscape conversion failed with code ' . $returnCode . '. Output: ' . implode("\n", $output));
            }

            // Verify if output file exists and has size
            if (file_exists($outputPath)) {
            } else {
            }

            // Create image from the PNG
            $image = new \OCP\Image();
            $loaded = $image->loadFromFile($outputPath);

            // Clean up temporary files
            unlink($outputPath);
            $this->cleanTmpFiles();

            // Resize if needed
            if ($image->valid()) {
                $image->scaleDownToFit($maxX, $maxY);
                return $image;
            } else {
            }
        } catch (\Exception $e) {
            \OC::$server->get(LoggerInterface::class)->error(
                'CorelDraw preview generation failed: ' . $file->getPath(),
                [
                    'exception' => $e,
                    'app' => 'core',
                ]
            );
            
            // Clean up on error
            if (file_exists($outputPath)) {
                unlink($outputPath);
            }
            $this->cleanTmpFiles();
        }

        return null;
    }
}

  1. add in file nextcloud\config\config.php
  'enabledPreviewProviders' => [
  'OC\Preview\CorelDraw',
],

Log entries

Nextcloud

Please provide the log entries from your Nextcloud log that are generated during the time of problem (via the Copy raw option from Administration settings->Logging screen or from your nextcloud.log located in your data directory). Feel free to use a pastebin/gist service if necessary.

https://bomtech.info/nextcloud.log

Configuration

Nextcloud

The output of occ config:list system or similar is best, but, if not possible, the contents of your config.php file from /path/to/nextcloud is fine (make sure to remove any identifiable information!):

root@central:/var/www2# sudo -u www-data php /var/www/occ config:list system
{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "192.168.254.3"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "30.0.4.1",
        "overwrite.cli.url": "http:\/\/192.168.254.3",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "default_phone_region": "br",
        "maintenance_window_start": 1,
        "maintenance": false,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "apc.enable_cli": true,
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379,
            "timeout": 0
        },
        "mail_smtpmode": "smtp",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtpport": "587",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "trashbin_retention_obligation": "auto, 30",
        "versions_retention_obligation": "auto, 30"
    }
}


Apps

The output of occ app:list (if possible).

root@central:/var/www2# sudo -u www-data php /var/www/occ app:list
Enabled:
  - activity: 3.0.0
  - app_api: 4.0.3
  - bruteforcesettings: 3.0.0
  - circles: 30.0.0
  - cloud_federation_api: 1.13.0
  - comments: 1.20.1
  - contactsinteraction: 1.11.0
  - dashboard: 7.10.0
  - dav: 1.31.1
  - federatedfilesharing: 1.20.0
  - federation: 1.20.0
  - files: 2.2.0
  - files_downloadlimit: 3.0.0
  - files_pdfviewer: 3.0.0
  - files_reminders: 1.3.0
  - files_sharing: 1.22.0
  - files_trashbin: 1.20.1
  - files_versions: 1.23.0
  - firstrunwizard: 3.0.0
  - logreader: 3.0.0
  - lookup_server_connector: 1.18.0
  - nextcloud_announcements: 2.0.0
  - notifications: 3.0.0
  - oauth2: 1.18.1
  - password_policy: 2.0.0
  - photos: 3.0.2
  - privacy: 2.0.0
  - provisioning_api: 1.20.0
  - recommendations: 3.0.0
  - related_resources: 1.5.0
  - serverinfo: 2.0.0
  - settings: 1.13.0
  - sharebymail: 1.20.0
  - support: 2.0.0
  - survey_client: 2.0.0
  - systemtags: 1.20.0
  - text: 4.1.0
  - theming: 2.5.0
  - twofactor_backupcodes: 1.19.0
  - updatenotification: 1.20.0
  - user_status: 1.10.0
  - viewer: 3.0.0
  - weather_status: 1.10.0
  - webhook_listeners: 1.1.0-dev
  - workflowengine: 2.12.0
Disabled:
  - admin_audit: 1.20.0
  - encryption: 2.18.0
  - files_external: 1.22.0
  - previewgenerator: 5.7.0 (installed 5.7.0)
  - suspicious_login: 8.0.0
  - twofactor_nextcloud_notification: 4.0.0
  - twofactor_totp: 12.0.0-dev
  - user_ldap: 1.21.0


Tips for increasing the likelihood of a response

  • Use the preformatted text formatting option in the editor for all log entries and configuration output.
  • If screenshots are useful, feel free to include them.
    • If possible, also include key error output in text form so it can be searched for.
  • Try to edit log output only minimally (if at all) so that it can be ran through analyzers / formatters by those trying to help you.
1 Like

Hello,

to answer your actual question, I guess the preview generation itself takes significant time. I did something similar with inkscape to convert SVG → PNG. The startup of inkscape took on my machine in the range of a few seconds.

Maybe you could simply check how long the actual conversion takes? Do something like

time inkscape somCorelFile.cor $(mktemp).png --export-dpi=150

on the same machine that you are running Nextcloud (if you use docker or such, go inside the containers). Just have a look how long the inkscape command takes.

If this is the culprit, you might check if you find a way to speed up inkscape. I might be interested in a solution in general :wink: .

How to fix this (if my assumption is true)

The point is that the preview generation takes significant amount of time. How much, you can measure as above. You might want to consider using previewgenerator app. It allows to build the previews nightly and store the generated images permanently. Upon browsing the NC instance, the pre-rendered previews are queried first.

You will however trade speed for space required. So, you should be aware of the extra storage requirements.

Why you should not alter the core

The changes you made are in the NC core. The golden rule is: Do not change the core unless you are a core developer.

  • Your instance will report a broken installation as the signatures of the files will no longer match.
  • Any update you install will revert the changes you made.
  • It is not guaranteed that the behavior in the core stays as it is. Maybe the changed need to be made in a different file? Or in a different location? Or a different method?

So long story short: Do not alter the core.

What you should/could do instead?

As you seem to be happy with coding in general, I suggest that you create a minimal app that registers the corresponding preview generator. I sketch the steps here very briefly:

  1. Create a boilerplate app in the app store. This will give you a good starting point.
  2. Update the Application class to implement IBootstrap
  3. In the boot method, you can register a preview provider using registerPreviewProvider.
  4. The provider you can use as you already did.
  5. You can get rid of any additional classes and API endpoints if you like. For now this is a very basic app. If you wanted to add configuration etc later, you can keep and just remove the frontend installation (so that the boilerplate framework is not shown) in the appinfo/info.xml.
  6. You do not have to publish the app to the app store. Putting the code in the corresponding apps folder of the NC instance should be sufficient to find and enable the app. Publication would however allow others to benefit from your work.

If you have any further questions, feel free to ask!

Christian

2 Likes

Thank you for your answer, understood I will try not to change the core, but my problem is related to inkscape not being able to deal with very large files, even in the terminal if I try to convert them I am unsuccessful and this can take several minutes. In this case, the official preview is no longer a copy of this extension: Preview Generator - Apps - App Store - Nextcloud? For me I thought it had been incorporated into the nexcloud core.

I thought the thumbnails were saved permanently, but for me that’s not a problem, I have enough space, the problem is that inkscape is slow, there’s no point in decreasing the dpi the conversion will still be slow, I was thinking further, about understanding how inkscape deals with cdr files, from So create a library to render the images directly from the .cdr without having to convert them, I believe it is possible with c++.

1 Like

This sounds like a basic problem. I found this SO question that indicates a problem with loading the fonts. You could try that in a VM or docker container if that was possible to speed up.

Creating your own library in C(++) is a completely new level but could be doable in general. Then you would ideally just compile a binary and replace the inkscape command.

One more idea could be to look at ImageMagick. I think they support corel as well. Might be at least worth a try on the CLI.

You might have two things intermixed in your mind:

The generation of thumbnails is located in the core (where you hacked into the code in fact). This might have been migrated to the core recently, I would have to check it.

The app on the other hand is to pre-generate the images. So, without the app, the images are rendered each time they are needed. When rendering takes significant time, this is pure poison. Here, storing the pre-generated images by means of the preview generator app in a cache speeds up the actual loading.

Internally, the app uses the same algorithm from the main core to actually create PNG version of the files in question and store them (in different resolutions) in the app folder (do not mind where exactly, the app knows :smile:).

Chris

1 Like

I understand perfectly, I’m having a hard time, building my own library would be a dream, but I’m not capable of that, I would love to be able to share it here.

But unfortunately I’m facing other problems that Inkscape itself doesn’t know how to deal with, such as opening the latest versions of Corel files.
My users used samba server and I recently migrated to nextcloud, but now at this point I ended up deciding to go back to samba because of these thumbnails, I spent 7 days to configure our nextcloud because it deals with a lot of files.

I’m looking forward to being able to use it again, but without a thumbnail of the .cdr it’s impossible, the problem with all this is that Corel is always being updated and if there were any good Samaritans to create a library it would need to be constantly updated.

Regarding Imagemagick, it is not capable of dealing with Corel files, from everything I have researched, only Inkscape and LibreOffice are capable, Uniconvertor is possible to use, but it uses a very old version of Phyton and is out of date, being incompatible with current files from corel.

How about libreoffice in headless mode?

1 Like

If I use libreoffice it also works, I had already tested this.

libreoffice --headless --convert-to svg file.cdr

However, we are back to square one, the problem is dealing with file sizes and the delay in processing this conversion.

I need to optimize this, searching better in the inkscape and libreoffice repository folders to learn how they can deal with .(cdr) files.

I ended up seeing that inkscape uses a libreoffice library to convert these files called libcdr.

From then on I forgot any solution with inkscape, let’s go straight to the source, I downloaded the library directly to my linux server and I’m trying to use it on my apache server using php ffi.

But by far this will be the solution, I believe that the slowness will continue, I am seeing the possibility of incorporating the code directly into PHP as an extension (.so), this way perhaps we could reduce the steps, speeding up the process.

Today’s browsers can display (.svg), perhaps it would be an alternative to deal with the files directly without the need for multiple conversions.

I’m crazy with no experience dealing with this, sorry if something is wrong.

Maybe I won’t even use this solution, I’m just curious and accepting the challenge, in relation to Corel’s copyright, would there be any implications in developing something to view Corel’s files?

I find it so strange that no one has done something similar.

1 Like