Hello!
TL:DR:
occ encryption:decrypt-all running super slow (2-3 seconds for each few KB file)
I’ve been using ownCloud since the very beginning and switched to NextCloud after it was released. Since this particular instance should hold my and my families data I thought it would be a good idea to enable the encryption. But it was always a pain in the butt - especially on ownCloud upgrades. The NextCloud upgrades went way smoother but still there were always hiccups on the way.
2 days ago I decided that I don’t want to deal with these issues anymore and think of another way. So I went ahead and figured out how to remove the encryption. According to what I’ve found running occ encryption:decrypt-all would set the instance to maintenance, decrypt all files and disable encryption. So I took a backup and started the process sudo -u www-data php occ encryption:decrypt-all. It found my 4 hours and started decrypting the files without any error messages.
The issue I’m now facing is that the decryption is running veeeeeery slow. About 2-3 seconds for a few KB large file. At this rate it would take weeks to decrypt all my data.
- Is this slow speed normal?
- How can I speed up the process?
- Can I interrupt the process and restart it later in case any configuration changes would provide a speed improvement?
System details:
Nextcloud version: 18.0.1.3
Operating system and version: 16.04.6 LTS (Xenial Xerus)
Apache or nginx version: Apache/2.4.41 (Ubuntu)
PHP version: 7.3.15-3 (FPM/FastCGI)
Server hardware:
The server is a dedicated root server with quite beefy hardware:
The EX40 dedicated root server houses a high-performance Intel® Core i7-4770 Quad-Core processor of the well-known Haswell architecture and 32 GB DDR3 RAM. Furthermore, two 2 TB SATA 6 Gb/s Enterprise HDDs are used. Both hard drives are specially designed for sustained rapid growth in data volume. They also guarantee high reliability during continuous operation.
RAM: 32 GB DDR3 RAM
Hard Drive: 2 x 2 TB SATA 6 Gb/s 7200 rpm HDD (Software-RAID 1) Class Enterprise
CPU
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 60
Model name: Intel(R) Xeon(R) CPU E3-1246 v3 @ 3.50GHz
Stepping: 3
CPU MHz: 3823.066
CPU max MHz: 3900.0000
CPU min MHz: 800.0000
BogoMIPS: 7000.19
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 8192K
NUMA node0 CPU(s): 0-7
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm xsaveopt dtherm ida arat pln pts
RAM: 32GB
Hard disk hdparm:
/dev/md2:
Timing cached reads: 29886 MB in 2.00 seconds = 14958.83 MB/sec
Timing buffered disk reads: 114 MB in 3.05 seconds = 37.43 MB/sec
The output of my Nextcloud file:
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:56:59+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#761","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:05+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"stream_wrapper_register(): Protocol ocencryption:// is already defined. at /var/www/x22889/cloud/lib/private/Files/Stream/Encryption.php#203","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:05+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#760","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:05+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#761","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:08+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"stream_wrapper_register(): Protocol ocencryption:// is already defined. at /var/www/x22889/cloud/lib/private/Files/Stream/Encryption.php#203","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:08+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#760","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:08+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#761","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:10+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"stream_wrapper_register(): Protocol ocencryption:// is already defined. at /var/www/x22889/cloud/lib/private/Files/Stream/Encryption.php#203","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:10+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#760","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:10+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#761","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:12+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"stream_wrapper_register(): Protocol ocencryption:// is already defined. at /var/www/x22889/cloud/lib/private/Files/Stream/Encryption.php#203","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:12+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#760","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:12+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#761","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:15+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"stream_wrapper_register(): Protocol ocencryption:// is already defined. at /var/www/x22889/cloud/lib/private/Files/Stream/Encryption.php#203","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:15+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#760","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:15+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#761","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:17+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"stream_wrapper_register(): Protocol ocencryption:// is already defined. at /var/www/x22889/cloud/lib/private/Files/Stream/Encryption.php#203","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:17+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#760","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:17+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#761","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:20+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"stream_wrapper_register(): Protocol ocencryption:// is already defined. at /var/www/x22889/cloud/lib/private/Files/Stream/Encryption.php#203","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:20+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#760","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:20+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#761","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:22+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"stream_wrapper_register(): Protocol ocencryption:// is already defined. at /var/www/x22889/cloud/lib/private/Files/Stream/Encryption.php#203","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:22+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#760","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:22+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#761","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:24+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"stream_wrapper_register(): Protocol ocencryption:// is already defined. at /var/www/x22889/cloud/lib/private/Files/Stream/Encryption.php#203","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:24+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#760","userAgent":"--","version":"18.0.1.3"}
{"reqId":"zrlDBZRTYuaCUQCyKSC5","level":3,"time":"2020-03-07T14:57:24+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, string given at /var/www/x22889/cloud/lib/private/Files/Storage/Wrapper/Encryption.php#761","userAgent":"--","version":"18.0.1.3"}
The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):
<?php
$CONFIG = array (
'activity_expire_days' => 14,
'auth.bruteforce.protection.enabled' => true,
'blacklisted_files' =>
array (
0 => '.htaccess',
1 => 'Thumbs.db',
2 => 'thumbs.db',
),
'instanceid' => 'xxx',
'passwordsalt' => 'xxx',
'trusted_domains' =>
array (
0 => 'cloud.example.com',
1 => 'example.com',
),
'datadirectory' => '/var/www/oc_data',
'dbtype' => 'mysql',
'version' => '18.0.1.3',
'dbname' => 'dbname',
'dbhost' => 'localhost',
'dbtableprefix' => 'oc_',
'dbuser' => 'dbusr',
'dbpassword' => 'dbpw',
'installed' => true,
'enable_previews' => true,
'enabledPreviewProviders' =>
array (
0 => 'OC\\Preview\\PNG',
1 => 'OC\\Preview\\JPEG',
2 => 'OC\\Preview\\GIF',
3 => 'OC\\Preview\\BMP',
4 => 'OC\\Preview\\XBitmap',
5 => 'OC\\Preview\\Movie',
6 => 'OC\\Preview\\PDF',
7 => 'OC\\Preview\\MP3',
8 => 'OC\\Preview\\TXT',
9 => 'OC\\Preview\\MarkDown',
),
'loglevel' => 2,
'log_rotate_size' => 104857600,
'logtimezone' => 'Europe/Berlin',
'theme' => '',
'maintenance' => true,
'forcessl' => true,
'mail_smtpmode' => 'smtp',
'mail_from_address' => 'cloud',
'mail_domain' => 'example.com',
'secret' => 'xxx',
'forceSSLforSubdomains' => false,
'singleuser' => false,
'trashbin_retention_obligation' => 'auto',
'filelocking.enabled' => 'true',
'debug' => false,
'overwrite.cli.url' => 'https://cloud.example.com',
'mail_sendmailmode' => 'smtp',
'mail_smtphost' => 'x22889.de',
'mail_smtpsecure' => 'tls',
'mail_smtpport' => '25',
'app_install_overwrite' =>
array (
0 => 'calendar',
),
'mysql.utf8mb4' => true,
'updater.release.channel' => 'stable',
'updater.secret' => 'xxx',
);
This text will be hidden
The output of your Apache/nginx/system log in /var/log/____:
Logs for this subdomain are not logging anything
Thanks,
Stefan