Decryption actually encrypted somehow my files? NC 21.0.1 on a trunas plugin-jail

Hello,

I’m afraid, that nextcloud unintentionally encrypted all my files on occ encryption:decrypt-all.
file

In order to get rid of a couple of messages regarding encryption, I thought decrypting all files and then disabling the default-encryption-module will let those mesages disappear.

“Please enable server side encryption in the admin settings in order to use the encryption module.” - “Bitte aktiviere server-seitige Verschlüsselung in den Administrator-Einstellungen um das Verschlüsselungsmodul nutzen zu können”

“encryption:scan:legacy-format” “does not have a proper header”

“Verschlüsselung-App ist aktiviert, aber Deine Schlüssel sind nicht initialisiert. Bitte melde Dich ab und wieder an.” - “Encryption App is enabled, but your keys are not initialized. Please log-out and log-in again.”

Translation: server/apps/encryption/l10n/de.json at master · nextcloud/server · GitHub

root@nextcloud:/usr/local/www/nextcloud # ~/occ.sh encryption:decrypt-all
PHP Warning:  Use of undefined constant ‘loglevel’ - assumed '‘loglevel’' (this will throw an Error in a future version of PHP) in /usr/local/www/nextcloud/config/config.php on line 19
The process control (PCNTL) extensions are required in case you want to interrupt long running commands - see https://www.php.net/manual/en/book.pcntl.php
Disable server side encryption... done.


You are about to start to decrypt all files stored in your Nextcloud.
It will depend on the encryption module and your setup if this is possible.
Depending on the number and size of your files this can take some time
Please make sure that no user access his files during this process!

Do you really want to continue? (y/n) y
prepare encryption modules...
 done.


 Fetch list of users... finished
 [============================]


 decrypt files for user floogy (2 of 2): /floogy/files/Documents/application/2021/20210204.pdf
[...]
. decrypt files for user floogy(2 of 2): /floogy/files/Documents/Arbeit/beitragstyp-mitarbeit starting to decrypt files... finished
 [=]


all files could be decrypted successfully!
root@nextcloud:/usr/local/www/nextcloud # ~/occ.sh encryption:list-module

root@nextcloud:/usr/local/www/nextcloud # ~/occ.sh encryption:scan:legacy-format
The process control (PCNTL) extensions are required in case you want to interrupt long running commands - see https://www.php.net/manual/en/book.pcntl.php
Scanning all files for legacy encryption
Scanning all files for admin
Scanning all files for floogy
/floogy/files_trashbin/files/20200829-TrueNAS-SCALE.iso.d1606769189 does not have a proper header
[...]
root@nextcloud:/usr/local/www/nextcloud # file "data/floogy/files/Documents/application/20210204.pdf : data
root@nextcloud:/usr/local/www/nextcloud # od -c "data/floogy/files/Documents/application/20210204.pdf | head -n30
0000000    H   B   E   G   I   N   :   o   c   _   e   n   c   r   y   p
0000020    t   i   o   n   _   m   o   d   u   l   e   :   O   C   _   D
0000040    E   F   A   U   L   T   _   M   O   D   U   L   E   :   c   i
0000060    p   h   e   r   :   A   E   S   -   2   5   6   -   C   T   R
0000100    :   s   i   g   n   e   d   :   t   r   u   e   :   H   E   N
0000120    D   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -
0000140    -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -
*
0020000    q   C   D   9   N   5   i   Q   V   B   y   0   b   6   j   w
0020020    s   t   e   b   P   c   0   u   V   W   S   f   h   S   s   d
0020040    v   n   q   A   w   E   N   T   9   Z   f   S   h   P   w   Z
0020060    2   q   I   K   n   S   3   Z   3   a   Y   6   h   K   a   H
0020100    v   S   K   D   R   w   b   I   m   p   9   /   z   c   R   q
0020120    v   P   X   d   h   I   7   8   m   r   g   C   S   S   g   L
0020140    f   V   a   C   U   0   r   O   U   8   W   v   u   T   A   t
0020160    W   7   6   F   u   t   q   r   z   T   j   q   x   r   /   T
0020200    Z   p   d   8   D   Z   6   6   u   2   F   2   k   i   B   n
0020220    G   4   d   E   k   n   m   o   7   e   e   S   O   6   B   E
0020240    1   i   f   n   T   8   Z   6   q   x   n   Y   b   N   j   1
0020260    v   F   N   W   v   a   p   D   X   9   T   6   K   o   r   +
0020300    0   G   x   P   d   e   p   7   +   0   p   r   4   p   Q   O
0020320    z   3   y   e   v   o   R   1   B   p   X   l   z   p   3   h
0020340    W   Y   T   N   E   8   Q   a   I   H   8   W   J   t   O   k
0020360    3   4   v   b   q   G   D   i   w   U   3   1   +   x   u   S
0020400    G   K   6   F   6   k   X   S   Q   x   N   2   h   G   f   6
0020420    O   d   w   Y   k   Y   i   g   y   u   W   +   m   P   u   l
0020440    V   v   8   8   o   h   Y   z   9   S   X   M   q   C   D   r
0020460    c   l   z   D   3   6   6   I   E   S   r   z   +   2   Y   U
0020500    R   C   H   J   j   9   B   c   G   0   3   O   c   v   9   B
0020520    j   w   a   x   v   M   U   5   i   i   X   l   1   c   V   g

I briefly tested encryption on Nextcloud a couple of years ago, and my conclusion was that unless you want to lose all your data, you’d better keep it disabled.

Good luck with data recovery. Hope you have a good backup.

I got a backup with veeam, but only on a client. How will that recovery be synced? Maybe it will be destroyed due to sync as well?

I also don’t understand the “transparent” NC encryption. I’m not sure if or how one can tell a file is encryped or not.

Unfortunately no longer available: GitHub - arno01/ocdec: ownCloud Data offline decryption tool

Maybe a clone of the github repository above: GitHub - rjaeckel/ocdec: ownCloud Data offline decryption tool

Okay, nextcloud seems to detect the restored files accoring the access or create date. The change Date is kept intact, so one can see when the file was altered in its content.

But I’m wondering why this happened to me with nextcloud. Why didn’t decrypt detect, that the files aren’t encrypted yet and encrypted them instead if decrypt those files that where encrypted beforehand? Bizarre to me.

I was wrong. Nextcloud overwrites the client backup with the server content (as I suspected in the first place).

Hm, what can I do? Delete the whole nextcloud content, then restore the backup again and let it synchronize? Or can I force the client to exchange the server content with the restored client content?

Disable all encryption in nextcloud, then restore the /data directory, then see what (if anything) you can do to decrypt any remaining encrypted files, then run occ files:scan to fix the database.

Unfortunately I only got a backup on client side. Maybe I should delete everything in “files” and restore the backup to the client, and then syncing it to the server side?

Oh, but you do have the data in non-encrypted form then? That being the case, clear the contents of data/{username}/files[_*]/, then run occ files:scan, then one of the two;

either synchronize it back to the server, or; copy it manually (i.e., scp or similar) to data/{username}/files/, chown/chmod it as needed, and run occ files:scan again.

1 Like

Okay, thank you.

I’m not sure, if I understand it right, especially “then one of the two;”

  1. As I understand, I can wipe the files under data/{username}/files[_*]/ on the server, and then (to reflect this step in the database accordingly) run occ files:scan to empty the entries in the database as well, right?
    Then I can restore the unencrypted intact files to the client nextcloud folder and let them sync to the server?
    To clean things up I then can run occ files:scan again?

  2. Alternatively I can also ssh/scp the files directly to the server, chown, chmod them as needed and run then occ files:scan?

I think I’ll go with 1) then …

If you use synchronization or upload-through-nextcloud method, you do NOT need to run files:scan a second time. That is only if you dump files directly into the directory, which is usually faster and easier and more reliable than synchronizing.

I’d honestly recommend NOT using a synchronization approach, since synchronization typically works in two directions, so deleting the files from the server could prompt your synchronization software to delete the backup.

The first half of what you have listed as 1 is something you MUST do regardless of what option you choose to copy data BACK to the server.

1 Like

It seems, that I enabled encryption on august 11th in 2020. I noticed, that 518 files were affected, only the newer ones since then.

So decryption didn’t worked on them. I guess the other 3890 files never were encrypted.

What I did so far with the encryption module, due to the nag-screen and error messages:

  1001  16:39   ~/occ.sh
  1002  16:40   ~/occ.sh integrity:check-core
  1003  16:42   ~/occ.sh encryption:status
  1004  16:42   ~/occ.sh encryption
  1005  16:42   ~/occ.sh encryption:list-modules
  1006  16:43   ~/occ.sh encryption:migrate-key-storage-format
  1007  16:48   ~/occ.sh encryption:disable
  1008  16:49   ~/occ.sh encryption:enable
  1009  16:49   encryption:set-default-module
  1010  16:49   ~/occ.sh encryption:set-default-module
  1011  17:14   ~/occ.sh
  1012  17:14   ~/occ.sh app:list
  1013  17:15   ~/occ.sh app:disable text
  1014  17:16   ~/occ.sh app:disable files_external
  1015  17:28   pushd /usr/local/www/nextcloud
  1016  17:28   vi config/config.php
  1017  17:29   nano config/config.php
  1018  17:29   nano config/config.php
  1019  17:29   pkg install nano
  1020  17:29   nano config/config.php
  1021  17:34   ~/occ.sh encryption:scan:legacy-format
  1022  18:32   ~/occ.sh encryption:set-default-module OC_DEFAULT_MODULE
  1023  18:33   ~/occ.sh encryption:
  1024  18:34   ~/occ.sh encryption:status
  1025  18:34   ~/occ.sh encryption:disable
  1026  18:54   tail /var/log/nginx/error.log
  1027  18:55   tail /var/log/nginx/access.log
  1028  18:55   grep -rn MKCOL /var/log/nginx/
  1029  18:56   grep -rn MKCOL /var/log/nginx/ | grep -v access.log
  1030  19:16   ~/occ.sh encryption:enable
  1031  19:16   ~/occ.sh
  1032  19:17   ~/occ.sh encryption:list-modules
  1033  19:17   ~/occ.sh encryption:set-default-module
  1034  19:26   ~/occ.sh encryption:decrypt-all
  1035  21:45   ~/occ.sh encryption:list-modules
  1036  21:45   ~/occ.sh encryption:status
  1037  21:46   ~/occ.sh maintenance:mode --off

Maybe disabling and re-enabling encryption did harm the files? I expect a warning or something like that, that the process of occ encryption:decrypt-all will only start, if the prerequisite is satisfied.

So, I may delete the 518 files, then rescan with occ files:scan and recover those files from the veeam backup by a one-liner or script on the client and let sync them.

$ time find /cygdrive/c/Users/floogy/Nextcloud -iname "*" | while read -r file; do echo -e "$file": "$(od -A x -t x1z -v "$file" | head -n1 | grep -o oc_encryp)" "$(file -b "$file") "$(stat -c "Type(F):%F %s bytes | %U(%u):%G(%g) | %A (%a) | Birth(w):%.19w, Access(x):%.19x, modified(y):%.19y, changed(z):%.19z" "$file")"" 2>&1 | tee -a  encrypted_nextcloud_files_20210421.log; done

$ egrep oc_encryp encrypted_nextcloud_files_20210421.log | awk -F'):' '{print $1" "$6}' | cut -d, -f 1 | sort -t: -k2

$ wc -l encrypted_nextcloud_files_20210421.log
   4408 encrypted_nextcloud_files_20210421.log
   4408 insgesamt
   
$ egrep -v oc_encryp encrypted_nextcloud_files_20210421.log | wc -l
3890

$ egrep oc_encryp encrypted_nextcloud_files_20210421.log | wc -l
518