Restore NCP with just "ncdata" possible?

Mageunic · July 19, 2021, 5:02pm

Hi @citizenserious , check if www-data user avec read permission to /media/data and /media
if yes check nextcloud.log files for more details about the internal error.

citizenserious · July 19, 2021, 5:39pm

@Mageunic I did set the “www-data” users permission, so:
drwxr-xr-x 3 www-data www-data for /media
drwxr-xr-x 1 www-data www-data for /media/data
drwxrwx— 1 www-data www-data for /media/data/ncdata

it still shows me the same errors, the log file doesn’t log anything with a timestamp from today, so I reset the location of the logfile back to default (with config.php) to “/var/www/nextcloud/data/nextcloud.log” to exclude permission write problems for the log, but there is still no new entry emerging even after several restarts and maintenance mode changes

Mageunic · July 19, 2021, 5:41pm

@citizenserious try this line: sudo -u www-data php /var/www/nextcloud/occ maintenance:repair

which error have you ?

citizenserious · July 19, 2021, 5:42pm

thats the output:

An unhandled exception has been thrown:
Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurre Stack trace:
#0 /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php(1486): OC\DB\Con #1 /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php(1014): Doctrine\ #2 /var/www/nextcloud/lib/private/DB/Connection.php(226): Doctrine\DBAL\Connecti #4 /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php(287): Doctrin #5 /var/www/nextcloud/lib/private/AppConfig.php(345): OC\DB\QueryBuilder\QueryBu #6 /var/www/nextcloud/lib/private/AppConfig.php(110): OC\AppConfig->loadConfigVa #7 /var/www/nextcloud/lib/private/AppConfig.php(301): OC\AppConfig->getApps() #8 /var/www/nextcloud/lib/private/legacy/OC_App.php(965): OC\AppConfig->getValue #9 /var/www/nextcloud/lib/private/Server.php(682): OC_App::getAppVersions()
#10 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(155) #11 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\ #12 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(122) #13 /var/www/nextcloud/lib/private/ServerContainer.php(137): OC\AppFramework\Uti #14 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): #15 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(177) #16 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(155) #17 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(114): OC\ #18 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(122) #19 /var/www/nextcloud/lib/private/ServerContainer.php(137): OC\AppFramework\Uti #20 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): #21 /var/www/nextcloud/lib/private/Server.php(1031): OC\AppFramework\Utility\Sim #22 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(155) #23 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\ #24 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(122) #25 /var/www/nextcloud/lib/private/ServerContainer.php(137): OC\AppFramework\Uti #26 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): #27 /var/www/nextcloud/lib/private/Server.php(1990): OC\AppFramework\Utility\Sim #28 /var/www/nextcloud/lib/private/Files/View.php(119): OC\Server->getLockingPro #29 /var/www/nextcloud/lib/private/Server.php(430): OC\Files\View->__construc #30 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(155) #31 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\ #32 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(122) #33 /var/www/nextcloud/lib/private/ServerContainer.php(137): OC\AppFramework\Uti #34 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): #35 /var/www/nextcloud/lib/private/Server.php(1353): OC\AppFramework\Utility\Sim #36 /var/www/nextcloud/lib/base.php(588): OC\Server->boot()
#37 /var/www/nextcloud/lib/base.php(1076): OC::init()
#38 /var/www/nextcloud/console.php(49): require_once(’/var/www/nextcl…’)#39 /var/www/nextcloud/occ(11): require_once(’/var/www/nextcl…’) d in the driver: SQLSTATE[HY000] [1045] Access denied for user ‘ncadmin’@‘localh ost’ (using password: YES) in /var/www/nextcloud/lib/private/DB/Connection.php:8 5
nection->connect()
DBAL\Connection->getWrappedConnection()
on->executeQuery(‘SELECT * FROM ...', Array, Array, NULL) #3 /var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php(210): OC \DB\Connection->executeQuery('SELECT * FROM …’, Array, Array)
e\DBAL\Query\QueryBuilder->execute()
ilder->execute()
lues()

s(false, ‘installed_versi…’)
: OC\Server->OC{closure}(Object(OC\Server))
AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility{closure}(Object(P imple\Container))
: Pimple\Container->offsetGet(‘OC\Memcache\Fac…’)
lity\SimpleContainer->query(‘OC\Memcache\Fac…’, false)
OC\ServerContainer->query(‘OC\Memcache\Fac…’)
: OC\AppFramework\Utility\SimpleContainer->get(‘OC\Memcache\Fac…’)
: OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility{closure}(Obj ect(OC\Server))
AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility{closure}(Object(P imple\Container))
: Pimple\Container->offsetGet(‘OCP\ICacheFacto…’)
lity\SimpleContainer->query(‘OCP\ICacheFacto…’, false)
OC\ServerContainer->query(‘OCP\ICacheFacto…’)
pleContainer->get(‘OCP\ICacheFacto…’)
: OC\Server->OC{closure}(Object(OC\Server))
AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility{closure}(Object(P imple\Container))
: Pimple\Container->offsetGet(‘OCP\Lock\ILocki…’)
lity\SimpleContainer->query(‘OCP\Lock\ILocki…’, false)
OC\ServerContainer->query(‘OCP\Lock\ILocki…’)
pleContainer->get(‘OCP\Lock\ILocki…’)
vider()
t()
: OC\Server->OC{closure}(Object(OC\Server))
AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility{closure}(Object(P imple\Container))
: Pimple\Container->offsetGet(‘OC\Files\Node\H…’)
lity\SimpleContainer->query(‘OC\Files\Node\H…’, false)
OC\ServerContainer->query(‘OC\Files\Node\H…’)
pleContainer->get(‘OC\Files\Node\H…’)
r>

Mageunic · July 19, 2021, 5:47pm

@citizenserious you have a DB connection error… go into config.php and note the information about the database and its user. check if the user of databe exist and password is correct.
sudo mysql -u user -p ( replace user by yours)

check if database can be used by the user :
SHOW databases;

send the outpout of this comand pls

citizenserious · July 19, 2021, 6:10pm

you where right the password in the config.php was wrong, I changed it to the right one I think.
The user for the database by default is “ncadmin”.

When executing sudo -u www-data php /var/www/nextcloud/occ maintenance:repair

It shows me this now:

Cannot write into “config” directory!
This can usually be fixed by giving the webserver write access to the config directory

Or, if you prefer to keep config.php file read only, set the option “config_is_read_only” to true in it.
See https://docs.nextcloud.com/server/21/go.php?to=admin-config

So the next problem was that the webserver needed write access… so I changed the permission for the config.php
from
-rwxr-xr-x 1 root root for config.php
to
-rwxr-xr-x 1 www-data www-data for config.php

Then I initiated -u www-data php /var/www/nextcloud/occ maintenance:repair again and it WORKED Oh my god, I am so happy right now.
When navigating to my server page it shows me a bunch of app updates now, can I simply install them? I hope it will work afterwards but I will wait for your reply. I don’t understand though why the “www-data” user works even when in the config.php the user for the database is “ncdata”. But if it works I am fine with it

Mageunic · July 19, 2021, 6:20pm

@citizenserious you can install the updates. there is 2 users, one is linux user ( www-data) used by nginx/apache to access to the filesystem and the other is mysql user for the database. all write/ read /execute permissions is related to linux user ( www-data).

www-data is the owner of nextcloud folder ( and subfolder) ? if not , please change it to www-data

sudo chown -R www-data:www-data /var/www/nextcloud/

citizenserious · July 19, 2021, 6:32pm

@Mageunic Just awesome, I can login, everything seems to be working except for my files I got the Error “Invalid private key for encryption app. Please update your private key password in your personal settings to recover access to your encrypted files.”

Anyway you are a absolute genius! Thank you so so very much.

The last issue must also be possible to solve somehow.
If everything works then, I will write a summary how to proceed when manually trying to restore an instance, I am sure someone else will encounter this problem. I would be very happy if you are willing to proofread that summary then (:

The /var/www/nextcloud/ and its subfolder are owned by www-data.

Mageunic · July 19, 2021, 6:37pm

@citizenserious I suggest you to decrypt your data and make a backup of it :

sudo -u www-data php /var/www/nextcloud/ encryption:decrypt-all

citizenserious · July 19, 2021, 7:16pm

@Mageunic What is odd is that when I reboot my instance, the password for the database in the config.php changes so I have to change it again and initiate maintenance:repair again.
Then it work’s again, until a restart.

When executing sudo -u www-data php /var/www/nextcloud/ encryption:decrypt-all nothing happens.

Mageunic · July 19, 2021, 7:20pm

@citizenserious this is strange maybe nextcloudpi software ( not nextcloud) change the password, i think you have too change the user database password to have no issue.

sudo mysql -u root -p
SET PASSWORD FOR ‘ncadmin’@localhost = PASSWORD(“newpassword”);
( change newpassword by yours)
EXIT;

my bad, there is a little mistake in my line : sudo -u www-data php /var/www/nextcloud/occ encryption:decrypt-all

I forgot to put occ at the end of the path

citizenserious · July 20, 2021, 4:03pm

hi @Mageunic thanks again.
The password change doesn’t work somehow… the weird thing is, that now I can put in whatever I want when sudo mysql -u root -p is asking me for a password. It just accepts anything now, maybe because of the try to change it?
When trying to change it, there just appears the next command line but no output whatsoever.
When restarting I get the same Internal server error because of the change of the database password in the config file.

I tried decrypting my files. I got the error “In Crypt.php line 505: Bad Signature”
Line 505 from “/var/www/nextcloud/apps/encryption/lib/Crypto/Crypt.php”, is throw new GenericEncryptionException('Bad Signature', $this->l->t('Bad Signature')); inside a if block.

Thats the block:

/**
 * check for valid signature
 *
 * @param string $data
 * @param string $passPhrase
 * @param string $expectedSignature
 * @throws GenericEncryptionException
 */
private function checkSignature($data, $passPhrase, $expectedSignature) {
	$enforceSignature = !$this->config->getSystemValue('encryption_skip_signature_check', false);

	$signature = $this->createSignature($data, $passPhrase);
	$isCorrectHash = hash_equals($expectedSignature, $signature);

	if (!$isCorrectHash && $enforceSignature) {
		throw new GenericEncryptionException('Bad Signature', $this->l->t('Bad Signature'));
	} elseif (!$isCorrectHash && !$enforceSignature) {
		$this->logger->info("Signature check skipped", ['app' => 'encryption']);
	}
}`

but I don’t know where everywhere I have to change the signature to the old one.

Mageunic · July 21, 2021, 11:03am

@citizenserious this is strange for the second comand, I will try to find a solution.
To change mysql user password, try this : https://linuxconfig.org/how-to-change-mariadb-user-password

citizenserious · July 21, 2021, 2:13pm

@Mageunic the website is awesome, I changed both the root and user password. But nevertheless the password gets changed in the config file with every reboot. So as a workaround I just used the password it changes to every time, now it works. Maybe I try to look into this particular phenomena later.

Edit:
I also tried to replace the Crypt.php with the old one, and give it the right permission, but it didn’t work. And I also tried to replace the whole directory “/var/www/nextcloud/apps/encryption” give it the right permission, but it also didn’t work.

That’s the log file with the old encryption directory:

[no app in context] Error: OCP\Encryption\Exceptions\GenericEncryptionException: Bad Signature at <>

/var/www/nextcloud/apps/encryption/lib/Crypto/Crypt.php line 480
OCA\Encryption\Crypto\Crypt->checkSignature("*** sensitive parameters replaced ***")
/var/www/nextcloud/apps/encryption/lib/Crypto/Crypt.php line 429
OCA\Encryption\Crypto\Crypt->symmetricDecryptFileContent("*** sensitive parameter replaced ", " sensitive parameter replaced ***", “AES-256-CTR”, 0)
/var/www/nextcloud/apps/encryption/lib/KeyManager.php line 237
OCA\Encryption\Crypto\Crypt->decryptPrivateKey("*** sensitive parameters replaced ***")
/var/www/nextcloud/apps/encryption/lib/Users/Setup.php line 64
OCA\Encryption\KeyManager->validateMasterKey()
/var/www/nextcloud/apps/encryption/lib/AppInfo/Application.php line 57
OCA\Encryption\Users\Setup->setupSystem()
/var/www/nextcloud/apps/encryption/appinfo/app.php line 38
OCA\Encryption\AppInfo\Application->setUp(OC\Encryption\Manager {})
/var/www/nextcloud/lib/private/legacy/OC_App.php line 303
require_once("/var/www/nextcl … p")
/var/www/nextcloud/lib/private/legacy/OC_App.php line 185
OC_App::requireAppFile(OCA\Encryption\AppInfo\Application {})
/var/www/nextcloud/lib/private/legacy/OC_App.php line 138
OC_App::loadApp(“encryption”)
/var/www/nextcloud/lib/base.php line 979
OC_App::loadApps()
/var/www/nextcloud/index.php line 37
OC::handleRequest()

The log file with the default encryption directory is the exact same, so nothing changes when replacing the encryption directory…

Mageunic · July 21, 2021, 5:53pm

@citizenserious
have you try to disable and enable encryption module ?

maybe this can help but not sure for your issue …

citizenserious · July 21, 2021, 6:13pm

@Mageunic I don’t think that the server encryption still has a legacy encryption module, there was an update once. Anyway I deactivated the encryption in the settings and the app, then enabled it again like you suggested but unfortunately it did not work.
My files were not on legacy anymore I am really sure about it because I disabled the legacy extension while it was still working, and I had to make sure first that there are no files left with the legacy encryption. Thats what ncc encryption:scan:legacy-format was for.

Mageunic · July 22, 2021, 3:42pm

@citizenserious I don’t know how to solve your issue at the moment…

citizenserious · July 22, 2021, 5:11pm

@Mageunic you already helped me a lot. And the main problems are solved. Thank you again. I will try to write a summary soon (: Also I got the most important data synchronized with other devices, so I will put my instance back to work again now. And take care about the rest later.
Thanks again mate, you safed me so much trouble.

citizenserious · July 23, 2021, 12:33pm

How to manually restore an nextcloudpi instance (with enabled encryption): many thanks to @Mageunic

What you need:

/var/www
/var/lib/mysql
your data directory, by default it is at: /var/www/nextcloud/data

Preparation:

Setup a new NCP instance
update/upgrade
activate the instance and note the passwords
update nextcloud pi (sudo ncp-config). Update nextcloud inside the nextcloud itself.
$ sudo apt-get install imagemagick php-imagick
$ ncc db:add-missing-indices
$ ncc db:convert-filecache-bigint

Restore the database:

remove mysql: $ rm -rf /var/lib/mysql
copy your database: $ cp -r -p /location/of/database /var/lib/mysql
set the right permission for the database: $ sudo chown -R mysql:mysql /var/lib/mysql/

Restore www:
1.0 remove www: $ rm -rf /var/www
2.0 copy your www: $ cp -r /location/of/www /var/www
3.0 setting permissions:
3.1: $ sudo chown -R www-data:www-data /var/www/nextcloud
3.2: $ sudo chown -R www-data:www-data /var/www/ncp-previewgenerator
3.3: $ sudo chown -R www-data:www-data /var/www/ncp-web
3.4: $ sudo chmod 711 /var/www/nextcloud

— REBOOT – $ sudo reboot now

look for the database password for ncadmin in /var/www/nextcloud/config/config.php
by the way → make sure the location of your data directory (ncdata) is set right in the config.php (default location of database: /var/www/nextcloud/data)
Change MariaDB Passwords for ncadmin to the password discovered (nicely described here: https://linuxconfig.org/how-to-change-mariadb-user-password)
- § sudo mariadb -u root -p
  - it asks for a password, you can type anything here, it doesn’t mater, just hit enter.
    MariaDB [(none)]> use mysql;
    MariaDB [mysql]> ALTER USER 'ncadmin'@'localhost' IDENTIFIED BY 'the_password_from_the_config.php'
    MariaDB [mysql]> FLUSH PRIVILEGES;
    MariaDB [mysql]> exit

Thats it, your instance should work with all your encrypted files and settings you had before.

Mageunic · July 23, 2021, 1:24pm

@citizenserious I forgot to say that the other installation that will be used to restore the server must be in the same version as the one you want to restore, otherwise you may encounter problems or not be able to access the server.