How to use data from HD?

Hello,

I had an issue with my NCP instance who crashed for no apparent reason.
Long story short, I was proposed to reinstall NCP on a new SD card, which I did, and it worked well.
Then I tried to restore the backup I made earlier (after the crash), but it seemed that this backup was corrupted in some way (ie. it crashed nextcloud and everything was worse). So I decided to flash the SD card again, to reinstall NCP from scratch

Now, I have under /media/myCloudDrive the new ncdata and 2 old ncdata-xxxx, one with my actual data (ncdata-good) and one with the other ncdata (ncdata-useless) from the failed restore try.

2 questions:

  • How can I link the data from the old ncdata-good to ncdata ? Should I just move them under ncdata/username/files ? Should I use nc-scan, or nc-fix-permission ? Could you give me the exact command for recovering these data ? I guess I can’t just rename ncdata-good to ncdata, or could I ?

  • How can I remove ncdata-useless ? sudo rm -r ncdata-useless gives me Permission denied.

Thanks

Move the empty datadir to ncdata.empty, and then move the good datadir with your data to ncdata (or wherever your NC instance is pointing to, you can see it in ncp-web). Then run nc-scan from the web interface.

In order to remove an old datadir try sudo btrfs su del ncdata.bad

Ok, and how it will handle the fact that in the good datadir, there might be users that are not yet in the empty one ? Do I need the recreate the exact same users ? And how it will handle changes that occurred meanwhile (that are not on the datadir)?

Thanks

Hello,

So I’ve done that, and it scanned the user that I created again. However, now I can’t log to my NC instance…:

Internal Server Error

The server encountered an internal error and was unable to complete your request.
Please contact the server administrator if this error reappears multiple times, please include the technical details below in your report.
More details can be found in the server log.

on the local version (nextcloudpi.local), the page somehow loads, but without the CSS.

Here is the ncp-report:

NextCloudPi diagnostics

NextCloudPi version  v0.64.10
NextCloudPi image    NextCloudPi_10-05-18
distribution         Raspbian GNU/Linux 9 \n \l
automount            yes
USB devices          sda 
datadir              /media/USBdrive/ncdata
data in SD           no
data filesystem      btrfs
data disk usage      328G/932G
rootfs usage         2.1G/15G
swapfile             /var/swap
dbdir                /var/lib/mysql
Nextcloud check      ok
Nextcloud version    14.0.1.1
HTTPD service        up
PHP service          up
MariaDB service      up
Redis service        up
Postfix service      up
internet check       ok
port check 80        open
port check 443       open
IP                   ***REMOVED SENSITIVE VALUE***
gateway              ***REMOVED SENSITIVE VALUE***
interface            eth0
certificates         ***REMOVED SENSITIVE VALUE***
NAT loopback         no
uptime               23:53

Nextcloud configuration

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": {
            "0": "localhost",
            "5": "nextcloudpi.local",
            "7": "nextcloudpi",
            "8": "nextcloudpi.lan",
            "1": "192.168.0.4",
            "4": "vercingeto.freeboxos.fr"
        },
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "14.0.1.1",
        "overwrite.cli.url": "https:\/\/vercingeto.freeboxos.fr\/",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "timeout": 0,
            "password": "***REMOVED SENSITIVE VALUE***"
        },
        "mail_smtpmode": "sendmail",
        "mail_smtpauthtype": "LOGIN",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "overwriteprotocol": "https",
        "maintenance": false,
        "logfile": "\/media\/USBdrive\/ncdata\/nextcloud.log",
        "loglevel": "2",
        "log_type": "file"
    }
}

HTTPd logs

[Thu Oct 25 21:45:55.996147 2018] [proxy_fcgi:error] [pid 4801:tid 1867625504] [client 192.168.0.10:62900] AH01071: Got error 'PHP message: PHP Warning:  ini_set(): A session is active. You cannot change the session module's ini settings at this time in /var/www/ncp-web/index.php on line 41\nPHP message: PHP Warning:  ini_set(): A session is active. You cannot change the session module's ini settings at this time in /var/www/ncp-web/index.php on line 43\nPHP message: PHP Warning:  fopen(ncp-web.cfg): failed to open stream: No such file or directory in /var/www/ncp-web/L10N.php on line 145\nPHP message: PHP Warning:  fopen(ncp-web.cfg): failed to open stream: No such file or directory in /var/www/ncp-web/L10N.php on line 145\nPHP message: NACHO auto\n - auto\nPHP message: NACHO en\n - auto\nPHP message: NACHO de\n - auto\nPHP message: NACHO es\n - auto\nPHP message: NACHO zh\n - auto\n'
[Thu Oct 25 21:45:56.509818 2018] [authz_host:error] [pid 4801:tid 1934767136] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /minified.js failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.509818 2018] [authz_host:error] [pid 4801:tid 1859232800] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /ncp.js failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.511945 2018] [authz_host:error] [pid 4801:tid 1867625504] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /ncp.css failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.527376 2018] [authz_host:error] [pid 4801:tid 1934767136] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /img/loading-small.gif failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.527376 2018] [authz_host:error] [pid 4801:tid 1859232800] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /img/poweroff.svg failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.529093 2018] [authz_host:error] [pid 4801:tid 1884410912] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /img/reboot.svg failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.767191 2018] [authz_host:error] [pid 4801:tid 1934767136] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /img/nc-button.svg failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.767191 2018] [authz_host:error] [pid 4801:tid 1884410912] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /core/img/actions/triangle-s.svg failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.768066 2018] [authz_host:error] [pid 4801:tid 1867625504] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /img/ncp-logo.svg failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.770391 2018] [authz_host:error] [pid 4801:tid 1867625504] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /img/dashboard.svg failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.794159 2018] [authz_host:error] [pid 4801:tid 1934767136] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /img/info-white.svg failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.794160 2018] [authz_host:error] [pid 4801:tid 1884410912] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /img/wizard.svg failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.795253 2018] [authz_host:error] [pid 4801:tid 1867625504] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /img/settings-white.svg failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.861092 2018] [authz_host:error] [pid 4801:tid 1884410912] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /ncp-output.php failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:56.861412 2018] [authz_host:error] [pid 4801:tid 1867625504] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /ncp-launcher.php failed, reason: unable to get the remote host name
[Thu Oct 25 21:45:57.877070 2018] [proxy_fcgi:error] [pid 4801:tid 1884410912] [client 192.168.0.10:62900] AH01071: Got error 'PHP message: PHP Warning:  touch(): Utime failed: Permission denied in /var/www/ncp-web/ncp-output.php on line 81\n'
[Thu Oct 25 21:46:04.532381 2018] [proxy_fcgi:error] [pid 4801:tid 1867625504] [client 192.168.0.10:62900] AH01071: Got error 'PHP message: PHP Warning:  fopen(ncp-web.cfg): failed to open stream: No such file or directory in /var/www/ncp-web/L10N.php on line 145\n'
[Thu Oct 25 21:46:04.581575 2018] [authz_host:error] [pid 4801:tid 1867625504] [client 192.168.0.10:62900] AH01753: access check of 'localhost' to /ncp-launcher.php failed, reason: unable to get the remote host name
[Thu Oct 25 21:46:07.768999 2018] [proxy_fcgi:error] [pid 4801:tid 1867625504] [client 192.168.0.10:62900] AH01071: Got error 'PHP message: PHP Warning:  fopen(ncp-web.cfg): failed to open stream: No such file or directory in /var/www/ncp-web/L10N.php on line 145\n'

Database logs

2018-10-24 21:58:31 1995644912 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2018-10-24 21:58:31 1995644912 [Note] InnoDB: The InnoDB memory heap is disabled
2018-10-24 21:58:31 1995644912 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-10-24 21:58:31 1995644912 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2018-10-24 21:58:31 1995644912 [Note] InnoDB: Compressed tables use zlib 1.2.8
2018-10-24 21:58:31 1995644912 [Note] InnoDB: Using Linux native AIO
2018-10-24 21:58:31 1995644912 [Note] InnoDB: Using generic crc32 instructions
2018-10-24 21:58:31 1995644912 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2018-10-24 21:58:31 1995644912 [Note] InnoDB: Completed initialization of buffer pool
2018-10-24 21:58:31 1995644912 [Note] InnoDB: Highest supported file format is Barracuda.
2018-10-24 21:58:32 1995644912 [Note] InnoDB: 128 rollback segment(s) are active.
2018-10-24 21:58:32 1995644912 [Note] InnoDB: Waiting for purge to start
2018-10-24 21:58:32 1995644912 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.35-80.0 started; log sequence number 3430349
2018-10-24 21:58:32 1430238000 [Note] InnoDB: Dumping buffer pool(s) not yet started
2018-10-24 21:58:32 1995644912 [Note] Plugin 'FEEDBACK' is disabled.
2018-10-24 21:58:32 1995644912 [Note] Server socket created on IP: '127.0.0.1'.
2018-10-24 21:58:32 1995644912 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.1.23-MariaDB-9+deb9u1'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  Raspbian 9.0

The nextcloud logs are quite big, it seems to contain the code of I don’t know what, I put here instead: https://hebdo.framapad.org/p/fzcxmwxi7i

Here is the log of the nc-scan:

nc-scan logs
[ nc-scan ]

Scanning files for 2 users
Starting scan for user 1 out of 2 (Jean)
Exception during scan: Failed to get mimetype id for application/vnd.ms-excel after trying to store it
#0 /var/www/nextcloud/lib/private/Files/Type/Loader.php(86): OC\Files\Type\Loader->store('application/vnd...')
#1 /var/www/nextcloud/lib/private/Files/Cache/Cache.php(353): OC\Files\Type\Loader->getId('application/vnd...')
#2 /var/www/nextcloud/lib/private/Files/Cache/Cache.php(263): OC\Files\Cache\Cache->buildParts(Array)
#3 /var/www/nextcloud/lib/private/Files/Cache/Cache.php(229): OC\Files\Cache\Cache->insert('files/Work/Ed_p...', Array)
#4 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(292): OC\Files\Cache\Cache->put('files/Work/Ed_p...', Array)
#5 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(221): OC\Files\Cache\Scanner->addToCache('files/Work/Ed_p...', Array, -1)
#6 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(438): OC\Files\Cache\Scanner->scanFile('files/Work/Ed_p...', 3, '79404', false, true)
#7 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(406): OC\Files\Cache\Scanner->handleChildren('files/Work/Ed_p...', true, 3, '79404', true, 8218084)
#8 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanChildren('files/Work/Ed_p...', true, 3, '79404', true)
#9 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanChildren('files/Work/Ed_p...', true, 3, '26168', true)
#10 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanChildren('files/Work', true, 3, '264', true)
#11 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(409): OC\Files\Cache\Scanner->scanChildren('files', true, 3, '184', true)
#12 /var/www/nextcloud/lib/private/Files/Cache/Scanner.php(338): OC\Files\Cache\Scanner->scanChildren('', true, 3, '183', true)
#13 /var/www/nextcloud/lib/private/Files/Utils/Scanner.php(245): OC\Files\Cache\Scanner->scan('', true, 3)
#14 /var/www/nextcloud/apps/files/lib/Command/Scan.php(172): OC\Files\Utils\Scanner->scan('/Jean', true, NULL)
#15 /var/www/nextcloud/apps/files/lib/Command/Scan.php(249): OCA\Files\Command\Scan->scanFiles('Jean', '/Jean', false, Object(Symfony\Component\Console\Output\ConsoleOutput), false, true, false)
#16 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(251): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /var/www/nextcloud/core/Command/Base.php(162): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /var/www/nextcloud/3rdparty/symfony/console/Application.php(946): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 /var/www/nextcloud/3rdparty/symfony/console/Application.php(248): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /var/www/nextcloud/3rdparty/symfony/console/Application.php(148): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /var/www/nextcloud/lib/private/Console/Application.php(213): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 /var/www/nextcloud/console.php(96): OC\Console\Application->run()
#23 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#24 {main}
Starting scan for user 2 out of 2 (ncp)
Home storage for user ncp not writable
Make sure you're running the scan command only as the user the web server runs as

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 1404 | 72055 | 00:16:37 |
+---------+-------+--------------+

Thanks for helping !

try running nc-fix-permissions from ncp-web

It did not fix the problem !

So I renamed the file back to their initial name (sudo mv ncdata ncdata_good; sudo mv ncdata.empty ncdata), and now it works. But I don’t have my data then !

you have to copy your files (in datadir/user/files) to ncdata, then fix permissions, then scan