Explain NextCloudPi restoration w/ data

Support intro

Sorry to hear you’re facing problems. :slightly_frowning_face:

The community help forum (help.nextcloud.com) is for home and non-enterprise users. Support is provided by other community members on a best effort / “as available” basis. All of those responding are volunteering their time to help you.

If you’re using Nextcloud in a business/critical setting, paid and SLA-based support services can be accessed via portal.nextcloud.com where Nextcloud engineers can help ensure your business keeps running smoothly.

Getting help

In order to help you as efficiently (and quickly!) as possible, please fill in as much of the below requested information as you can.

Before clicking submit: Please check if your query is already addressed via the following resources:

(Utilizing these existing resources is typically faster. It also helps reduce the load on our generous volunteers while elevating the signal to noise ratio of the forums otherwise arising from the same queries being posted repeatedly).

The Basics

  • Nextcloud Server version:

    • NextCloud 30.0.1.2
  • Operating system and version:

    • NextCloudPi v1.55.4
    • Armbian-unofficial 24.8.2 Bookworm \l . 6.6.45-current-bcm2711 (aarch64)
  • Web server and version:

    • Apache/2.4.65 (Debian)
  • Reverse proxy and version _(e.g. nginx 1.27.2)

    • N/A

    PHP version:

    • PHP 8.3.12 (cli)
  • Are you using CloudfIare, mod_security, or similar?

    • No

Summary of the issue you are facing:

First of all, I am new to Linux and Nextcloud, and I am slower than a disgruntled DMV employee. Talk to me like I am 5.
Moving on: I am testing Nextcloud restoration processes. Scenario: Data is stable and stored in an external hard drive connected directly to my Pi via USB, but the NextCloudPi OS crashes and I need to do a fresh install. What is the simplest way to tell my new Nextcloud installation to use the existing database and data on my external hard drive?

Steps to replicate it:

  1. Built NextCloudPi on micro SD card Alpha. Booted pi with mirco SD Alpha to fresh NectCloudPi installation, and activated Nextcloud.

  2. Mounted to sda1: /media/USB-HDD

  3. Mounted to sda2: /media/USB-Backups

  4. Used nc-datadir to move the data directory to /media/USB-HDD/ncdata

  5. Used nc-database to move the database to /media/USB-HDD/ncdatabase

  6. Created Nextcloud user “Hardvark” and uploaded like 10 GB of test data.

  7. Exported NextCloudPi configs to /media/USB-Backups using nc-export-ncp

  8. Used nc-backup with “Include data” checked, and stored the backup in /media/USB-Backups/ncp-backups

  9. Shut down the pi, and yoinked mirco SD Alpha.

  10. Built NextCloudPi on micro SD card Beta. Booted pi with mirco SD Beta to fresh NectCloudPi installation, and activated Nextcloud.

  11. Mounted to sda1: /media/USB-HDD

  12. Mounted to sda2: /media/USB-Backups

  13. Used nc-import-ncp to restore configurations. While importing configurations, I get an error that my database is invalid, and to make sure that .ncdata is in the root of my data folder.

  14. Attempt to log into “Hardvark,” but cannot. But I can log into the default ncp user of the new installation on Beta.

I see this error reported in the forums here and there, but I am not finding a real solution or explanation. I think the leading hypothesis is a permissions issue with the /media/USB-HDD/ncdata/data folder.

My understanding of nc-restore is that we are wanting to restore a broken database or corrupted data etc., unpacking our .tar backup to the directory that our Nextcloud instance is currently using. However, there is nothing wrong with my data or database. I don’t want to make a new copy of it. I just want to point my new Nextcloud install to existing data.

If the issue is a permissions issue, I am unsure how to go about resetting ownership properly. Running

$ ls -a -l /media/USB-HDD/ncdata/data

I get

drwxr-x--- 1 www-data www-data    182 Nov  1 23:23 .
drwxr-xr-x 1 root     root          8 Nov  1 16:08 ..
drwxrwxr-x 1 www-data www-data     60 Nov  9  2024 appdata_ocut2g9g78ea
-rw-r--r-- 1 www-data www-data    542 Nov  1 15:54 .htaccess
-rw-r--r-- 1 www-data www-data      0 Nov  1 15:54 index.html
-rw-r--r-- 1 www-data www-data     52 Nov  1 15:54 .ncdata
drwxr-xr-x 1 www-data www-data     20 Nov  9  2024 ncp
drwxr-xr-x 1 www-data www-data     94 Nov  1 15:55 ncp-update-backups
-rw-r----- 1 www-data www-data 351970 Nov  1 23:59 nextcloud.log
drwxrwxr-x 1 www-data www-data     64 Nov  8  2024 .opcache
drwxrwxr-x 1 www-data www-data      0 Nov  1 15:55 tmp

So .ncdata exists, and www-data is listed as the owner. (Ignore the weird dates, I had some timesync issues before.)

Logs

The following is an excerpt of the notable error that was output during nc-import-ncp in step 13.

Running nc-datadir
INFO: /media/USB-HDD/ncdata is not empty. Creating backup /media/USB-HDD/ncdata-11-01-25.1762031302
Maintenance mode enabled
moving data directory from /opt/ncdata/data to /media/USB-HDD/ncdata/data...
BTRFS filesystem detected
Create subvolume '/media/USB-HDD/ncdata'
{"reqId":"eXNzgGpxbwWiG2M6JoKu","level":2,"time":"2025-11-01T21:08:28+00:00","remoteAddr":"","user":"--","app":"no app in context","method":"","url":"--","message":"Temporary directory /opt/ncdata/data/tmp is not present or writable","userAgent":"--","version":"30.0.1.2","data":[]}
Nextcloud is in maintenance mode, no apps are loaded.
Commands provided by apps are unavailable.
Your data directory is invalid.
Ensure there is a file called ".ncdata" in the root of the data directory. It should have the content: "# Nextcloud data directory"

Cannot create "data" directory.
This can usually be fixed by giving the web server write access to the root directory. See https://docs.nextcloud.com/server/30/go.php?to=admin-dir_permissions

An unhandled exception has been thrown:
Exception: Environment not properly prepared. in /var/www/nextcloud/lib/private/Console/Application.php:137
Stack trace:
#0 /var/www/nextcloud/console.php(81): OC\Console\Application->loadCommands()
#1 /var/www/nextcloud/occ(33): require_once('...')
#2 {main}Nextcloud is in maintenance mode, no apps are loaded.
Commands provided by apps are unavailable.
System config value logfile set to string /media/USB-HDD/ncdata/data/nextcloud.log
Nextcloud is in maintenance mode, no apps are loaded.
Commands provided by apps are unavailable.
System config value tempdirectory set to string /media/USB-HDD/ncdata/data/tmp
Installing template 'php/opcache.ini.sh'...
Maintenance mode disabled
Installing template 'ncp-metrics.cfg.sh'...
The NC data directory has been moved successfully.

There is surely something really simple that I am missing here.

Maybe step 4 and 5 have to be repeated for your beta setup, on second device set.

Hello geoW. :slight_smile: Thank you for the feedback.

From what I can tell, nc-import-ncp automatically runs nc-datadir. But looking back at logs, I didn’t see nc-database ran when importing configs.

I went ahead and tried manually running nc-datadir. This produced the same log/error noted in my post.

Your data directory is invalid.
Ensure there is a file called ".ncdata" in the root of the data directory. It should have the content: "# Nextcloud data directory"

Running nc-database just returns

Running nc-database
/media/USB-HDD/ncdatabase is not empty
Done. Press any key...

Maybe the database is part of the problem, too? Maybe I need to do a manual install of Nextcloud on Pi OS Lite so that I can better understand what’s happening. :frowning:

You did it right in your first (alpha) setup. but stoped to early in your second (beta), database and data dir are at default location, that doesn’t match with the backup file.
You need an empty usb-drive and on your beta do the steps 2-5, you then may try to get the backup file in place and try to restore.

I want to tell Nextcloud to point to an existing set of data without migrating a bunch of data onto, say, a blank USB drive. I feel like nc-restore is overkill if my data is still stable and accessible on USB-HDD.

I also don’t understand how my Nextcloud instance knows where my database is. My Alpha config file says:

  'datadirectory' => '/media/USB-HDD/ncdata/data',
  'dbtype' => 'mysql',
  'version' => '30.0.1.2',
  'overwrite.cli.url' => 'https://nextcloudpi/',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'ncadmin',
  'dbpassword' => [OMITTED-ALPHA],
  'installed' => true,
  'instanceid' => 'ocut2g9g78ea',

There is no mention of the path to my database: /media/USB-HDD/ncdatabase. If I knew how to point my Beta to the right database, I was going to try replace the dbpassword on my Beta with the dbpassword on my Alpha. Idk if this would get around the error concerning no .ncdata though.

Try sudo ncp-config → system → nc-info

pi@nc-rspios5:~ $ sudo ncp-config
Running nc-info
Gathering information...
NextcloudPi version  v1.55.4
NextcloudPi image    NextcloudPi_06-01-25
OS                   Debian GNU/Linux 12. 6.12.47+rpt-rpi-2712 (aarch64)
automount            no
USB devices          none
datadir              /opt/ncdata/data
data in SD           yes
data filesystem      ext2/ext3
data disk usage      6.5G/14G
rootfs usage         6.5G/14G
swapfile             /var/swap

dbdir                /var/lib/mysql

Nextcloud check      ok
Nextcloud version    31.0.10.2
HTTPD service        up
PHP service          up
MariaDB service      up
Redis service        up
HPB service          up
Postfix service      up
Internet check       ok
Public IPv4          ***
Public IPv6          ***
Port 80              closed
Port 443             closed
IP                   192.168.2.46
Gateway              192.168.2.1
Interface            eth0
Certificates         none
NAT loopback         no
Uptime               9days


You should run Lets Encrypt for trusted encrypted access

You should open your ports for Lets Encrypt and external access
Done. Press any key...

pi@nc-rspios5:~ $
OS                   Armbian-unofficial 24.8.2 Bookworm \l . 6.6.45-current-bcm2711 (aarch64)
automount            yes
USB devices          sda sdb
datadir              /media/USB-HDD/ncdata/data
data in SD           no
data filesystem      btrfs
data disk usage      13G/466G
rootfs usage         4.8G/58G
swapfile             /var/swap
dbdir                /var/lib/mysql
Nextcloud check      ok
Nextcloud version    30.0.1.2
HTTPD service        up
PHP service          up
MariaDB service      up
Redis service        up
HPB service          up
Postfix service      up
Internet check       ok
Public IPv4          46.xxx.xxx.xxx
Public IPv6          not found
Port 80              closed
Port 443             closed
IP                   192.168.4.219
Gateway              192.168.4.1
Interface            end0
Certificates         none
NAT loopback         no
Uptime               2min

This is great, but do we know how nc-config populates that dbdir value? I don’t know how to find these scripts and open them myself. I guess I was looking for an easy place to edit the value, like in config.php, to change where Nextcloud looks for opening its database manually.

sudo vi /var/www/nextcloud/config/config.php
You will see

'datadirectory' => '/opt/ncdata/data',
  'dbtype' => 'mysql',
  'version' => '31.0.10.2',
  'overwrite.cli.url' => 'https://nc-rspios5/',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,

But that is beyond my knowledge at time, we have to read the docs.

Btw, I have discovered that the Nextcloud System Info → dbdir comes from a file that, for me, is located in /etc/mysql/mariadb.conf.d/90-ncp.cnf.

#Use 'super user do' to access root
sudo su
#Use nano or preferred editor to open 90-ncp.cnf
nano /etc/mysql/mariadb.conf.d/90-ncp.cnf
#Change datadir value
#datadir = /path/to/database
#For me, my database is in /media/USB-HDD/ncdatabase
datadir = /media/USB-HDD/ncdatabase
#CTRL+X   - to exit
#Y   - to save
#ENTER   - to confirm save with listed file name, overwriting the previous file

I have also discovered that simply changing that file to the path of my Alpha database and change the dbpassword in /var/www/nextcloud/config/config.php allows me to access my Alpha database.

#as root
nano /var/www/nextcloud/config/config.php
#Change dbpassword to match the dbpassword previously used for the intended database. For me, I found my dbpassword in my configs from my Alpha set-up.

Not sure it was necessary, or if it even did anything, but I tried to run this command purely off of a hunch.

systemctl restart mysql

My terminal took a solid 30 seconds to become responsive again, but after that…

Voila, I was able to access all of my users and folders again from test set-up Alpha.

BUT I noticed afterwards that if you reboot your server (in my case, my pi), the dbpassword value in config.php resets to the original value. Fascinating. I don’t know why that happens at the moment. I don’t understand a lot of things regarding databases or linux at the moment.

1 Like

I have found that the best direct solution for what I was seeking was to create a backup without data. If your data is valid, accessible, and stable, and you have a backup consisting of just your database and directory, then recovering that should be enough without transferring terabytes of data. On another hand, there doesn’t seem to be a way to automatically create backups of a database and a separate backup for data. It would be nice to see this pursued, or maybe I will learn to write a script to make it happen on my system.

I never figured out how to adjust things so that my dbpassword value doesn’t reset after reboot.