How to move data directory on VM installation under Windows 8.1 host PC

Nextcloud version: 13
Operating system: (Ubuntu 16.04 in Hyper-V VM running on WIndows 8.1 Host PC)
Apache (Apache 2.4.25)
PHP version:(?):

Newbie to Ubuntu and Nextlcoud. New installation, everything working.

Want to move the current data directory to an different hard drive in the PC. Hard drive is currently formatted as NTFS. 3tb drive, but I only want to dedicate 500gb for Nextcloud data.

I’m guessing I need to format 500gb for the data directory, but I don’t know how to do that. So that, I think, is my first question:

  1. How do I format 500gb on a separate attached hard drive to use for Nextlcoud data, and then create the necessary folder/directory?

Second, I have read the following guide on moving the data directory after installation by Michalng:

And this guide to change the data directory using snap (not sure what that means):

  1. Are those two guides still the best method to move the data directory to a new location?

  2. Can any of the above be accomplished using Webmin (which I have installed)?

Nextcloud snap is sort of a container running Nextcloud and all it’s dependencies. Read more about snaps: https://en.m.wikipedia.org/wiki/Snappy_(package_manager)
As the data directory change guide you linked does not contain any database adjustment or file rescan, I guess this is done on the snaps startup automatically.

As you asked, I think you have a regular non containerized Nextcloud instance, where you should follow my guide. There is still no better way I heard of.

Correction: Thought I had found how to move the Nextcloud “data” to a new drive/location using Hyper-V Manager. Turns out that the method described below will move the entire Nextcloud installation. I’ll leave it here because it is still useful, but now I need to go back and use Michalng’s guide to actually move only the Data folder.

I discovered that moving the Nextcloud “installation” to another location is actually rather easy within the Hyper-V Manager.

First, I created a new folder within Windows using File Explorer at the new desired location. I just called it “NC Data”.

Then, I used Hyper-V Manager to create a new Virtual Disk (VHDX) using this guide (selecting the folder I had just created):

Hyper-V How To: Create a Virtual Hard Disk

This gave me the opportunity to define the storage space size.

Next, I “attached” the new VHDX Virtual Disk using this guide:

How To Attach an Existing Virtual Disk (VHD/X) in Hyper-V

Then I “moved” the Nextcloud VM to the new VHDX using this guide:

How to Perform Hyper-V Storage Migration

During the move operation, I had the option to select folders for the target directory (for which I used the new NC Data location), and also the Configuration File, Checkpoints and SmartPaging – all of which I kept on the same disk as the Nextcloud installation, but in a new folder location that I created there.

For anyone using Hyper-V to host their Nextcloud installation, this worked really well to create a new VHDX virtual drive and move the Nextcloud Installation. Hope this may help someone down the road. Really pleased with my setup now.

1 Like

In your guide, Option 1, Item #4, it says:

nano /path/to/nextcloud/config/config.php
’datadirectory’ => ‘/new/path/to/data’,

What is ‘datadirectory’? Do I use that exact text? Or is that a variable placeholder for some other input?

As always, the settings name on the left side, the set value on the right :wink:. Just adjust /new/path/to/data to fit your real desired data directory path.
Admin manual about this (and all other config.php settings): https://docs.nextcloud.com/server/13/admin_manual/configuration_server/config_sample_php_parameters.html#default-parameters

Ok. Got that. Now, I don’t know how to copy/dump/move the sql database. Not sure what the “mysqldump” command does. I’m looking at the directions here: https://mariadb.com/kb/en/library/mysqldump/ – but I don’t understand what I’m reading. Where your directions say:

  1. mysqldump -u -p > /path/to/dbdumymp/dump.sql

Does that copy an existing sql file to a new location? I don’t know where to find the current nextcloud MariaDB database file (sql) that I’m supposed to copy or move. Also don’t know what correct way to enter “-u” . I saw that I can also use “–user=rootuser” . Since I haven’t changed the nextcloud admin name yet, would it still be “ncadmin”?

Ok, looks like that copies the database. I can’t get it to work, but I don’t have any added data so I’m going to skip that step.

Next is:

  1. Adjust "oc_storages"database table to reflect the new data folder location:

$ mysql -u<rootuser> -p
//enter mysql root password [what is the mysql root password for a nextcloud installation?]
$ use <nextclouddb>;
$ update oc_storages set id=‘local::/new/path/to/data/’ where id=‘local::/path/to/data/’; //take care about backslash at the end of path!!
$ quit;

Questions: I was running this as root which has the # symbol. I only get the “$” symbol if I exit root. Is that what the $ means? Now it’s asking for user name (which I entered as “ncadmin”) but it’s saying access denied when I try to use my ncadmin password. Is it different for Mysql? I get ERROR 1084. It also says, “using password YES”. But then it said, “using password NO”.

Also, is “oc_storges” a key word or the name of a database file? Can’t find anything about it anywhere.

I know it may sound like I’m over my head but I think I’ve completed everything correctly up to this point. And, I don’t want to start using nextcloud until I get the data directory moved. The Tech and Me installation I used only configured it for 20gb, so right off I want to increase the size, plus I don’t want the larger data directory on the current SSD drive. The Tech and Me installation doesn’t offer the option to install the data directory separately.

mysqldump exports the database as sql file, which works fine as backup, can be imported to newer and older MySQL and MariaDB versions without issues and theoretically even into other kind of SQL database software, although I never tried this.
The raw database files can be found at /var/lib/mysql in most cases, but you can’t directly copy them as backup. MySQL will show errors when you copy them back, as log time stamps and others do not match with it’s internals.

The $ symbols are indeed irritating here, just ignore them, I just wanted to make clear that these are terminal commands. As the above mysqldump command exports the whole database, not just the Nextcloud parts, it is ever better, in case of MariaDB eventually necessary to do this as root user.

You need to use mysql/mysqldump commands as MySQL internal root user, which is by default as well just root, at least on all distro I use: mysql -uroot -p (no gap after -u, which does the same as --user, -p will ask for a password)
With MariaDB on new versions with default setup, you do not even need to enter user or password at all. MariaDB authenticates with the unix root user itself, so mysql is enough to log into MariaDB console. Just try out what works for you.

This is the name of the database table, where Nextcloud stores it’s data folder path. Just carefully adjust the paths to match your new and old one, be sure to have a slash / at the end of them.

Ok, got passed that part. I remembered seeing “dbpassword” in the config.php file, along with dbuser (which in my case was “oc_ncadmin”. The dbpassword was quite long, so I copied it to a txt file for storage and re-use. That got me me into mysql.

I ran “use” command and used the dbname “nextcloud_db” from the config file.

I’m still stuck at, “update oc_storages” line (item 6). I keep getting a syntax error. Couldn’t find anythign on “oc_storages”. Is that how I enter it exactly “oc_storages” (without the quotes)? And, do I put the directories in single quotes ‘/new/path/’ or leave off the single quotes?

Try:
update oc_storages set id='local::/new/path/to/data/' where id rlike 'local::';
The table contains anyway just this one entry.

€: Dangerous, since the table can indeed contain 2 local:: entries (even that just one is really used) and then this leads to duplicate entries. Stick with command from the HowTo!

Jep oc_storages as is, at least if your database prefix is oc_, which can be checked within config.php as well. The table field values with single quotes as above. In case try to type/correct them manually, as I remember copying from here sometimes leads to this other kind of non-straight quote symbols.

But the table exists, right? Check show tables; within MySQL console after use nextcloud_db;.

Yes, table exists. Now that I understand how to correctly enter the update command, I’m getting the following error:

ERROR 1062 (23000): Duplicate entry ‘local::/dev/sdb/’ for key ‘storages_id_index’

If I shut down and restart the Nextcloud VM, and try to access it via the web browser, I get this message:

Internal Server Error

The server encountered an internal error and was unable to complete your request.
Please contact the server administrator if this error reappears multiple times, please include the technical details below in your report.
More details can be found in the server log.

Not sure how to delete the duplicate db entry, but I think that may fix it. I imagine it was caused by typing the wrong path in failed attempts.

@scooter
Okay I see now the issue with where id rlike 'local::'. In some cases somehow there is more than one entry for the local data folder location, even that of course just one can really be in use (?).

So database backup from before all of this is successfully in place, yes?

Okay then log into MySQL/MariaDB console:

mysql -uroot -p
use nextcloud_db;
select * from oc_storages;

Now you should see all rows inside the table, in my case this showed up:

+---------------------------------------------+------------+-----------+--------------+
| id                                          | numeric_id | available | last_checked |
+---------------------------------------------+------------+-----------+--------------+
| local::/mnt/dietpi_userdata/nextcloud_data/ |          2 |         1 |         NULL |
| home::Micha                                 |          3 |         1 |         NULL |
| local::/mnt/sda/ncdata/                     |          4 |         1 |         NULL |
| home::Marilyn                               |          5 |         1 |         NULL |
+---------------------------------------------+------------+-----------+--------------+

The 1st row local:: contains the default location on fresh Nextcloud installations via DietPi and I remember that I needed to switch it back to my external drive (3rd row). Nowadays we added an option to DietPi to adjust the data folder in front of installation process, but whatever the reason for 2 entries, update oc_storages set id='local::/new/path/to/data/' where id rlike 'local::'; would have now adjusted both and thus produced a duplicate entry.

To remove the first wrong row, I now chose the numeric_id as unique identifier and did the following:

delete from oc_storages where numeric_id=2 limit 1;
select * from oc_storages;
+-------------------------+------------+-----------+--------------+
| id                      | numeric_id | available | last_checked |
+-------------------------+------------+-----------+--------------+
| home::Micha             |          3 |         1 |         NULL |
| local::/mnt/sda/ncdata/ |          4 |         1 |         NULL |
| home::Marilyn           |          5 |         1 |         NULL |
+-------------------------+------------+-----------+--------------+

Nextcloud works well as before, the entries seem to just contain the “possible” data folder locations, which can be then “selected” via config.php entry, or something like this. And sadly there seems to be no cleanup done :wink:.

So in your case there should be two identical rows, just with numeric_id different. Just use the above delete command and replace the 2 from numeric_id=2 with e.g. the higher number of the two duplicate entries.

This is excellent. I actually am more familiar with SQL than Ubuntu/Linux commands. Plus, had to learn the MariaDB quirks (like adding the semicolon each line).

I’ve now got only two entries in the oc_storages table. I’ll double-check everything then report back.

I actually made one other known mistake. I previously moved the entire nextcloud installation to my new “storage” directory, when I intended to move only the “data” directory. Going to move it back to the SSD drive, then check everything again using your guide.

I got the DB fixed. The oc_storages table had three entries. 1 was for home::ncadmin. 2 was for local::/var/ncdata (original) & 3 was for my new data “local::/dev/sdb/ndata”. So I deleted “2” then reindexed the table so that “2” is now the id for local::/dev/sdb/ncdata. Though, now I realize that if some other table references “2” in the oc_storages table that might cause a problem, but I don’t think it will.

My problem now is that I can’t copy the contents from /var/ncdata to /dev/sdb.

I tried creating a new directory on sdb called “ncdata” using mkdir, but I don’t think I got it right. When I use cp, it says “/dev/sdb/ncdata is not a directory”. Guess I don’t know how to create a new directory for the data on my sdb drive.

UPDATE: I think I may have been trying to copy the contents of /var/ncdata to a “mount point” on my sbd location (which I had named “/ncdata”). In other words, I was trying to copy it to /dev/sdb/ncdata. I tried copying the entire directory and now I have /dev/sdb/ncdata/ncdata with the contents copied.

I think I’m getting closer. The thing I don’t understand is how to enter the address to the new ncdata directory.

In your example, you use “local::/mnt/sda/ncdata/”

My hard drive is installed at “/dev/sba/ncdata/ncdata/”

I think that “sba” is just a pointer (or shortcut), and not the actual drive reference. And, I think that when I used “mkdir” to create the top level “ncdata” directory, it set it as a mount point (just guessing). I was unable to copy data to that location. But I was able to use the -R and copy all of the original ncdata directory/folder to /ncdata/ncdata/ (I think I had to mount the first level ncdata to do that).

If this makes any sense, what should I use as the new data directory? I know how to edit the oc_storages record to update it there, just not sure how to enter it.

Here’s the error message I get when I try to log into nextcloud via browser:

image

Also just noticed that if I try to change to the /ncdata/ncdata directory, I get “Permission denied”. But now I just tried logging in using sudo -i first and was able to get into the directory, but now I’m not seeing the files. Previously, there were there.

UPDATE: Got it working! Haven’t checked everything but, I’m able to log in and the original files/folders are present. Unless you see something wrong with what I posted above, regarding using /ncdata/ncdata vs maybe /mnt/ncdata, I think I’m on the right path.

Hmm, the /dev/... folder contains the direct hardware connectors, you shouldn’t be able to store something there and cannot copy data FROM there. From there you can mount drives the the actual mount points, which are/should be located inside /mnt/....
You could check your fstab where the drive is really mounted to or add the related entry to mount it to a desired location: cat /etc/fstab
Or use df to get an overview:

root@micha:~# df
Filesystem      1K-blocks      Used  Available Use% Mounted on
...
/dev/sda       2884154032 219167796 2518456524   9% /mnt/sda
...
  • Hardware connector /dev/sda, mount point where I can store to and open files from: /mnt/sda

So I guess, /dev/sdb is mounted to /mnt/sdb, which would be at least the usual way/location. Now the question is, if you want the whole drive as Nextcloud datadir or just a subdirectory /mnt/sdb/ncdata.

cp -a /var/ncdata/. /mnt/sdb         # to use the whole drive, or
cp -a /var/ncdata /mnt/sdb/          # to use `/mnt/sdb/ncdata`

€: Ah okay, if it’s working now and the data is on the correct drive, do not mess around, the above is just cosmetic then :wink:.

Sounds good. I’m pretty particular, so I may do more research and try to actually figure out what I’ve done and ‘maybe’ make the additional adjustments.:roll_eyes:

Can’t say enough how much I appreciate your help. I just noticed that comments can be posted under your guide that I relied on (plus this direct help). I may post a summary there if you don’t mind, as it concerns a Hyper-V installation (which I now realize doen’t really affect the data directory move).

These last tips you provided about the /mnt/ option help a lot. I just need to decide if I want to change anything. I also just discovered that the new ncdata directory shows up in Webmin | File Manager. Having a GUI to look things over helps.

I learned a lot about Ubuntu/Linux and MySQL commands in this exercise, and that was part of the objective too. Thanks!

Just discovered something interesting that I have not found reported anywhere before:

When the config.php path to the new datadirectgory is updated/changed, a new entry is automatically added to the oc_storages table. That is why I was getting the “duplicate entry” error message. I confirmed this with tests.

I typed up a guide using my real world data which I’d like to add to your “how to” if you don’t mind.

Interesting, this is indeed new to me. Will test myself around. The question then is, if one can skip the database adjustment or if the numeric_id needs to be the same, so that the file IDs will be taken (including comments, previews etc., everything attached to files via database).

Additional thoughts. I’m not sure what triggers the automatic update to oc_storages after config.php is updated but it may be done when exiting maintenance mode.

In my test, I made a “dummy” datadirectory path entry in config.php. I then exited maintenance mode and received a message showing several broken links. In fact, I don’t think it allowed me to fully exit maintenance mode at that point (not sure). But I started mysql and did a “select * from oc_storages” and confirmed that the new “dummy” path had in fact been added.

I do not think I got the error messages when using a real (existing) path for the datadirectory in config.php.

The table showed the original “local::xxxxx” path plus the newly added one with the next incremental id number.

So, it’s possible that the best sequence may be:

Create new data directory -> update os_storages using UPDATE command on the locall:: field -> enter maintenance:mode -> modify config.php with new datadirectory path -> update fstab -> exit maintenance mode.

Since the “id” field contains the data directory path information, I’m thinking that “id” is the primary key field and numeric_id is just an auto-number key field that must be unique. I don’t know how to query the database to see what other tables might link to oc_storages, and what filed they use (if any). If it is the “id” field than changing the numeric_id wouldn’t destroy any links. I’m inclined to want to keep the table tidy, so I re-numbered the id with no apparent adverse effects (still can log into nextcloud using browser, but didn’t try anything like adding users are new files).

I also think that as users are added and removed from Nextcloud, an entry is added for each user. And, I think the respective entries are automatically removed when the user and their data are removed. I don’t think the “local::” field ever gets removed automatically, so must be manually adjusted. But that is just a hunch.

Interesting, this is indeed new to me. Will test myself around. The question then is, if one can skip the database adjustment or if the numeric_id needs to be the same, so that the file IDs will be taken (including comments, previews etc., everything attached to files via database).

I confirmed that for my installation, running nextcloud 13.1, I can indeed crate a new data directory, copy original files there, edit config.php to add new datadirectory; edit fstab as needed; after which I can open nextcloud in a browser and all files & folders are there with no apparent issues. Here’s the sequencing of tests I did:

TEST 1

  • Maintenance mode on
  • Edit config.php (add new datadirectory entry)
  • Edit fstab (enter new drive / mount point)
  • Check oc_storages (no new entries)
  • Maintenance mode off (no error message)
  • Check oc_storages (new entry added for “local::” using datadirectory path from config.php)
  • Exit mysql
  • Exit root
  • Test in browser (login successful, files & folders present)

TEST 2

  • Maintenance mode on
  • Edit config.php (restore original datadirectory entry)
  • Edit fstab (restore previous drive / mount point)
  • Maintenance mode off (no error message)
  • Exit root
  • Test in browser (login successful, files & folders present)
  • Check oc_storages (no changes since previous check; two entries for “local::”)

CONCLUSION: In version 13.1 of nextcloud, it does not appear to be necessary to make any adjustments to oc_storages database table after updating config.php and exiting maintenance mode. A new entry will be added to oc_storages for the new datadirectory, and the old one will be preserved as well (unless the orphaned entry is automatically deleted during some subsequent cleanup – will check in a day or two).