HowTo: Change / Move data directory after installation


#1

As there still appear questions about that topic, I thought sum up all possibilities here in howto would be a good idea. There is already a good thread that handles this topic: Is there a safe and reliable way to move data directory out of web root?

First of all: Changing data directory after installation is not officially supported. Consider re-installing Nextcloud with new data directory, if you did not use it too much/added users/created shares/tags/comments etc.

Background

Nextcloud looks up the location of the data directory inside its /path/to/nextcloud/config/config.php by the 'datadirectory' directive: https://docs.nextcloud.com/server/13/admin_manual/configuration_server/config_sample_php_parameters.html#default-parameters
As seen '/path/to/nextcloud/data' is default value/location, if it is not changed during Nextcloud installation process.

Nextcloud stores the data directory location and all it’s files locations inside it’s database, that can be chosen during installation as well. Therefore it creates the tables oc_storages for the data directory and oc_filecache for the files. Other than it’s name suggests, it is actually not a cache, more an index of all files inside the data directory. If a file is not listed inside this table, it can’t be handled by Nextcloud, nor will it be seen inside the web ui.

So by just moving the data directory to another location and change the 'datadirectory' directive inside config.php, the oc_filecache will be full of wrong entries from the old location, where the files entries for the new location are missing, thus no files will be shown inside Nextcloud.
Nextcloud itself will regularly clean oc_filecache from old/obsolete entries (by cron job occ files:cleanup), but the new entries would need to be manually created by running occ files:scan --all. For apache webserver the full command would look like:

sudo -u www-data php /path/to/nextcloud/occ files:scan --all

Your files will be now accessible again through Nextcloud. The remaining problem now is, that recreating the file index will also create new file IDs. Other related information, like shares, tags, comments, previews etc., on the other hand are connected to the old file IDs, thus will be lost.

To avoid this whole file index recreation problem, the oc_storages database table could be manually adjusted from within the database server. Where I don’t know any way to do that for sqlite, for mysql/mariadb with root access this is quite simple. But indeed manually adjusting the database always bears the risk of corruption, thus database access for Nextcloud should be prevented during the process, simply by putting nextcloud into maintenance mode, and a full database dump/backup should be done.

Solution 1

I refer to Is there a safe and reliable way to move data directory out of web root? where the steps are given and list/complete them here with commands where apache webserver and mysql/mariadb database is assumed:

1. sudo -u www-data php /path/to/nextcloud/occ maintenance:mode --on
2. mkdir -p /new/path/to/data
3. cp -a /path/to/data/. /new/path/to/data
4. chown -R www-data:www-data /new/path/to/data
5. nano /path/to/nextcloud/config/config.php
       'datadirectory' => '/new/path/to/data',
6. mysqldump -u<rootuser> -p > /path/to/dbdump/dump.sql
7. Adjust "oc_storages"database table to reflect the new data folder location:
      mysql -u<rootuser> -p
      //enter mysql root password, then within mysql console:
      use <nextclouddb>;
      update oc_storages set id='local::/new/path/to/data/' where id='local::/path/to/data/'; //take care about backslash at the end of path!!
      quit;
8. sudo -u www-data php /path/to/nextcloud/occ maintenance:mode --off

After that, carefully test Nextcloud, the files inside web ui, shares, tags, comments etc. If everything is working fine and Nextcloud indeed handles the files on the new location, you could remove the backups:

 1. rm -R /path/to/data //old location!!
 2. rm /path/to/dbdump/dump.sql

or just keep them as regular backups ;).

Solution 2

This solution is far simpler and less risky in data corruption concerns, but it does NOT fulfil the security recommendations, e.g. mentioned in admin manual. So if you just need to move the data directory for storage reasons, not for security reasons, just create a symbolic link from old to new data directory location. For that you only need to be sure, that symbolic links are correctly handled by your webserver. Nginx should handle symlinks correctly by default, for apache the related option needs to be set in your Nextcloud vhost, as already given by the admin manual example: https://docs.nextcloud.com/server/13/admin_manual/installation/source_installation.html#apache-web-server-configuration

<Directory /path/to/nextcloud/>
  Options +FollowSymlinks
  ...
</Directory>

If so, then do:
Steps are also given here: Is there a safe and reliable way to move data directory out of web root?

 1. sudo -u www-data php /path/to/nextcloud/occ maintenance:mode --on
 2. mkdir -p /new/path/to/data
 3. cp -a /path/to/data/. /new/path/to/data
 4. mv /path/to/data /path/to/dataBackup
 5. ln -s /new/path/to/data /path/to/data
 6. chown -d www-data:www-data /path/to/data # To set symlink ownership
 7. chown -R www-data:www-data /new/path/to/data # To set actual data dir ownership
 8. sudo -u www-data php /path/to/nextcloud/occ maintenance:mode --off

Again, if everything works fine, you could remove /path/to/dataBackup or just keep it as regular backup.

Note: If you want to adjust permissions of your new data folder, you need to chmod -R XYZ /new/path/to/data, as chmod does not follow symlinks. Thanks for the hint @STrike :slight_smile:


Setting a new data directory
Is there a safe and reliable way to move data directory out of web root?
Moving data directory in Nextcloud 11.0.2
How-to/FAQ WIKI
Is there a safe and reliable way to move data directory out of web root?
Troubles after moving the data folder
Speicherort auf andere Partition verschieben
[Solved] Error when upload You don’t have permission to upload or create files here
How to move data directory on VM installation under Windows 8.1 host PC
Can't log in to web interface. Win client struggling
[Solved] Error when upload You don’t have permission to upload or create files here
Nextcloud 13.0.1, 12.0.6 and 11.0.8 available with improved password protection
Calendar app not working
Best solution for external storage
Externer Speicher zum synchronisieren als Standard
Where is my Apache configuration
Is Nextcloud for Home Enthusiasts or Home Experts?
Änderung des Datadirectory in der Config.php. Auswirkungen?
Problem upgrading 14.03 to 14.04; and need help understanding directory structure
Change Data Directory BEFORE or WHILE Installation to /media/
Moving to new Nextcloud installation (same version)
How to change Data Folder after install NC13
Extra files found when trying to update to 12.0.3
Nextcloud Directory Showing for all Users
Installation Questions
Moving data directory
Cant change data direcotry
Data Migration App
Fehlende Rechte in Nextcloutpi
Can I change the data folder cause problems?
Gruppenordner auf eingebunder Festplatte erstellen
Nextcloud "data" Ordner außerhalb von root
#2

After installing nextcloud using the packaged VM from Tech and Me, I realized that it doesn’t offer the opportunity to move the data directory during installation. The one I used also comes preconfigured as a 20GB disk VHD. Immediately after installation I realized the need to move the data directory to a larger virtual disk. Michalng’s guide tougher with his personal assistance made it all possible as described here:

During that exercise I discovered that under nextcloud 13.1, it wasn’t necessary to modify the oc_storages table. Once a new entry is added to config.php for the datadirectory (described below), after exiting maintenance mode, oc_storages is automatically updated with the new data directory path added.

Since I’m new to Ubuntu and Nextcloud, I drafted a guide using actual defaults and variables for my installation, which is posted below. Note: This is assuming a new installation with NO DATA other than files & folders installed by default.

Move Data to New Directory Location
1. Create New Data Directory & Mount Point

mkdir /mnt/ncdata # create new directory.
sudo mount /dev/sdb /mnt/ncdata # mount new directory [FN1].
sudo chmod 0770 /mnt/ncdata # change permissions to 770.

Note: Default Permissions is set to 0770 on original data directory in Nextcloud 13 [FN2].

Get list of all attached drives and mountpoints to confirm to confirm that sdb mount point /mnt/ncdata was created.

lsblk # list blocks
dvices: lshw -class disk -short # confirm disks
fdisk -l # confirm partition tables

2. Modify Owner for New Data Directory

chown -R www-data:www-data /mnt/ncdata # change new directory owner.
cd /mnt/ncdata # change to the mount directory.

3. Copy Folders & Files to New Data Directory

cd /var/ncdata # change to new directory.
rsync -av . /mnt/ncdata # use resync to copy files.
ls -a # confirm that files were copied.

Note: As an alternative you can use Webmin File Manager or other file utility to copy files to new location and confirm Owner and Rights settings.

4. Turn Maintenance Mode On

sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --on

5. Edit fstab (File System Tables) Files
Get drive & mount point information.

blkid

Copy UUID for /dev/sdb to update fstab (Recommend using the UUID instead of device Name.) [FN3]

Unmount new drive.

umount /mnt/ncdata

Edit fstab

sudo vim /etc/fstab

Add new line at bottom for the new drive mount point. Use the UUID for the disk ID.

UUID=02C22F1539D54F25 /dev/ncdata ext4 0 2

Save and exit. (Note: UUID is a dummy – use your actual drive ID.)

6. Edit config.php File to Add New datadirectory

sudo vim /var/www/nextcloud/config/config.php

Modify ‘datadirectory’ line to add new mount point/directory. Use Insert key to start editing:

‘datadirectory’ => ‘/mnt/ncdata’,

Be sure to include single quotes, also preserve the coma at end of the line.
Hit Esc to end editing mode, save & exit: :wq

Note: After updating the cofig.php file, the new data directory entry will be automatically added to the oc_storages table if the table was not updated before the config file. [FN4]

7. Turn Maintenance Mode Off

sudo -u www-data php /var/www/nextcloud/occ maintenance:mode –off

8. Exit & Re-Start Nextcloud

Test logging in using browser.

FOOTNOTES
[1] The device “sdb” was used to create the mount point since the drive will be used exclusively for the Nextcloud VM data. Change as required to match installation.

[2] Default Permissions for the data directory on original install of Nextcloud 13 is set to 0770. Some other versions of Nextcloud may use a different permission classification. Confirm for your installation.

[3] It is recommended to use the UUID for a storage drive rather than the mount/path.

The advantage of using the UUID is that it is independent from the device number the operating system gives your hard disk (such as sdb1, etc). If another hard disk is added to the system the OS could change the drive label (sda, sdb, sda1, etc). In which case, the drive would no longer be associated with the intended mount point for a given device. However, if the UUIDs, the mount point will maintain its association with the specific drive/device.

[4] When the ‘datadirectory’ line is changed in config.php the newly added data directory path is automatically added to the oc_storages table in the nextcloud_db database. It does not delete the original path which may be left in place.


#3

@scooter
Many thanks for investigating this.

Indeed, I just found the time to test it as well. Skipping the adjustment of oc_storages table indeed works. The new data directory is added to the table automatically. Comments and file tags are preserved, thus file cache is (still) valid.

The steps within OT do still work and properly the cleaner method, as there will be no obsolete entry left, but indeed the manual MySQL table edit might be a reason for one to hesitate using ‘Solution 1’, for me it’s the only argument for ‘Solution 2’ actually :wink:.

@JasonBayton @tflidd
Did you hear something of this, is this new since Nextcloud 13? Couldn’t find something in the changelogs or github commits.

This indeed would mark most of OT as obsolete. I would shrink it to just one single solution:

maintenance --on
cp -a /old /new
config.php 'datadirectory'
maintenance --off

But verification/official statement would be nice :slightly_smiling_face:.


#4

This would need to come from @nickvergessen, @MorrisJobke @jospoortvliet or someone else :slight_smile:


#5

Usually you “only” need to adjust the entry in the oc_storages and the entry in the config.php after the move. So there is the change of the oc_storages table missing in your steps.

Little disclaimer: we do not recommend to do this, because it could be that other apps use this somehow and thus need additional adjustments. So try it and check if everything works as expected. Deleting the oc_storages entry looks fishy to me and is definitely not the way you should go.


#6

I moved it twice already, always changed config.php and UPDATED the oc_storages entry.

Never had a problem so far.


#7

@MorrisJobke @nickvergessen
Okay, this underlines the “known” procedure in OT, but what about the oc_storages entry being added automatically? As mentioned within discussion, scooter and me could successfully change the datadirectory without adjusting oc_storages (my short steps list above). Even comments and flags, thus stuff that especially relies on file IDs, is preserved.


#9

Under 13.1, oc_storages gets the correct update automatically. I never tried prior versions, so I can’t say for sure. It seems that prior versions did not update automatically and required manual entries/adjustments to oc_storages.

It also seems that, after updating config.php while in maintenance mode, then exiting maintenance mode, a check is done between the datadirectory entry and oc_storages to see if the path is already in the table and if not, it adds it, PLUS it also checks fstab to see if the directory is mounted at startup. If not, an error message is given when exiting maintenance mode saying that the datadirectory entry is invalid.

Here are the tests I performed:

TEST 1
Maintenance mode on
Edit config.php (add new datadirectory entry)
Edit fstab (enter new drive / mount point)
Check oc_storages (no new entries)
Maintenance mode off (no error message)
Check oc_storages (new entry aded for “local::” using datadirectory path from config.php)
Exit mysql
Exit root
Test in browser (login successful, files & folders present)

TEST 2
Maintenance mode on
Edit config.php (restore original datadirectory entry)
Edit fstab (restore previous drive / mount point)
Maintenance mode off (no error message)
Exit root
Test in browser (login successful, files & folders present)
Check oc_storages (no changes since previous check; two entries for “local::”)

I did numerous other tests before determining that this order produced no errors. In retrospect, it’s probably better to update fstab before config.php, but as long as both are done while in mainteance mode, it doesn’t seem to matter.


#10

Semi noob here-

  1. Would having Nginx vs apache change any of the guide for method #2?

  2. Does having NC 13.05 affect anything as well?


#11

@Dove7

  1. Looks like you just leave out the apache-specific configuration parts, and nginx does the expected thing automatically.
  2. Nothing new in 13.05 should change method #2. :slight_smile:

#12

@ MichaIng Thanks for your guide!

As of Nextcloud version 13.0.5 it was not possible for me to get the symlink to work. After symlinking and starting up the server, it would always reply with

“ERROR Your data directory is invalid Ensure there is a file called “.ocdata” in the root of the data directory”

when requesting the index.php page.

But I got it to work still! By modifying your solution #2 a bit by using $(mount --bind) instead of symlink, and it seems to work good!

Here are the exact steps I took

Setup:

  • I have a disk mounted at /mnt/nextcloud-data where I want to put the nextcloud data in, in a directory named “data”, thus full path of where I want to place the nextcloud data files is /mnt/nextcloud-data/data.
  • I’m installing Nextcloud using Snap, on a Ubuntu 16.04 server.

Steps:

# Enter maintenance mode.
sudo apt-get install php
sudo -u www-data php /snap/nextcloud/current/bin/occ maintenance:mode --on

# Copy over existing files to the new destination.
sudo rsync -a --progress --stats /var/snap/nextcloud/common/nextcloud/data /mnt/nextcloud-data/

# Move old data dir to .bak extension, and create a new empty data dir.
sudo mv /var/snap/nextcloud/common/nextcloud/data{,.bak}
sudo mkdir /var/snap/nextcloud/common/nextcloud/data

# Mount the new data destination over the data dir.
mount --bind /mnt/nextcloud-data/data/ /var/snap/nextcloud/common/nextcloud/data

# Make this permanent.
sudo cat << EOF >>/etc/fstab
/mnt/nextcloud-data/data/ /var/snap/nextcloud/common/nextcloud/data none bind 0 0
EOF

# Leave maintenance mode.
sudo -u www-data php /snap/nextcloud/current/bin/occ maintenance:mode --off

#13

Perhaps the snap setup inhibits symlinks, not sure.
I guess its bundled with Apache webserver? Is there a way to configure the webserver?

€: Jep Apache with symlinks explicitly enabled just for the htdocs folder: https://github.com/nextcloud/nextcloud-snap/blob/master/src/apache/conf/httpd.conf
Not sure about defaults on this Apache version but this might be the reason it does not work.


#14

move data where you want and
make changes in
path/config/config.php

$CONFIG = array (
	***
	***
	***
	'datadirectory' => '/var/ncdata/data', //new path of data directory
	***
	***
	***
);

maybe this helps.


#15

Jep during all tests this worked at least since NC13. The needed database entry is created automatically, so no manual mysql step seems to be needed.

However before editing to guide above I am hoping for an official statement, since @nickvergessen still mentioned to database update ;). I guess on older NC versions it is indeed still needed, so would be good to know when this was changed.

Anyway the database update does not hurt and is even cleaner since no obsolete entry remains.


#16

You won’t hear anything different ever from me. Not updating the existing record but automatically creating a new one leaves the database in an inconsistent state and could lead to problems in the future.


#17

Thanks for the clear statement. I will follow it and won’t change the guide.


#18

Hi there, and first thank you all for these precious information.

I figured out the solution to my problem while writting but still, I spent way more time than I would have wanted on this so : To anyone it may help ^^

With Nextcloud 14.0.3, I when performing the update on the database :
> update oc_storages set id=‘local::/new/path/to/data/’ where id=‘local::/path/to/data/’:wink:

I got the following error :
"ERROR 1062 (23000): Duplicate entry ‘local::/nas/data/’ for key ‘storages_id_index’"

I looked throuh the database using phpmyadmin I found that it probably came from the fact that a line was created automatically with value “local::/nas/data/” even before I executed the following line from the tutorial :

update oc_storages set id=‘local::/new/path/to/data/’ where id=‘local::/path/to/data/’;

So I was trying to create the line a second time, basically duplicating it. This is probably because Nextcloud detects that the path in the config.php is not the same as the one in the db, and so it automatically generates a new line with the right path.

Actually, everything works fine with this automatically generated line and so the update of the path on the database seems to become quite obsolete now.

However, if you want to make your installation totally clean and remove the line with the old path, what you have to do is just go fast and :

  • delete the line with the new path (last line in oc_storages)
  • quickly update the old path into the new one before a line with the old path is generated.

(I did this from phpmyadmin so I don’t know the commands)


#19

Thanks for the info, even that you managed to resolve yourself.

Did you put Nextcloud into maintenance before doing the change in config.php, according to the guide?
Would be very strange that Nextcloud automatically adjusts the database entry when in maintenance mode :thinking:.

That the database entry is added automatically is known if you follow the discussion above. It does work without adding it manually, currently, but according to Nick’s statement above and for consistency it makes sense to replace/remove the old obsolete entry from database. So I didn’t change the guide.

But if somehow Nextcloud adds the entry, even if in maintenance, then this needs rework. Properly do a check first and add only if not already contained, or simply remove the old entry and rely on Nextcloud to add the new one automatically. Not sure which is the safest/failsafe solution with minimal manual user steps/checks required.


#20

I confirm the same behaviour with Nextcloud 14.0.3 (Maintenance mode : On)


#21

So again to assure:

  • You put Nextcloud into maintenance, did the data dir moving.
  • You did update oc_storages set id='local::/new/path/to/data/' where id='local::/path/to/data/'; within MySQL console.
  • Then you disabled maintenance mode and the local::/path/to/data/ database entry was doubled then, so Nextcloud UI refuses to open, or already occ maintenance:mode --off failed?

Will test/verify as well tomorrow. Very strange in case. Then instead of replacing the old entry, it should be removed only.