HowTo: Change / Move data directory after installation

Snap/Docker/VM users click here!

Note that this HowTo does not apply to Nextcloud containers/appliances or the official Nextcloud VM from Hansson IT.


As there still appear questions about that topic, I thought to describe all possibilities here as HowTo would be a good idea. Most information was originally taken from this topic: Is there a safe and reliable way to move data directory out of web root?

For the record: Changing the data directory after installation is not officially supported. Consider re-installing Nextcloud with new data directory, if you did not use it too much/added users/created shares/tags/comments etc.

Background

Click to show some background information

Nextcloud looks up the location of the data directory inside its /path/to/nextcloud/config/config.php by the 'datadirectory' directive: Configuration Parameters — Nextcloud latest Administration Manual latest documentation
As seen '/path/to/nextcloud/data' is default value/location, if it is not changed during Nextcloud installation process.

Nextcloud stores the data directory location and all it’s files locations inside it’s database, that can be chosen during installation as well. Therefore it creates the tables oc_storages for the data directory and oc_filecache for the files. Other than it’s name suggests, it is actually not a cache, more an index of all files inside the data directory. If a file is not listed inside this table, it can’t be handled by Nextcloud, nor will it be seen inside the web ui.

So by just moving the data directory to another location and change the 'datadirectory' directive inside config.php, the oc_filecache will be full of wrong entries from the old location, where the files entries for the new location are missing, thus no files will be shown inside Nextcloud.
Nextcloud itself will regularly clean oc_filecache from old/obsolete entries (by cron job occ files:cleanup), but the new entries would need to be manually created by running occ files:scan --all. For apache webserver the full command would look like:

sudo -u www-data php /path/to/nextcloud/occ files:scan --all

Your files will be now accessible again through Nextcloud. The remaining problem now is, that recreating the file index will also create new file IDs. Other related information, like shares, tags, comments, previews etc., on the other hand are connected to the old file IDs, thus will be lost.

To avoid this whole file index recreation problem, the oc_storages database table can be manually adjusted within the database. But indeed manually adjusting the database as inexperienced users always bears the risk of mistakes, hence a full database dump/backup should be done.


Solution 1: Move data dir with database change
Solution 2: Link data dir without database change

Solution 1

I refer to Is there a safe and reliable way to move data directory out of web root? - #17 by robertb where the steps are given and list/complete them here with commands where Apache webserver and MySQL/MariaDB database is assumed. Instead of prefixing all (non-www-data) commands with sudo, you could also start an interactive root session:

sudo -s
  1. For simplicity, we store all relevant paths into variables to be used later. Replace those placeholder paths to match your setup.

    ncdir='/path/to/nextcloud'
    olddata='/old/path/to/data'
    newdata='/new/path/to/data'
    
  2. Put Nextcloud into maintenance mode

    sudo -u www-data php "$ncdir/occ" maintenance:mode --on
    
  3. Copy Nextcloud data to new location. The below command copies the whole directory, including owner and modes, so there is no need to pre-create the new directory, change ownership or permissions.

    sudo cp -a "$olddata/." "$newdata"
    
  4. Update the data dir in config.php to
    'datadirectory' => '/new/path/to/data',. The sed command below should do it, but at best verify this afterwards, or edit the file manually.

    sudo sed -i "s|$olddata|$newdata|" "$ncdir/config/config.php"
    
  5. Create a database dump/backup. Depending on the root user’s database authentication method, you might need to add the -p option (mysqldump -A -p ...) to get a password prompt. Replace /path/to/backup with any directory, suitable to keep a backup of your database.

    sudo mysqldump -A > /path/to/backup/dump.sql
    
  6. Adjust oc_storages database table to reflect the new data folder location. We store the required database name and credentials, obtained from the config.php, into variables, to avoid typos. You might want to verify that those were obtained correctly, but be careful to print such sensitive information to the console, when others may watch, and exit the shell session afterwards.

    dbname=$(sudo awk -F\' "/'dbname'/{print \$4;exit}" "$ncdir/config/config.php")
    dbuser=$(sudo awk -F\' "/'dbuser'/{print \$4;exit}" "$ncdir/config/config.php")
    dbpass=$(sudo awk -F\' "/'dbpassword'/{print \$4;exit}" "$ncdir/config/config.php")
    #echo "$dbname $dbuser $dbpass"
    mysql -u"$dbuser" -p"$dbpass" -e "update $dbname.oc_storages set id='local::$newdata/' where id='local::$olddata/';"
    unset -v dbname dbuser dbpass
    
  7. Disable maintenance mode

    sudo -u www-data php "$ncdir/occ" maintenance:mode --off
    

After that, carefully test Nextcloud, the files inside web ui, shares, tags, comments etc. If everything is working fine and Nextcloud indeed handles the files on the new location, you could remove the backups:

sudo rm -R "$olddata"
rm /path/to/backup/dump.sql

or just keep them as regular backups ;).

Solution 2

This solution is simpler and less risky, as it does not require to edit the database, but it does NOT fulfil security recommendations, as e.g. mentioned in the admin manual. So if you just need to move the data directory for storage reasons only, not for security reasons, just create a symbolic link from old to new data directory location. For that you only need to be sure, that symbolic links are correctly handled by your webserver. Nginx should handle symlinks correctly by default, for Apache the related option needs to be set in your Nextcloud vhost, as given by the admin manual example: Installation on Linux — Nextcloud latest Administration Manual latest documentation

<Directory /path/to/nextcloud>
    Options +FollowSymlinks
    ...
</Directory>

The steps were originally taken from here: Is there a safe and reliable way to move data directory out of web root? - #4 by JasonBayton
Instead of prefixing all (non-www-data) commands with sudo, you could also start an interactive root session:

sudo -s
  1. For simplicity, we store all relevant paths into variables to be used later. Replace those placeholder paths to match your setup.

    ncdir='/path/to/nextcloud'
    olddata='/old/path/to/data'
    newdata='/new/path/to/data'
    
  2. Put Nextcloud into maintenance mode

    sudo -u www-data php "$ncdir/occ" maintenance:mode --on
    
  3. Copy Nextcloud data to new location. The below command copies the whole directory, including owner and modes, so there is no need to pre-create the new directory, change ownership or permissions.

    sudo cp -a "$olddata/." "$newdata"
    
  4. Better safe than sorry, move the original data directory to a backup location, before creating the symlink to the new one. If you do not need the backup, replace, remove the old data dir instead.

    sudo mv "$olddata" /path/to/backup/data
    #sudo rm -R "$olddata"
    sudo ln -s "$newdata" "$olddata"
    
  5. Disable maintenance mode

    sudo -u www-data php "$ncdir/occ" maintenance:mode --off
    

Again, if everything works fine, you could remove the backup or just keep it as regular backup.

sudo rm -R /path/to/backup/data

Note: If you want to adjust permissions of your new data directory, you need to chmod -R XYZ /new/path/to/data, as chmod does not follow symlinks. Thanks for the hint @STrike :slight_smile:

34 Likes

After installing nextcloud using the packaged VM from Tech and Me, I realized that it doesn’t offer the opportunity to move the data directory during installation. The one I used also comes preconfigured as a 20GB disk VHD. Immediately after installation I realized the need to move the data directory to a larger virtual disk. Michalng’s guide tougher with his personal assistance made it all possible as described here:

During that exercise I discovered that under nextcloud 13.1, it wasn’t necessary to modify the oc_storages table. Once a new entry is added to config.php for the datadirectory (described below), after exiting maintenance mode, oc_storages is automatically updated with the new data directory path added.

Since I’m new to Ubuntu and Nextcloud, I drafted a guide using actual defaults and variables for my installation, which is posted below. Note: This is assuming a new installation with NO DATA other than files & folders installed by default.

Move Data to New Directory Location
1. Create New Data Directory & Mount Point

mkdir /mnt/ncdata # create new directory.
sudo mount /dev/sdb /mnt/ncdata # mount new directory [FN1].
sudo chmod 0770 /mnt/ncdata # change permissions to 770.

Note: Default Permissions is set to 0770 on original data directory in Nextcloud 13 [FN2].

Get list of all attached drives and mountpoints to confirm to confirm that sdb mount point /mnt/ncdata was created.

lsblk # list blocks
dvices: lshw -class disk -short # confirm disks
fdisk -l # confirm partition tables

2. Modify Owner for New Data Directory

chown -R www-data:www-data /mnt/ncdata # change new directory owner.
cd /mnt/ncdata # change to the mount directory.

3. Copy Folders & Files to New Data Directory

cd /var/ncdata # change to new directory.
rsync -av . /mnt/ncdata # use resync to copy files.
ls -a # confirm that files were copied.

Note: As an alternative you can use Webmin File Manager or other file utility to copy files to new location and confirm Owner and Rights settings.

4. Turn Maintenance Mode On

sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --on

5. Edit fstab (File System Tables) Files
Get drive & mount point information.

blkid

Copy UUID for /dev/sdb to update fstab (Recommend using the UUID instead of device Name.) [FN3]

Unmount new drive.

umount /mnt/ncdata

Edit fstab

sudo vim /etc/fstab

Add new line at bottom for the new drive mount point. Use the UUID for the disk ID.

UUID=02C22F1539D54F25 /dev/ncdata ext4 0 2

Save and exit. (Note: UUID is a dummy – use your actual drive ID.)

6. Edit config.php File to Add New datadirectory

sudo vim /var/www/nextcloud/config/config.php

Modify ‘datadirectory’ line to add new mount point/directory. Use Insert key to start editing:

‘datadirectory’ => ‘/mnt/ncdata’,

Be sure to include single quotes, also preserve the coma at end of the line.
Hit Esc to end editing mode, save & exit: :wq

Note: After updating the cofig.php file, the new data directory entry will be automatically added to the oc_storages table if the table was not updated before the config file. [FN4]

7. Turn Maintenance Mode Off

sudo -u www-data php /var/www/nextcloud/occ maintenance:mode –off

8. Exit & Re-Start Nextcloud

Test logging in using browser.

FOOTNOTES
[1] The device “sdb” was used to create the mount point since the drive will be used exclusively for the Nextcloud VM data. Change as required to match installation.

[2] Default Permissions for the data directory on original install of Nextcloud 13 is set to 0770. Some other versions of Nextcloud may use a different permission classification. Confirm for your installation.

[3] It is recommended to use the UUID for a storage drive rather than the mount/path.

The advantage of using the UUID is that it is independent from the device number the operating system gives your hard disk (such as sdb1, etc). If another hard disk is added to the system the OS could change the drive label (sda, sdb, sda1, etc). In which case, the drive would no longer be associated with the intended mount point for a given device. However, if the UUIDs, the mount point will maintain its association with the specific drive/device.

[4] When the ‘datadirectory’ line is changed in config.php the newly added data directory path is automatically added to the oc_storages table in the nextcloud_db database. It does not delete the original path which may be left in place.

4 Likes

@scooter
Many thanks for investigating this.

Indeed, I just found the time to test it as well. Skipping the adjustment of oc_storages table indeed works. The new data directory is added to the table automatically. Comments and file tags are preserved, thus file cache is (still) valid.

The steps within OT do still work and properly the cleaner method, as there will be no obsolete entry left, but indeed the manual MySQL table edit might be a reason for one to hesitate using ‘Solution 1’, for me it’s the only argument for ‘Solution 2’ actually :wink:.

@JasonBayton @tflidd
Did you hear something of this, is this new since Nextcloud 13? Couldn’t find something in the changelogs or github commits.

This indeed would mark most of OT as obsolete. I would shrink it to just one single solution:

maintenance --on
cp -a /old /new
config.php 'datadirectory'
maintenance --off

But verification/official statement would be nice :slightly_smiling_face:.

2 Likes

This would need to come from @nickvergessen, @MorrisJobke @jospoortvliet or someone else :slight_smile:

Usually you “only” need to adjust the entry in the oc_storages and the entry in the config.php after the move. So there is the change of the oc_storages table missing in your steps.

Little disclaimer: we do not recommend to do this, because it could be that other apps use this somehow and thus need additional adjustments. So try it and check if everything works as expected. Deleting the oc_storages entry looks fishy to me and is definitely not the way you should go.

I moved it twice already, always changed config.php and UPDATED the oc_storages entry.

Never had a problem so far.

1 Like

@MorrisJobke @nickvergessen
Okay, this underlines the “known” procedure in OT, but what about the oc_storages entry being added automatically? As mentioned within discussion, scooter and me could successfully change the datadirectory without adjusting oc_storages (my short steps list above). Even comments and flags, thus stuff that especially relies on file IDs, is preserved.

Under 13.1, oc_storages gets the correct update automatically. I never tried prior versions, so I can’t say for sure. It seems that prior versions did not update automatically and required manual entries/adjustments to oc_storages.

It also seems that, after updating config.php while in maintenance mode, then exiting maintenance mode, a check is done between the datadirectory entry and oc_storages to see if the path is already in the table and if not, it adds it, PLUS it also checks fstab to see if the directory is mounted at startup. If not, an error message is given when exiting maintenance mode saying that the datadirectory entry is invalid.

Here are the tests I performed:

TEST 1
Maintenance mode on
Edit config.php (add new datadirectory entry)
Edit fstab (enter new drive / mount point)
Check oc_storages (no new entries)
Maintenance mode off (no error message)
Check oc_storages (new entry aded for “local::” using datadirectory path from config.php)
Exit mysql
Exit root
Test in browser (login successful, files & folders present)

TEST 2
Maintenance mode on
Edit config.php (restore original datadirectory entry)
Edit fstab (restore previous drive / mount point)
Maintenance mode off (no error message)
Exit root
Test in browser (login successful, files & folders present)
Check oc_storages (no changes since previous check; two entries for “local::”)

I did numerous other tests before determining that this order produced no errors. In retrospect, it’s probably better to update fstab before config.php, but as long as both are done while in mainteance mode, it doesn’t seem to matter.

1 Like

Semi noob here-

  1. Would having Nginx vs apache change any of the guide for method #2?

  2. Does having NC 13.05 affect anything as well?

@Dove7

  1. Looks like you just leave out the apache-specific configuration parts, and nginx does the expected thing automatically.
  2. Nothing new in 13.05 should change method #2. :slight_smile:

Perhaps the snap setup inhibits symlinks, not sure.
I guess its bundled with Apache webserver? Is there a way to configure the webserver?

€: Jep Apache with symlinks explicitly enabled just for the htdocs folder: https://github.com/nextcloud/nextcloud-snap/blob/master/src/apache/conf/httpd.conf
Not sure about defaults on this Apache version but this might be the reason it does not work.

move data where you want and
make changes in
path/config/config.php

$CONFIG = array (
	***
	***
	***
	'datadirectory' => '/var/ncdata/data', //new path of data directory
	***
	***
	***
);

maybe this helps.

1 Like

Jep during all tests this worked at least since NC13. The needed database entry is created automatically, so no manual mysql step seems to be needed.

However before editing to guide above I am hoping for an official statement, since @nickvergessen still mentioned to database update ;). I guess on older NC versions it is indeed still needed, so would be good to know when this was changed.

Anyway the database update does not hurt and is even cleaner since no obsolete entry remains.

You won’t hear anything different ever from me. Not updating the existing record but automatically creating a new one leaves the database in an inconsistent state and could lead to problems in the future.

2 Likes

Thanks for the clear statement. I will follow it and won’t change the guide.

Hi there, and first thank you all for these precious information.

I figured out the solution to my problem while writting but still, I spent way more time than I would have wanted on this so : To anyone it may help ^^

With Nextcloud 14.0.3, I when performing the update on the database :
> update oc_storages set id=‘local::/new/path/to/data/’ where id=‘local::/path/to/data/’:wink:

I got the following error :
“ERROR 1062 (23000): Duplicate entry ‘local::/nas/data/’ for key ‘storages_id_index’”

I looked throuh the database using phpmyadmin I found that it probably came from the fact that a line was created automatically with value “local::/nas/data/” even before I executed the following line from the tutorial :

update oc_storages set id=‘local::/new/path/to/data/’ where id=‘local::/path/to/data/’;

So I was trying to create the line a second time, basically duplicating it. This is probably because Nextcloud detects that the path in the config.php is not the same as the one in the db, and so it automatically generates a new line with the right path.

Actually, everything works fine with this automatically generated line and so the update of the path on the database seems to become quite obsolete now.

However, if you want to make your installation totally clean and remove the line with the old path, what you have to do is just go fast and :

  • delete the line with the new path (last line in oc_storages)
  • quickly update the old path into the new one before a line with the old path is generated.

(I did this from phpmyadmin so I don’t know the commands)

1 Like

Thanks for the info, even that you managed to resolve yourself.

Did you put Nextcloud into maintenance before doing the change in config.php, according to the guide?
Would be very strange that Nextcloud automatically adjusts the database entry when in maintenance mode :thinking:.

That the database entry is added automatically is known if you follow the discussion above. It does work without adding it manually, currently, but according to Nick’s statement above and for consistency it makes sense to replace/remove the old obsolete entry from database. So I didn’t change the guide.

But if somehow Nextcloud adds the entry, even if in maintenance, then this needs rework. Properly do a check first and add only if not already contained, or simply remove the old entry and rely on Nextcloud to add the new one automatically. Not sure which is the safest/failsafe solution with minimal manual user steps/checks required.

I confirm the same behaviour with Nextcloud 14.0.3 (Maintenance mode : On)

So again to assure:

  • You put Nextcloud into maintenance, did the data dir moving.
  • You did update oc_storages set id='local::/new/path/to/data/' where id='local::/path/to/data/'; within MySQL console.
  • Then you disabled maintenance mode and the local::/path/to/data/ database entry was doubled then, so Nextcloud UI refuses to open, or already occ maintenance:mode --off failed?

Will test/verify as well tomorrow. Very strange in case. Then instead of replacing the old entry, it should be removed only.

Is it safe to remove old entries in oc_storage table that refer to old, unused local storages? I seem to still have an old storage back from owncloud, and very old one from nextlcoud (changed mountpoints years back).