Tutorial: How to migrate mass data to a new NextCloud server

@fab

Fab, I read through your tutorial and i was relieved to see it was possible to migrate mass data onto my NextClould quickly. Thank You.

I followed your tutorial and i can see that the (test) file i attempted to move was in fact relocated to the destination.
But, when i ran
sudo -u www-data php console.php files:scan --all
command in " /var/www/nextcloud " an error was returned which says:

Your data directory is invalid

Ensure there is a file called ".ocdata" in the root of the data directory.

Your data directory is readable by other users

Please change the permissions to 0770 so that the directory cannot be listed by other users.

An unhandled exception has been thrown:

Exception: Environment not properly prepared. in /var/www/nextcloud/lib/private/Console/Application.php:148

Stack trace:

#0 /var/www/nextcloud/console.php(89): OC\Console\Application->loadCommands(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))

Also, i canā€™t even access any of my user accounts . iā€™m getting an error that says: "
" Your data directory is invalid
Ensure there is a file called ā€œ.ocdataā€ in the root of the data directory. "

In my case, I didnā€™t realize that the files needed to be inside of the users ā€œ./filesā€ folder /data/username/files and I was placing them in the /data/username folder. The scan indexed the files, but they didnā€™t show up in the web interface. Moving them to the correct sub-directory and rescanning fixed it.

2 Likes

Hi. How can I migrate when Nexcloud using S3 has external storage. Becauseā€¦ the directory structura doesent exist.

For exampleā€¦ I have directory structure with files (/tmp/user1 for example. 1GB of data distributed into 100 files) and I need upload the directory structure and its files into an spesific user account. Butā€¦ I not known the password of the user. Im know the admin user and password.

How can I do?

My regards.

Thanks for sharing this information with us, printer offline fix helped me to get the solution of this. I never knew bout the console.php script to force a scan and update the database so I always spent a few days using the sync app for each user.

Would be nice to do something like this from one NextCloud iocage to another NextCloud iocage. This is cause my instance was done from third-party install and some features donā€™t work.

My db and files folders are located outside of the iocage/jail. Theoretically, if itā€™s safe to map the files folder without the data being lost, that should be as simple as editing the files location, but Iā€™ve yet to get any replies on the subjectā€¦ aside from ā€œdonā€™t knowsā€. There are no comments or contacts, etc. There are some notes that would be easily copied over.

Thanks in advance!

My db and files folders are located outside of the iocage/jail. Theoretically, if itā€™s safe to map the files folder without the data being lost, that should be as simple as editing the files location, but Iā€™ve yet to get any replies on the subject get all the help from the Epson Printer Error code 0xf1

Hi,

I am new to the community and a question regarding this post.
I have a one node googleā€™s Ganeti cluster with multiple VMā€™s, nextcloud and the backup VMā€™s are the only VMā€™s with external data storage. The Ganeti master /host) and all VMā€™s run Ubutuntu 16.04.

  • For nextcloud I have a raid 5 attached over nfs from the host (Ganeti) to the guest (nextcloud)
  • I am using samba 4 AD to manage userā€™s accounts
  • the userā€™s folder in the data directory is the UUID of the users

Now I want to move from Ubuntu to debian for the Ganeti master (host) as well as for the VMā€™s (guests). That means I have to delete every thing, keep the existing raid and reinstall host and guests.
So when setting up a new samba 4 AD and create the users again, they will have a different UUID.
Is there any way to migrate the data of the existing accounts to the new accounts with the new UUID?

Thanks in advanced

After moving to Debian I decided to move from mysql-5.7 to mariadb-server 10.1.37-0+deb9u1, from the default stretch repo. So I did the following:

  1. installed alll needed packages from default debian pero which are needed for nextcloud
  2. created my nextcloud vhost with the new server name
  3. copied the /var/www/nextcloud/directory to the new server
  4. made an sqldump from the old server and copied it to the new server

The first problem ist that in mariadb-server 10.1.37 the innodb_large_prefix is not enabled which caused me a headache with some tables while importing mysqldump. While it is enabled in mysql-server 5.7. So I did the following:

# mysql -u root -p
MariaDB [(none)]> SET GLOBAL innodb_file_format=Barracuda;
MariaDB [(none)]> SET GLOBAL innodb_file_per_table=ON;
MariaDB [(none)]> SET GLOBAL innodb_large_prefix=1;

the log out from mysql. Next I created the nextcloud database and imported the mysqldump and upgraded:

mysql -u root -p -e "CREATE DATABASE nextcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci"
mysql -u root -p nextcloud < /tmp/nextcloud.sql
mysql_upgrade -u root -p --force

then I added the certificate for apache ssl vhost and LDAPS and restarted apache2 and mysql.

The last step was to add the new IP of the server to the hosts /etc/exports and to auto mount the share in /etc/fstab in the new nextcloud server.

And there you go, everything is there and works perfectly.

NOTE:

if the new nextcloud server has a different host name, you have to do the following:

  • replace the server name in the apache vhost

  • replace the server name in /var/www/nextcloud/config/config.php

  • replace all appearances of the old server name with the new one on nextcloud.sql dump before importing it to the database:

    cp nextcloud.slq nextcloud-org.sql
    sed -i 's/old\.cloud\.server/new.\.cloud\.server/g' nextcloud.sql 
    

Done

Note:

If you are using mariadb fro debian buster which is mariadb-server 1:10.3.13-1 , the innodb_large_prefixis reenabled due compatibility. I didnā€™t tested it yet though.

1 Like

Everything is working well except that the scan doesnā€™t result in folders or files that are seen by the nextcloud client. The output is the same as above (ie the small table) and the files are in the correct folder (on an external local disk) but just cannot be seen by the nextcloud client.

If I create a file or folder in the folder that theyā€™re in on the nextcloud client then these are seen both by the client and by cdā€™ng into the folder on the server.

I assume that this is a permission issue? If so what permissions should I assign?

I think it would be a case of reassigning the correct permissions:
sudo chown -R www-data:www-data /path/to/nextcloud/data/
In pure laymanā€™s terms, the process in total seems to be: move the files, change their ownership, run the php console.
At least, thatā€™s what worked for me!

Hey! this is an awesome tutorial shared by you, I am a general user of the NextCloud services. I have no idea about more but I try to use the NextCloud client sync app and or browser upload features.
I have an issue when goes to print from the cloud then appear hp printer offline how to resolve this problem.

Hey, i know this is an old thread however iā€™m having an issue. I have migrated successfully however forcing Nextcloud to search for the files is having a mental breakdown.

The following i am getting in my terminal and have no idea what to do as the folder its stating missing is actually there.

Command - root@freenas:/mnt/NAS/iocage/jails/Nextcloud/root/usr/local/www/nextcloud # php console.php files:scan data

Error - App directory ā€œ/usr/local/www/nextcloud/appsā€ not found! Please put the Nextcloud apps folder in the Nextcloud folder or the folder above. You can also configure the location in the config.php file.

Iā€™m running on a FreeNAS, the previous parts in the tutorial worked a charm itā€™s just this part im having issues with.

Hi
Great post helps tons. I am in the middle of switching drives but I have a slightly different setup due to Nextcloudpi, I believe. I will attach a screen shot. not sure what to do with the linked directories.
Thanks in advance.

Appreciate this thread is a little old now, but just chiming into say it was helpful, though my Docker container didnā€™t have sudo installed. The trick was to exec the container as the www-data user:

docker exec -it nextcloud-app --user www-data /bin/bash

then I could run the php console.php files:scan --all command and it scanned the files - no sudo etc needed.

It should be:

sudo -u www-data php occ files:scan --all

I am doing it right now :slight_smile:

Thanks for clear tutorial. Do you know how to do this with snap version of Nextcloud?

defininatly a valuable Tutorial! Thanx

It made my day, as I moved my Synology Drive Data to Nextcloud with ease :slight_smile:
I just skipped the external drive and copied directly into the Folder
not to forget to change ownership the new folder to http:http for synology-users:

chown http:http <new-folder> -R