Feedback on plans for migrating from a Digital Ocean instance to a physical server

Let me start by saying that I don’t have a particular problem—rather, I’m planning to migrate my Nextcloud instance to a physical server in a few months and I wanted to ask for general feedback on whether my plan sounds workable, if there are any pitfalls I should watch out for, and if there’s anyway I can modify the sequence of steps to make my plan more robust to failure. I have included a few specific questions at the end, though.

I have never done a server migration before, and am a fairly new user of Nextcloud, but I have used Linux as my primary OS for a few years and am comfortable moving files around, manipulating configs etc on the command line.

Migration goals

I have: A working Nextcloud 24.03 instance running on an Ubuntu Server 22.04 Digital Ocean droplet. Nextcloud is installed as a Snap according to this tutorial from DO. I own the domain nextcloud.example.com which is pointed to this instance by an A record.

I want: To migrate Nextcloud to a physical server—either a used desktop or a Raspberry Pi—with the file storage mapped to a RAID-1 array. In the process of this, I want to add about 50GB of music files to Nextcloud that don’t fit on the DO Droplet.

I have read through the server migration page in the docs as well as the import/export tutorials linked there. Since I intend to use the Nextcloud snap both before/after migration, not all of these instructions are relevant, and I can do things more simply using sudo nextcloud.import/export as described here.

I’m the sole user of the Nextcloud, so I can handle a few days of downtime if necessary.

Planned migration steps

Setting up the hardware

The first few steps don’t involve changing the DO Droplet at all—I will continuing using it while doing the following

  1. Get the physical server: Either a Raspberry Pi or a used desktop.

    • In the latter case, the Pi will have its OS installed on an SD card or USB drive, and two USB external SSDs or HDs for the RAID array.
    • In the former case, I’ll look for a desktop that has space for one NVMe drive for the boot partition and two internal SSDs or HDs. Or just three internal storage drives.
  2. Install Ubuntu Server 22.04 to the boot drive.

    • Or maybe install a lightweight desktop distro like Xubuntu so I can manipulate files with a GUI if I want to. If I do this, what other packages do I need to install to bring it to feature parity with Ubuntu Server?
  3. Use mdadm to set up the RAID array, with a permanent mount point, say /media/raid0. Set this to mount automatically on boot. This guide seems OK if a little outdated.

  4. Copy my music files to a temporary directory on the RAID array, say /media/raid0/music

  5. Set up a static IP address for the new server.

    • I plan on doing this from my router settings first, then following this tutorial for Ubuntu.
  6. Get SSH access working with firewall and API keys, test that I can SSH into the server from offsite.

  7. Now go to the new server a run snap install nextcloud. This might require doing some boilerplate config steps, but it doesn’t matter because we will override these settings in the next step.

Migrating the NC instance

Now the Nextcloud migration itself:

  1. Put the DO instance into maintenance mode.

  2. From the DO droplet, run snap nextcloud.export from here. .tar this file and rsync it to a USB drive (it shouldn’t be more than about 2GB).

  3. Copy the file from the USB drive to the /var/snap/nextcloud/current/ directory on the boot partition, chown to root, run snap nextcloud.import on this file as explained here (same snap migration link as above).

Now, before taking this instance online, I want to configure it to use the RAID array instead of the boot partition for file storage.

  1. Following this tutorial from the NC snap docs, first I give the NC snap the removable-media permission.

  2. Stop the Nextcloud service, move the files currently in /var/snap/nextcloud/common/nextcloud/ to /media/raid0/nextcloud/, change the file storage path in config.php accordingly, restart the Nextcloud service.

Verifying that the migration worked, migrate URL

  1. Take the new server’s NC out of maintenance mode and type the new server’s IP address in the browser to make sure that all the files, database, etc. were imported properly.

  2. If everything is working, change the A record of nextcloud.example.com to point to the new server’s IP.

  3. Wait a few hours for the change to propagate across DNS.

Now, if I go to my client devices and trigger a Nextcloud sync, they should sync seamlessly, reporting no file changes, right? (:spaghetti:)

Adding the music files

  1. Move the /media/raid0/music directory to /media/raid0/nextcloud/files/fun/, where fun is my NC user account name.

  2. Use sudo -u www-data php occ files:scan --all to trigger NC to detect these new files.

The purpose of steps 4 and 17 is to avoid having to transfer my music library over the network. I know that dropping files straight into the NC files directory is an anti-pattern because NC expects “sovereignty” over its data directory, but for a one-time operation this should be safe, right?

Finally, if this setup works normally for a week or so, I will shut down the DO droplet.

Questions

  • Is this plan workable? Is there anything I have overlooked?
  • Are there any ways to optimize the sequence of steps above to minimize the amount of time my NC spends in maintenance mode?
  • Any advice on whether I should choose the Raspberry Pi or desktop server option?
    • Note that if I use the Raspberry Pi, I still intend to use Ubuntu instead of the NextcloudPi image since it’ll let me install NC as a snap and have access to the easy import/export feature.
  • Is the statement above (:spaghetti:) about seamless syncing after migration true?
    • “My client devices” includes various desktop/mobile Nextcloud client apps, as well a few third-party apps that I have given access to NC using app passwords, as well as the Android WebDAV for calendar sync.
  • Is my proposal for transferring in my music files locally (steps 16–17) safe? Would it be better to stop the Nextcloud service and/or or put it in maintenance mode for these steps?
1 Like

Hello,

It’s snap and you already have a running setup so your chances of a successful & easy migration is really good.

Your write-up is bit of an overkill but you are right on documenting every step. Helps a lot in actual migration.

I have done similar migration for a snap setup (VPS to physical desktop). Here are my personal view on the process in no particular order.

  • When you are choosing physical server, few things to keep in mind with file servers like Nextcloud →
  1. Storage speed won’t matter unless you are on 10Gbit+ network since with standard 1Gbit LAN/WAN, HDD is faster than network.
  2. SSD or NVMe boot drive won’t matter with nextcloud server, unless you are rebooting it every few mins.
  3. Nextcloud benefits heavily from a faster or high core count processor. More user you have with simultaneous use, better processor it requires. Same is true with RAM use also. A personal setup for 8-10 users may do just fine with 4GB RAM and a decent multi-core processor (think 3rd gen i5+) but if you are adding resource hungry apps on top of it, consider 8GB or up depending on your user count and apps installed.
  • Snap export (backup command) in it’s default state will backup everything (nextcloud data, database, config, apps) so be-aware, backup size should be much lower than your total available space of the VPS. → Usually in my case, I would use -bc with that backup command to just backup database and config and copy the entire data directory manually. → When restoring, same process, just copying the user data folders to new installation’s data directory (don’t overwrite any files of new installation) and then restore the backup with that -bc command.

  • However, since your total user data size is less than 2GB, you can safely proceed with default snap backup command and backup everything.

  • Your backup won’t contain your local server’s LAN NAT IP as trusted domain so run the snap command to add that IP as trusted domain to access NextCloud via your LAN NAT IP.

  • For that step 12, after you move the files, you may need to reconfigure that new directory’s ownership (root:root) and permissions (0770). Not sure why, but in my case, it won’t without re running the ownership and file permission commands !!

  • For music files, I find it lot easier to simply upload them via web interface when the server is successfully migrated to local physical LAN and running.

  • One strange behavior I have noticed with my setup. New Server, after all data copying and backup restoration, acts up for first few mins. Random & wired errors. After few mins and attempts, it starts working and then after a reboot, it’s fully functional and works flawlessly. Not sure why.

Overall, all looks good.

To simplify,

  1. Run snap backup command in your DO to Generate a full backup
  2. scp or rsync to download that backup in your local machine.
  3. Install Ubuntu Server in your local physical server (set static LAN NAT IP) and then install snap nextcloud
  4. Restore that snap backup → Add your local NAT IP as trusted domain
  5. Done → Enjoy

Thanks.

1 Like

Thank you so much! I appreciate the tips about device selection. I don’t expect many simultaneous connections, but I’d like to set my server up with a little bit of extra specs in case I want to try serving other apps besides Nextcloud in the future.

Re: adding the new server’s LAN NAT IP as a trusted domain, I see on this post on the forum here that the way to do this is to edit config.php and simply add a key for the new IP address. This is also how this random tutorial says to do it.

But you said there is a snap command for doing this—would you mind elaborating on this? I don’t see any mention of trusted domains on the nextcloud-snap GitHub. In the linked post, the guy used

$ sudo snap run nextcloud.occ config:system:set trusted_domains 1 --value=your.fancy.domain

but this seems like a way to add nextcloud.example.com (my fancy domain) rather than my LAN NAT IP. Or does this command automatically populate the local IP into config.php?

And “LAN NAT IP” is just the normal IP address I see in ifconfig under inet, right?

Edit:

Oh, I think I figured it out. This line

Your backup won’t contain your local server’s LAN NAT IP as trusted domain so run the snap command to add that IP as trusted domain to access NextCloud via your LAN NAT IP.

pertains to the interim time when I am setting up the new server but still have nextcloud.example.com directed to the old one. At this time, if you just type the new server IP into the browser, you’ll get a trusted domain error because this IP address isn’t in config.php’s trusted domains. (The trusted domains on my working installation are localhost and nextcloud.example.com and no IP addresses.)

Therefore, the command you are referring to is

$ sudo snap run nextcloud.occ config:system:set trusted_domains 1 --value=123.456.78.9

where the value is just the normal IP from ifconfig.

Right?


Interesting, I wonder if this results from some kind of data race where Nextcloud is trying to index the files at the same time as the filesystem?

1 Like

Hello,

That’s correct.

By LAN NAT IP, I meant the local LAN Static IP of that physical Ubuntu server NIC

Your Backup will have the domain added in it’s trusted domain section so after restoration, Nextcloud will work fine when accessed via domain name but if you wish local access, then new IP needs to be added in the trusted domain section.

No idea. I am no expert on this. It is only my personal observation !! Just mentioned it since if it takes a while, someone impatient may assume things didn’t went right and give up early. I did, on my first attempt :yum:

Lastly, with snap edition, it’s snap which is suppose to handle everything, there are snap commands for most of the configuration related tasks. It’s better not to touch config.php unless it’s absolutely necessary (like that data directory movement) !! Let snap do everything since that’s the whole idea behind snap.

Thanks.

1 Like