I am following the steps for backing up my Cloud data provided here:
Still there are 2 open questions in my mind:
1.) Whatās the main difference between using nc-backup including my data, and nc-snapshot? Can I use nc-backup excluding data and then doing nc-snapshot, having all my relevant data together and ready to easily restore?
2.) If so - why does nc-snapshot by default (without any UI option to change that) create the snapshot on the same drive the datadir is on? Is it safe to just move the ncp-snapshots directory to my backup drive? And can I, without interfering with anything, alter etc/nextcloudpi-config.d/nc-snapshot.sh to create the snapshot directly on my backup drive?
nc-snapshot uses no additional space, while backups with data store a whole copy of your data, which could be many gigabytes. You can have hundreds of snapshots of a 20GB datadir in a 100 GB harddrive, but only five data backups.
Yes, I recommend doing that
Due to the nature of BTRFS snapshots, the snapshots need to be created in the same drive. There is no option to change the path for simplicity.
If you want a second copy of your snapshots in another hard drive, or in another machine, you can use nc-snapshot-sync.
This tid bit was what I was looking for. It answered one major question for me. I have been trying to set up this backup strategy you have written about, before entering any data at all.
Now When I try to set up nc-snapshot-sync I get an error:
stat: cannot read file system information for ā/media/hc2disk2/ncp-snapshotsā: No such file or directory
/media/hc2disk2/ncp-snapshots is not in a BTRFS filesystem
Running btrfs filesystem show hc2disk shows up just like my data location. If I run ls On my data location ncp-snapshots directory shows up. What am I doing wrong?
Update: I found that leaving off the /ncp-snapshots in the destination path allows the sync to take place, and snapshots are showing up, but they are not in their own folder like in the data location. Is that the way it should be?
Hey, could you maybe help me stting my nc-snapshot-sync up? Im only a beginner but for me it seems as you have quite a bit of knowledge. Nobody else is answering my question so you are my last hopeā¦ Here is my Question.
I would appreciate any help!
Iāve been trying to wrap my head around this whole backup strategy for daysā¦
I have two 480GB SSDs I want to use; the first as the primary data storage, and the second as the backup. Given that snapshots appear to be the best way to go from a data efficiency standpoint, but given that
snapshots only work on a BTRFS formatted drive, and
you cannot install NCPi on a BTRFS formatted drive [<- this is my understanding from NPās post above]
ā¦my perception is I need to install NCPi on an SD card and boot from that, and then change the nc-datadir to the primary SSD. I can then enable ncsnapshots (as well as a NCPi config backup) on the primary drive, and use nc-snapshot-sync to push a backup to the secondary SSD.
My main challenge so far was trying to figure out how to format the SSDs into BTRFS using the NCPi nc-format-USB, given that I had a bunch of data on one of them I needed to transfer into NC before formatting into BTRFS. However, prior to reading this thread, I thought I could get away installing NCPi on one SSD and using the second as the back up for snapshots. My preference for booting and running NCPi from the SSD is Iāve read itās faster. But Iād rather have a robust backup strategy.
Is my understanding of how this all works correct? Am I missing anything or is there a factor Iām not considering / aware of?
In your reply to @zweiflamme you encourage using nc-backup in combination with nc-snapshot. Iām trying to understand the best way to actually perform that.
I currently have NCPi booting from an SD card on an RPi 4, connected to two Kingston SSDs (primary and backup). Iāve set nc-datadir to a folder on the primary ssd. Iāve also activated nc-snapshot-auto and nc-snapshop-sync, with sync pushing to my backup SSD.
My question is what / how should I use nc-backup with this configuration? Should the destination directory be on my primary SSD, or the backup SSD? Should I run nc-backup to both SSDs? If I run nc-backup to the primary SSD, is there a way to have that automatically sync with the backup SSD?
Lastly, I still donāt quite understand how snapshots work as a back up for data? I understand conceptually that itās an incremental save for data that has changed since the last snapshotā¦ but what about the baseline data? If I upload 20GB of data, where / how is that 20GB of data backed up (assuming I run nc-backup without checking the include data box)? If a snapshot is only storing a data change referenced against the full 20GB of full data, how does a full data restore work if the SD or SSD that contains the data directory fails, and all you have are the snapshots that have been syncād to the backup SSD?
I think you understand correctly. If you flash the ncp img to a SSD and have no SD, there is no way to use snapshot.
To format a USB drive, the drive has to be empty. You canāt keep any data.
use nc-backup with a path directly to the backup SSD. No need to have it first on the datadir.
Iām working with the SSD boot setup and have set a full-auto-backup every 5 days to be started some time late at night. As I use USB 3 it only takes about 3-4h for over 400 GB of data. So I can live with that. Iāll just be frustrated If I have a hardware failure on day 5 before the backup. heheā¦
I also still donāt fully the point you describe about snapshots.
But as I have a spare Piā¦ Some day I will play around with it and find out. If by that time there is still no foolproof explanation, I will write one.
Thank you @michuvon. Iāve got my system set up now as you describe. Insofar as my question about the snapshotsā¦ my understanding is the snapshots are capturing the incremental changes to my data that occur over time. What is less clear to me is where the āfoundationalā data lives? For example, Iāve set up Nextcloud and uploaded 15GB of data within a 24 hours period. Does the first 24 snapshots āstoreā that 15GB, and then each subsequent snapshot whatever additional data is added (or changed) going forward? I guess what Iām trying to understand is it sounds like the snapshots are a relative reference to a main data source, but I donāt quite understand how that relationship works functionally.
Hi
Iām new to this Nexcloud system (and not that proficient with linux systems) and have recently built a Nexcloudpi instance running on a bootable 240gb ssd drive. Having done this, and all seeming well, iām now trying to get my head around a backing up strategy - hence my reading this thread.
My original thought was to initially back up to a duplicate 240gb SSD to guard against disk failure and to utilise the snapshot facility to guard against idiotic/accidental deletions.
Reading around, it seems to me that in order to use the snapshot facility, the instance has to be on a btrfs formatted disk. As I understand it, my installation process has created ext4 volume for the data, from the image file I originally wrote to the SSD to begin the installation - is this correct?
So, firstly - is/was there any way to have a bootable usb drive installation and have it create a btrfs volume instead of an ext4 one?
With my current setup, is it possible to resize the ext4 volume and create a btrfs volume on the same physical SSD, and then migrate the required data folders over to the btrfs volume so as to enable snapshots?
If I were to attach a 2nd SSD drive to the raspberry pi and have it formatted as btrfs and then migrate the required data folders over to it, thus allowing snapshots to work, could I then sync/copy these back to the original ext4 drive, so as to have backup copies of the snapshots together with fuller backup files in case the 2nd SSD drive failed?
i.e does it matter if the snapshots are subsequently copied and stored on an ext4 volume?
If you use the Raspi image proviede by @nachoparker , no, that is not possible.
This goes in to advanced areas, where Iām not feeling comfortable to give advice. So so take it with a grain of salt.
ā Maybe you could try to use Gparted to do that. Using another Linux machine or the command line. After that you would have to manually set the path for your directoryā¦ i guess
Sounds very complicatedā¦ and a waste of resources. If you have time try the one above (2) and please document it to post your steps here in the forum. If you get it to work we could even add it to the NCP Documentation page.
just thought iād update my investigations into this in case itās of use to anyone
I had a go, last night, experimenting with another instance, that i had originally written to a 32gb bootable usb stick, to see how i could try and achieve this.
So, using gparted, on a seperate raspbian installation, and then connecting up the usb drive, i shrank the ext4 partition to 10gb, then added btrfs functionality to gparted (sudo apt-get install btrfs-tools) and then created a btrfs partition in the space i had released (sda3 - 19gb), Labelled it ābtrfs1ā and then booted up using the usb.
The Nextcloudpi instance seemed to still run fine with the resized ext4 partition, so i then first used the setup wizzard in the web interface to enable usb storage, but did not format when asked. I then used nc-datadir to change the data location, whereby it threw up some error messages about permissions, but showed the location of the data as now being /media/btrfs1/ncdata in the settings, and i confirmed this location actually existed via command line.
However, there still seemed to be issues of permission and mention of the location not being properly prepared (i should have cut and pasted the errors but didnāt). It still showed several of these errors in nc-datadir even after several reboots.
I then tried the wizzard again,this time selecting the format usb option which seems to have simply removed the btrfs (sda3) partition along with the other partitions on the usb.
