Hi! I dont know if I post at the right place, I’m new here and find the forum pretty hard To understand.
I’m a newbie in this world and trying To setup my own cloud in order To store personal files.
My cloud works at this moment (running it on a raspberry pi 3 with dietpi os), but I use a USB key as storage.
I’m pretty afraid that the USB stick could fail at any moment and I wouldn’t like to loose my important files. I’d like to know if it is possible to have a mirrored storage.
What I mean is that I’d like to have 2 storages (2 HDD or USB sticks) plugged in my pi and everytime I upload something on the main storage, it copies itself on the second one.
It would be even better if everytime I drop a file in the cloud from somewhere in the world, it copies itself on my main pc, but since its not always running I dont think that could be a thing.
Is such a thing possible? If yes, how could I do that?
Last question is : If I got 2 storages plugged in the pi, can i switch from one To the other while being on the web panel? If yes, it could be a solution for me to upload twice each files even tho it would be less convenient.
That would be my thought as well.
What is your backup strategy? How are you doing your backups (files and db)?
You can mirror files pretty easy using rsync.
One recomendation: Do not use USB key storages at all.
I would consider that more of a backup than a mirror.
I don’t know if RPi supports this (and this is more of a Linux question than a Nextcloud question), but perhaps you could set up mdadm software RAID-1 for the USB drives? That would be an actual mirror where the system would continue to run if one of the drives failed.
I successfully set up a mdadm raid 1 for my two usb drives on my pi
now, I have put some files in the cloud and I’d, to be sure, that when I plug the usb drives in my pc, both have the same files in it, but when I plug them on my pc they both are empty, but aren’t when I look in the cloud.
Could I have made a mistake and Raid isn’t working and my cloud files are in my sd card, or is it normal that I don’t see any files when I plug my USB drives in my pc?
— NB Raid 1 resync isn’t finished yet.
With a software RAID set, you won’t be able to just take one drive out and use it as a normal drive. It has special formatting due to the RAID configuration.
The drives are used simultaneously, so there is no need to switch from one to the other. They are treated as one drive, and any changes are made simultaneously on both. The system will continue to run if one of them fails, and the failed one can be replaced and set up as a new mirror.
Also, it’s a bad idea in general to access the NC data folder directly (despite how many people seem to do it anyway). If you want to shut down NC and use the drive on a computer and then put it back, that isn’t compatible with the RAID setup.
I see, thanks for the swift answer.
Then, how will I know if a drive fails? And what could I do to get back my datas if the cloud fails?
Also, small question, my usb key keeps flashing red while the drive is running, but I’m not transfering any data… Is nextcloud constantly scanning the drive and if yes will it break down my usb key with time?
Know that for proper backups, you need to enforce the 3 copies rule :
Main copy ; always online and onsite by definition
One copy must be offsite to protect against physical incident like fire or flood
One copy must be offline to protect against logical incident like corruption
Software raid or second copy on a second drive in the same device would not even qualify as a second or third copy. Such a copy is still onsite and still online.
Here, my backend is a FreeNAS server. With ZFS, pool of mirrors and snapshots, it is a very robust first copy.
I do ZFS replication to a second server 400Km away to a second FreeNAS. That one is online, but offsite, so is my 2nd copy.
I also have a 3rd server that is on the same site as the first one. I keep that one offline most of the time. Once every other week and never less than once a month, I power it on. The main server detects it as soon as it is online and replicates everything to it. Once sync is over, I power it back down. That way, this server is offline, so qualify as a third copy.
Also, know that a backup is worthless until you successfully restored it. Here, I test my restore procedure at least once a year. Last time, I failed it. It allowed me to detect a problem in the way I encrypted the database backup. I fixed that and re-tested by backup / restore until I succeeded. Should I have wait for trouble, I would have ended up empty handed because my backups were corrupted and I did not knew about it.
Good thing for you to design and plan for backups. But be sure not to fool yourself into believing that you have a backup when what you have does not work / does not protect against all kind of losses.