I had a major failure with a local client when trying to sync down 110GB of photos (100s of photos deleted to recycle bin, some uploaded again. It is a complete mess). I figure my best course of action is to just do a filesystem rollback since I run Nextcloud on top of a ZFS file system.
My question is in how I should handle the database. To be honest I don’t really understand what is stored within it and how it interacts with Nextcloud. Do I need to rollback the database to? I can restore it with a snapshot rollback to, but that snapshot is not taken at the exact same time as the data snapshot and I am not sure if it might cause me problems if there are “mismatches” in between them. Or can I just leave it like it is and continue using it after a occ:filescan?
The failure happened two days ago in case it have any impact. I have made minimal file changes since it happened, and saved some task lists + bookmarks. These small changes are not important to me. My photo collection is what I value most
Before you do anything now take a snapshot you can restore to if you need to try again. I would rollback both ZFS and the server, then OCC scan. You may get away with not rolling back the server and an OCC scan will allow it to push the changes to the database, but it may be worth going back to when everything was clean.
Thanks for the answer! Just to clarify, If i try to just rollback the data and scan to fix the database, how would a failure be noticed? Data loss? Error in when data was last changed? Possible failure when I try to sync to my clients?
Running OCC scan should just update the database on the current state of the data directory. I can’t imagine there would be any database errors (though your logs will tell you otherwise) from the client doing a mass-delete, so you should be OK. Just ensure nothing is actively connecting and manipulating data while you do this.
Hi @Kebba, could you keep us updated how restoring your data goes? I can’t help you actually, but while I’m planning to setup a new server with ZFS and Nextcloud 12 as well, I’m curious if snapshots and rolling back works smoothly.
So you don’t have the data directory and the database on the same dataset, right? Is there a reason for that? Being a total newbie to ZFS I would have stored data directory and database on the same dataset as both are pretty much connected, in the meaning of: if data changes (like names or new files) the database will be updated accordingly. So I would imagine that rolling back both with the same snapshot might be the best in regards to data consistency. Or do I miss an important point with ZFS, datasets and databases?
There’s no explicit reason they shouldn’t; shares (links), settings, etc will be lost in some instances where the files are no longer present or the DB has been rolled back, but otherwise for the file pointers an OCC scan while the system is completely idle will bring the two in sync.
On many distros the database is installed onto the OS disk, while ZFS will use other disks/partitions. Unless you take steps to actively move this all around then you’ll typically be in this situation. Shouldn’t be a cause for concern.
The restore seems to have worked alright following the instructions I got, although in hindsight it seems like it was not necessary. I though pictures were lost since size did not match my offline backup. It must be some reporting difference since the same is still true, but the file count match. Might be something with the storage being a ZFS pool and my cold backup is a Veracrypt encrypted USB Harddrive.
Anyway:
It worked very smoothly, I restored a backup from 6 days ago, no problem at all. The entire process took 20 minutes for 330 GB, the most of the time was doing the file scan. The rollback is done in 30 seconds. Just remember that Nextcloud treats the server as “master” and will delete all clients local files created after the restore point. So if something happens, don’t wait long to restore and/or be careful.
About the datasets: I have them separately. I was a complete newbie when I set up this system, so I have done some stupid mistakes in my setup. I set up a system for my dad recently, and he has a dataset with two child datasets within. One for data and one for database. Snapshots are made on the parent and recursively as you suggested. I however recently learned that a dataset containing a database should be set to a 4kb block size (equal to the databases) for increased performance, very easy for a newbie to miss. I don’t think a single use server will suffer to much though.
Another thing I learned the hard way (If you already know this, please disregard this): I assume you are going to use FreeNas or similar? If that is the case, mounting a dataset in a folder will replace what is in the target folder. That means that you should NOT install mySQL and then mount the dataset. Mount everything in the jail in the correct folders and then install everything. That way you save yourself quite a lot of headache.
8 GB out of a hundred, so it is very possible. Stupid mistake to make, but oh well. I modified only 6 files since the event so still worth it to be 100% sure, I value the integrity of my pictures over everything. I also got rid of 35 GB unnecessary recycle bin data
Thank you very much for your replies and explanation. This prepared me a lot more for setting up my server!
Especially the thing about child datasets and the 4 KB blocksize for databases was something I didn’t know at all and I would have made that mistake for sure
So to be sure what to do, it is best to create a parent dataset and then one child dataset for the NC data with “normal” blocksize and one child dataset for the database with 4 KB blocksize? This is possible, right?
I was not planning to use FreeNAS on my NC server, no. I currently think that it is best not having to many services on one server; security-wise and performance-wise.
Please correct me if that shouldn’t concern me at all, but at least performance is very important to me, because I’m offering NC to 10 users and want them to have the best experience possible.
So this will be a complete new server where I plan to format all drives with ZFS and then install Arch Linux, then Nextcloud and then Onlyoffice. No more services are planned so far to run additionally on this server.
GB vs Gib thing sounds like a problem I should also read into it Do you have a link for me @JasonBayton that quickly points me to possible backup issues?
That’s via LXD on Ubuntu, though I believe it can be installed on other Linux systems now.
It’s not a problem, just a pause for thought - ZFS shows everything in GiB, which is larger than GB, so 4GB is 3.7GiB and as such has admins thinking they’ve lost data when switching back and forth between backup systems that aren’t exclusively ZFS.
Should be that yes, but not 100% sure.[quote=“Schmu, post:13, topic:14226”]
I was not planning to use FreeNAS on my NC server, no. I currently think that it is best not having to many services on one server; security-wise and performance-wise.
[/quote]
Should not be a problem. FreeNAS is the OS, and every other service is run in a jail for containerization. Jails basically behaves like light weight Virtual Machines, and the security implications should be minimal. I run 6 jails (webserver, plex, music streaming, NC and some other small stuff) on a 4 core 1.83Ghz Atom CPU without problem. Running OnlyOffice is a pain today however as docker containers are not supported yet, so that should be accounted for. I am not saying you should run FreeNAS, just that you could if the software suited your needs. Good luck!
@JasonBayton: I read the size from Nextcloud GUI, not from the OS, but I guess Nextcloud gets its info from the file system?
I suspect that guide won’t work on FreeBSD? I am really looking forward for docker and VM support on FreeNAS. Everything will get much easier, especially for beginners such as myself
Yes, that is what I wanted to say. I just did not the right words
Do you have a link for me 
