I am using Nextcloud 19.0.3 with S3 as primary storage. What is the best method to backup contents in S3? I don’t have much experience with the Nextcloud. But I know it’s somehow linked with the Database to display the files. As files in the S3 are shown in some random names( like urn:oid:11121).
As an admin, I want to restore files even if the user purposefully deletes the files(Trash also) or files are deleted through some malicious attack. Is it possible?
I saw there is a versioning option. Is it suitable for my case? if the user deletes the file reference in DB will also get deleted right? So can files be restored?
Nextcloud is already Live. Is it possible to implement versioning now? how it can be done?
Please let me know if there is an alternative method for backup or I there any other documentation link regarding S3 as primary storage backup.
The poster hasn’t asked about the S3 object identifier, and as the objects are replicated, solving the URN:OID issue in the primary region bucket will resolve the issue of the ‘unknown’ URN:OID’s in the replicated bucket.
However the original post related to ‘backup’, which in itself is many things to many people.
There is also very little detail in how the OP has setup their ‘S3’ primary storage and if it’s AWS or S3 compatible.
We use S3 compatible as primary storage, with region replication and has never experienced the issue of apparently orphaned URN:OID’s
My understanding is that NC allocates the urn:oid:xxxxxx and stores it in a lookup table effectively, between the actual and object name. So querying the NC DB will give the original file name?
So I am assuming the OP is also backing up the NC DB somewhere? Might as well be to the NC S3 bucket, and have the DB backups replicated cross region too for better disaster mitigation.
i’m not sure if you could just copy the s3 objects to another region instead of syncing. one could use external tools if aws doesn’t provide this as a feature.
so in this case you would end up with a lot of orphans. or?
I also consider AWS S3 as a good option. But in my client office, they were using some other storage facility. We just implemented NC as an alternative as they were having issues with their storage facility. The issue occurred when one of the employees resigned and while leaving he/she deleted the files. So my client doesn’t want to happen it again with NC.
AWS S3 or S3 compatible storage?
Ans: AWS S3 is used and Nextcloud is installed on ec2 server both are in same AWS region.
AWS S3 or NC versioning?
Ans: I am not sure which one use. I don’t want to lose existing user data. As per my understanding. If I install and enable NC versioning, it uses S3 versioning in the background right?
install and enable the versions app? talking about nc versioning. aws s3 versioning -> aws console.
Ans: Any suggestions which one to use?. I was planning to use NC versioning and I hope if I have installed and enable it now it won’t affect existing user data right?
Thanks, Cross-region replication of file along with DB is also a good option for backup. As I don’t know working NC, so my concern was about losing of reference in the database to files when the user deletes the files. As per my understanding backing up files alone is no use when AWS S3 primary storage is used. But in my case, as the user is purposefully deleting the files. I think files will also get deleted from the replicated region. right?
Also restoring the files of a single user with the help of DB will be a huge problem.
Sorry i do not use AWS S3. But i think there must be a version control and history backup that not a single user can delete the data. And perhaps it costs money.I think you must also have in AWS S3 backups from yesterday, last week, last month, …
If you not trust AWS S3 for temporal backups but more your own backups please do not use AWS S3 for primary storage. It makes no sense.
i’m not sure. but aws s3 versioning isn’t used neither by nextcloud nor by restic. that is to say if you turn on aws s3 versioning neither nc nor restic are not aware about that feature. so if you change a document in nextcloud that would result in two different urn:oid:xxxxxx objects. not two version of the same object. (someone with deeper knowledge may confirm or correct this.)
aws versioning could make sense to protect your documents from mal-/ransomware. the only thing you have to do is configure everything in this way that an attacker won’t get hand on the credentials to turn off versioning and would be able to delete old versions. i think that can be achieved with different iam roles. so if ransomware is encrypting all your docs in nextcloud and get hands the back mechanism you would have still and older version of the backup. hope you get the idea.
if your documents needs to be Armageddon proved it would make sense to use the cross region replication from aws.
If version roll-back is something you’re after why not just use:
As for if a user deletes files why not just get the user to make use of the deleted files functionality of nextcloud to retain deleted files and user restore them?
This would seem to completely fulfil your stated senario and is already available within nextcloud. No other infrastructure to implement, configure or manage!
If you wanted the belt and braces, then cross region replication of files and databases is the additional implementation you need. AWS S3 by design is, within a region, safe. AWS S3 file durability is detailed here: https://aws.amazon.com/s3/faqs/