Hi everyone,
Lately, I have been struggling with accessing my data over SMB as well as Nextcloud, which is not really what Nextcloud is designed for, as I have learned. The main reason, why I still absolutely want SMB shares is the seamless access to my zfs snapshots (I’m running Nextcloud in a FreeBSD jail on TrueNAS).
Deleted some files or folders a week ago, but still need them? Accidently overwritten some paragraphs in your document and unfortunately saved it? All of this can be corrected with snapshots WITH EASE! Just right-click on folder > restore previous versions, then either restore to some point in the past, or open the folder at that point in time, which I usually do. I then copy back the files and folders from the snapshots.
All this comes with a very acceptable storage penalty: my 700 GB of personel data currently have 52 GB of snapshot data. And I have snapshots every 10 minutes with a life of 2 hours, some intermediate snapshots, up to monthly snapshots with a life of one year. So really all scenarios are covered. It even protects from cryptolockers on the user side, just restore the dataset from the snapshoft previous to the encryption, as they are read-only and cannot be encrypted, except if the server is compromised.
What are the current options in Nextcloud:
- Versions: seems to have a nice time history, similar to my snapshots. But only for files, and no control over the frequency and age of the versions. For example my dataset with 10TB of movies and series does not need 10 minutes snapshots or a life of one year. It should only prevent me from accidental deletion or data corruption.
- Deleted Files: keeping your data 30 days after deletion, surely useful, but doesn’t protect from overwritten data or cryptolockers.
- Snapshots App: goes in the right direction, but it looks like it is not very actively maintained. There is an almost four year old feature request on Github for folder versions and one from last year for external storage support.
So, what is my idea: I’d like to see something similar like shadow copies in Windows for network drives. I imagine the workflow as follows:
- Context menu, click on “Restore”
- Get presented with list of all snapshots. Alternatively, a slider symbolizing the time axis could be displayed, in order to choose a date and time from the past, but I don’t really know the possibilities in Nextcloud UI.
- After selecting the time, the folder content from the past is presented. RMB on any subfolder or file(s) will allow only to copy, as they are read-only. Alternatively: Context menu reads “Restore”, which would skip the next step.
- Then, either the copy pop-up will show, where you can select where to copy it in the “today” dataset, or it could also just copy it automatically to the original folder
- The restored folder is appended with the snapshot name for preventing mixups.
Above, a flexible, but more extended workflow is listed. It could be shortened by skipping 3. and 4. Selecting the snapshot would automatically copy the data to the original path.
Below, you can find a series of mockup images. This is all copied together from a current Nextcloud instance. As you can see: in terms of user-interface, it’s all there. I think the effort for deploying this could actually be within limits. Back in the days, when I was using Ubuntu and missed the “previous versions” feature, I coded a simple script as add-on for Nautilus. This achieved exactly what I needed: right-click on any folder, choose a point in the past, and a new Nautilus window opened with the folder content from that time. I will also append this to my post, in case this could be useful for further discussions.
It could be debated, if this should be included to the existing Snapshots app, or be done from scratch. Versions from snapshots in the side-bar is probably still nice for short-term, single file restores.
So, what do you guys think? Possible? Interesting for many people?
Cheers,
weingeist
1 Right-click on file or folder > Restore
2 Get presented with list of all snapshots, newest first
2 Frequency and life determined flexibly by zfs snapshots
3 Select file(s) or folder(s) to copy
4 Optional: choose target folder. Note: only copy is possible, the snapshots are read-only
5 Restored data could be extended with snapshot name, or date and time. Either always, or only when in conflict with existing data
Example of Nautilus python script for accessing, and opening folder in an earlier version in a new Nautilus window
#!/usr/bin/python
import os
from Tkinter import *
from datetime import datetime
# debug log file
#home = expanduser("~")
#sys.stdout = open(home+'/Documents/scripts/tmp/restPrevVers.log', 'w')
##--- get path and split to folders
path = os.environ['NAUTILUS_SCRIPT_SELECTED_FILE_PATHS']
#path = '/media/weingeist/_import from paper/test1' #example path
path = path.rstrip()
print(path)
if not path.endswith('/'):
path+='/'
parts = path.split('/')
n_depth = len(parts)
##--- create snapshot path
# parts[1] + parts[2] for correct mount directory
path_snaps = "/" + parts[1] + "/" + parts[2] + "/.zfs/snapshot/"
snaplist = os.listdir(path_snaps)
snaplist = sorted(snaplist, reverse=True)
n_snaps = len(snaplist)
#print(n_snaps)
# clean snap names
timelist = [None]*n_snaps
for idx_snap in range(0,n_snaps):
tmp = snaplist[idx_snap].split('-')
objDate = datetime.strptime(tmp[1], '%Y%m%d.%H%M')
timelist[idx_snap] = datetime.strftime(objDate, "%a, %d. %b %Y at %H:%M")
# print (timelist[idx_snap])
#---------- get choice from user with Tkinter GUI -----------#
choice = 0
do_open = 0
def update_label(idx):
# print(int(idx))
l.configure(text=timelist[int(idx)])
l.pack()
global choice
choice = int(idx)
def leftKey(event):
w.set(max(0,choice-1))
def rightKey(event):
w.set(min(n_snaps-1,choice+1))
def enterKey(event):
print("--enter key pressed--")
confirm()
def escKey(event):
print("--esc key pressed--")
abort()
def confirm():
global do_open
do_open = 1
master.destroy()
def abort():
master.destroy()
master = Tk()
master.geometry("500x125")
master.title("Zeitmaschine")
master.pack_propagate(0)
master.bind('<Left>', leftKey)
master.bind('<Right>', rightKey)
master.bind('<Return>', enterKey)
master.bind('<Escape>', escKey)
l = Label( master,
text=timelist[0])
l.pack(pady=15)
w = Scale( master,
from_=0,
to=n_snaps-1,
showvalue=0,
orient=HORIZONTAL,
command=update_label)
w.pack(fill=X,padx=10)
b_cancel=Button(master,
text="Cancel",
command=master.destroy)
b_cancel.pack(padx=10,pady=10,side=RIGHT)
b_ok = Button( master,
text="Continue",
command=confirm)
b_ok.pack(pady=10,side=RIGHT)
mainloop()
# execute if confirmed with "Continue" button
if do_open==1:
# create path from choice
path_snap_choice = path_snaps + snaplist[choice] + "/"
for i in range(3,n_depth-1):
path_snap_choice+=parts[i]+"/"
print(path_snap_choice)
# open nautilus from chosen snap
os.system("nautilus '" + path_snap_choice + "'")
Thanks for the writeup!