Actually the upcoming Mountain Duck 3.0 (currently in public beta) does exactly what you try to achieve here. It will cache selected files locally while others will only be downloaded on demand using a virtual drive mounted as a drive letter.
I am not really convinced about the virtual drive solution because it will definitely break applications that need to access files from a new drive mounted at startup. Unless it is implemented as a service/daemon on boot any application that relies on files stored in Nextcloud wonāt be able to access them unless the drive is up and running.
Why not supporting placeholder files in the current folder configuration like for example Resilio Sync does?
So, can set it to make everything available offline, including all new additions. Like @cwmke, I use Nextcloud sync as a form off backup, so I want everything duplicated on my server and desktop.
Please, do not make Virtual Drive the āonlyā option. Iāve been a fan of FolderSync since Dropbox, and as an OS X/Linux user I would consider Virtual Drive a terrible regression in functionality.
āterrible regressionā depends on the use case.
Currently it is difficult (for me) to use nextcloud as a business-relevant network drive with linux (many users, many files; at least that is my experience with different clientside webdav implementations). If the Virtual Drive builds on another protocol (or leads to better performance for other reasons) I see this filling an important niche and making a use case possible that currently is not.
If the new method allows any NC folder to be a virtual drive it could work reliably with volume mount points.
Your apps will never know that there is some magic going on behind the scenes.
Iāve just installed the techpreview version.
First issue: folders with accented chars (French) have incorrect title (encoding) and are all empty when opened.
Example:
i tried it now on a new device and got massive errors, crashes and so on:
It tells it canāt syncronize folders what it shouldnt do anyway as you told:
If i want to download a document from google docs directly to the X drive i get:
This is huge! I canāt wait till the final version is released!
At my company we have thin clients with an old version of the Nextcloud client installed and weāre given only a 20 GB quota of disk space, so syncronizing all my nextcloud folders is a no-go. If I syncronize only a few folders, in accordance with Murphyās law, I always need files from folders that are not synced.
Connecting to the server via WebDAV turned out to be a pain because youāre asked for credentials every single time you save a Microsoft Office file. You donāt have to enter them again, clicking āCancelā is enough to make the dialog box disappearā¦ till the next time you hit āSaveā. Even then, at some point along the way, I started getting errors that prevent me from saving changes back to the WebDAV-mounted folder.
So Iām very frustrated and the only thing that could save me is this Virtual Drive! So, please, go ahead and keep up the good work. Once itās stable, Iām going to ask the IT guys to install it! It would be awesome if it could work without admin rights since I donāt have them, but I understand that this is pretty low level stuffā¦
While I appreciate the work being done to make the Desktop client better. I 'd like to take advantage of Frankās quote from the first post:
The Desktop Client is a centerpiece of our synchronization strategy. Thus it is important we keep pushing the boundaries and deliver on what customers ask for.
and bring to your attention the following two desktop client bugs:
There are many other similar bugs in the github desktop repository. And even though a lot of technical users are reporting the problems, nextcloud developers donāt seem to be paying them much attention. The desktop clientās problems are so severe that if it is not monitored by a technical user there is a serious chance of data loss. For this reason I have stopped suggesting nextcloud to anyone who asks about storage and sync solutions. Please walk the talk of how important the desktop client is and at least make it as stable as the 2.3.3 version was
Interesting development here, this is certainly a good idea to throw. I feel an urge to participate into the discussion because in our company NC is a key piece of some applications and if not suitable the VirtualDrive evolution might break the whole process.
Same things, different perspective
I have not seen in the debate the remark that in some sense the functions provided by the client with the VD are not fundamentally different: both allow local access to NC-stored data, both allow a local copy when the system is offline, both allow the sync thread to run automatically.
In that sense I view it as an important but incremental evolution (which is good to me).
The fact that the local replica becomes a āsecond gradeā storage (from real volume to cache) is definitely a warning signal. And I have to learn what Doken can do. Letās study!
Our needs
Talking about customer scenario, let me try to make a simple but complete list of what we would need from any sync mechanism, first with a must-have list
local copy of data
access to the local copy for local process to write (local applications create new files and directories all the time)
possibility to leave a local cache copy even if it is not synced (and would be synced for instance one month later)
possibility to remotely delete local cache files (deleted on the server, deletion replicated on the client and clientās cache)
better mechanism to handle merge conflicts or places where contradictions appear. Currently I have github issue 998 open which needs extra workaround procedures.
security and access handling to determine which data a user might potentially see (through sharing I guess)
And a nice-to-have list
possibility to tune the cache retention period by directory
efficient synchronisation in terms of speed, local memory and cpu consumption (current client is ok but not great)
functions to avoid conflicts in the mapped drive letter
Of course this is food for thought into the debate, so feel free to pick . I will continue to participate into the interesting discussion.
Thanks a lot for the hard work, NC is a great tool and has made giant steps.
While it is possible to have both at the same time, we believe the Virtual Drive is fundamentally the superior approach so we will likely move to it completely.
Please donāt do that. The two approaches cover different use cases, both have their right of existence. While I appreciate your work and look forward to the virtual drive, I still need the sync feature. I have many synced folders with local subfolders excluded from synchronization. With the virtual drive, this would be impossible. I would have to re-organize my folder structure, resulting in data that belongs together scattered on different drives. This is not useful.
The virtual drive approach makes sense only for folders which are synced entirely.
I concur. I really donāt want to lose traditional sync features unless Virtual Drives cover ALL the same needs. And that would seem to include, partial syncs, and having copies on both clients and server (so it acts as a backup as well). And possibly more. Iām nervous about an imposed change that may cost functionality.
I have as I understand similar functionality in OneDrive and BoxSync. I can choose once and for all if I want to access files online or always store them locally (eg the traditional sync).
I see this as a very good step forward, and probably necessary. I do not see the reason for despair.
The negative reactions in this thread is based on assumptions that are probably false. It would be good if someone from the development team could clarify how it is planned.
Iād like to back the voices voting for keeping the sync feature. Most arguments why both methods are useful for different use cases are already named.
I have some non-tech users on my instance, this kind of users simply use os-predefined folders for different types of files. They are not used to the concept of drives nor do they consider of downloading files that may be needed when going offline.
This users would really struggle if we force them to adopt the virtual drive setup.
Reading through this thread Iām getting a very confusing picture of what the Virtual Drive is actually capable of. Is there a place where the technical differences are laid out in a side by side comparison?
Either way, something that is going to require a reorganization of data in order to work as the user expects is a no-go. Sorry, but weāve got 40 years of electronic flat file structure and millennia of physical flat file structure against us. Ever heard of chronologically filing something? The idea that whatever landed on top of the stack is the newest. Yeah, thatās the same thing as people dumping everything on the desktop. The people wonāt change, they will change which tool they use.
The best of both worlds would be placeholder files (with the current setup of a dedicated sync folder) and if the user wishes to keep the files locally, it can be set by the user, the rest could be downloaded ad hoc if needed. This project is about keeping control of your own data, right?
Either way, something that is going to require a reorganization of data in order to work as the user expects is a no-go.
Absolutely, this would be a major break and could lead to serious problems for quite a few people running Nextcloud. All the paths used by any previously installed software would need to be updated, and if the virtual drive somehow doesnāt load at boot, the user is screwed if the drive letter isnāt mounted properly causing 3rd-party software relying on a fixed path to not work properly. This doesnāt happen with the current setup because the files stay where they are regardless of whether the client runs or not.
I see the drawbacks of a folder based sync. Yet having used it for a while, I would not like to abandon it until the virtual drive has proven to be just as error tolerant, connection loss tolerant and dirty write tolerant (in terms of no data being lost writing a confict file) as the folder based sync.
I have use cases where Iād like not to have to sync all files, but for me, thatās often a folder decision.
Plus, for my customerās documentation itās the main feature that ALL files are available on both sides as a local copy. So if there will be virtual drive only, there should be an option to mimic the old āalways sync all files in these foldersā behaviour, including new files and directories.
this is really a huge step, which is highly welcome, even for my private / home hosted setup of nextcloud. Since the current ādrive solutionā for windows (net mounting a webdav) doensĀ“t work with my configuration (as far as i understand, this is because SNI is not supported by windows).
WIthout any testing, iĀ“d say, that virtual drives in my usecase are mostly useful to store an sync the private picture collection or even music collection. An option to make it all or some folders in the drive offline available is very much appreciated in this use case.
On the other side, i use todays desktop app to sync (and backup) specific folders which contain program settings, savegames, etc. between serveral workstations and notebooks. These are usually stored locally, some where deep in the %appdata% folders of each userĀ“s account under windows.
I understand, that it could be possible to symlink all these folders / files to the new virtual drive, but i am not so sure, if the corrosponding software would handle broken symlinks very well - let say, the virual drive is not available (the app has been closed or crashed). I have tried to symlink under windows, when some programs didnĀ“t allow to change storage folders (e.g. didnĀ“t support network mounted drives). From my experience, i would predict that most programs are not prepared for a case, where a system path (like c:%username%\appdata) isnĀ“t availiable anymore, resulting in crashes of the program and even possible data loss.
What do you think about these szenarios?
At the moment IĀ“d vote for both possibilies to remain, if i was asked. But that may change after some testingā¦iĀ“ll let you know.
Thanks for your great work sofar and by thus making me able to ātake back control over my dataā!
Please leave both options, I use the usual dektop clients to sync folder between hosts, with the virtual drive i would need to use another soft for this, because i need that the folders are in specific places,
. I will continue to participate into the interesting discussion.