When I rename a file in an external mount using the Web interface, the comments and metadata of the file are preserved.
However, if I rename the same file using my file manager in that external mount, upon reloading the Nextcloud Web interface, the file is interpreted (ingested by Nextcloud) as an entirely new file (despite having the exact same contents, inode number, mod / birth dates / size). As a consequence, all metadata and comments associated to the file disappear.
We have a heterogeneous system here and we can’t enforce access of files solely through Nextcloud. NC is fine for casual file access but heavy file management is simply too slow to be done over NC.
Is there any extension or plugin or anything of the sort that would attempt to reconcile changes in a directory and update the files tables on the Nextcloud database when said data is refreshed as users browse or the periodic scan takes place? Alternatively, is there a plugin that would use something like extended attributes to brand files with unique identifiers, and detect when they’ve been moved or renamed using that info?
Thanks in advance for any pointers in useful directions. I am using NC 30.x as shipped by a Fedora Linux system with external file mounts over SAMBA. There are no noteworthy log entries to mention.
It’s worth noting that I’ve done some more hands-on research and I’ve determined the following:
File renames done on the system running SAMBA aren’t detected unless they are file renames on the root directory of the mount on Nextcloud. I can tell this is so, because I ran occ files_external:notify <mount number> and I could see rename events for files in said root directory, but not for files in any subdirectory.
File renames done via any SAMBA client connected to that system are detected by occ files_external:notify always. It accurately detects the file was renamed. This is great!
Unfortunately, even though the file was successfully renamed, and the notify system detected the rename, when I reloaded the Nextcloud Web interface, I could confirm that all Nextcloud comments and file activity history were lost anyway. This remained so even after confirming that the periodic cron job to process changed files ran.
It looks like Nextcloud sees me (the user) refresh the folder, scans the folder to update its contents right on the spot, and whatever notification / file modify information coming from the notify command is discarded. That looks like a bug to me, to be quite honest.
Why, if Nextcloud is clearly getting informed about the file rename, is Nextcloud still losing file metadata? How can I dig deeper into what’s going on?
So processing of renames doesn’t actually migrate the file metadata from the old file to the new file. It just causes the target folder to be scanned in addition to the source folder.
Not what I would have expected!
Furthermore, while I have verified that the notify thing is working, the scan does not show any new or updated or removed files, which is weird.
Feb 11 11:34:37 cloud nice[2087477]: +---------+-------+-----+---------+---------+--------+--------------+
Feb 11 11:34:37 cloud nice[2087477]: | Folders | Files | New | Updated | Removed | Errors | Elapsed time |
Feb 11 11:34:37 cloud nice[2087477]: +---------+-------+-----+---------+---------+--------+--------------+
Feb 11 11:34:37 cloud nice[2087477]: | 1 | 671 | 0 | 0 | 0 | 0 | 00:00:02 |
Feb 11 11:34:37 cloud nice[2087477]: +---------+-------+-----+---------+---------+--------+--------------+
So any renamed files are basically ingested as totally new. There is no detection of file rename (or, more clearly, if someone ever coded such a thing, it’s just not working).
Guh.
Whom can I pay to develop this feature? It’s key for us to keep file metadata and history upon rename (even if on a best-effort basis).
If I understand your issue in the right way, you are changing the file outside of nextcloud. Thus, it is expected that nextcloud cannot handle such situation.
Endpoints you need to use would be nextcloud client or webdav. Everything else isn’t really supported.
Why should I expect Nextcloud to have this deficiency?
I mean, maybe I’m too new to the software, but other software (such as Amarok or Jellyfin) have no problems identifying moved files and updating their metadata. Clearly it’s a solvable problem.
Anyway, now that we have established this is a deficiency, I want to know who is willing to receive money from me to correct that deficiency.
Essentially, what I want is any entry point that scans files (whether it be the routine that updates a stale folder when a user browses to it, or the occ files_external:scan command), to detect renames and moves. File identification could be accomplished in a number of ways, such as file fingerprinting (usually suffices to hash the initial and final X bytes of the file), inode matching, or even extended attributes with unique IDs per file. To speed up the process, renames could be explicitly tracked (it’s fairly clear that the occ files_external:notify CLI tool detects renames quite OK). Folder renames and moves should be detected as well.