I now āsort of reverse engineeredā the āoc_filecacheā table and have built a āhookā into that table of NextCloud and use it for itās cloud-drive capabilities and file management structure.
This is all working quite nicely, but when a file is moved the only way to process that change seems to do a complete database scan and compare the NextCloud table āoc_filecacheā to my table. This takes quit a long timeā¦
The field mtime holds the file date, so the storage_mtime appears to be an internal date.
Detecting a new/newer file is now done using the field storage_mtime
But I canāt see a āfile moveā nor a āfile renameā in the table, since the storage_mtime does not update/change.
In my opinion the field āstorage_mtimeā should also be updated to ānowā upon a move or a rename.
I would like to see this update done in a future version of Nextcloud, but could it be possible to build a āevent dispatchā to do this as a āquick fixā?
I canāt really help regarding the dispatcher, but I really wonder if hooking into the database (especially filescacheā¦) makes sense. Which underlying issue are you trying to solve?
I use Nextcloud for the NextCloud desktop client so I can use the network drive functionality, file management structure and the technology to use the S3 data storage technique.
With that I have the perfect bases to manage the vast amount of media for my History platform I call GeoArchive. an example:
Hooking into the āoc_filecacheā gives me access to the files. Hooking into the S3-account gives me access to the source files.
Combining that I can generate the images for my Archive project.
New additions to the Nextcloud account can easily be found by searching for newer file IDās then the last one I synced. File modifications can easily be found via the mentioned storage_mtime field.
But I now need to periodically sync the complete table. This is quite an intensive process. Especially when an archive has some 100.000 media elementsā¦
So I would really like to improve the incremental synchronization, especially the file moves and the renamesā¦
PS: a periodic full sync remains a smart thing to doā¦ but the whole performance of the Archive platform would greatly be improved if I could ācatchā the ārenames & movesāā¦
I think you do not want to create a custom event dispatcher but an event listener. There are a set of events that the core will trigger. E.g. OCP\Files\Events\Node\NodeRenamedEvent will be fired upon a rename of a file (move).
In an app, you could create a listener to react somehow (and this is how you handle the updating in your part). By the means of the event object, you can access old and new file name etc. Similarly, you can go with the other file system modifications.
First of all, thank you for taking the time to respond.
I have no experience in building a āNextCloud listnerā what so everā¦ and quite hesitant to start playing around in my live setupā¦
Can you tell my if this assumption is correct?
Is there no harm what so ever to set storage_mtime to ānowā upon a file move or rename?
And if that is the case, why not set it to ānowā natively within NextCloud? (*)
And to fill the gap in the meantime; if you could tell me the code for that listner (and where to put it) I would be very gratefull!
*) doing so would open the door for other applications to build extensions upon NextCloud! (and keep it in sync effectively)
Upon request I would be willing to share my queries (they are slightly complex)ā¦ with it the handy-man/woman can figure it out with the syncing!
Well, I posted the link to the documentation already. This assumes, you have a NC app crafted that does the interaction with NC for you.
I would advice against the direct access to the oc_filecache database table. It is prone to change at any major or minor release and might break without prior notice. Infact, each app should only rely on its own tables unless crafted very (!) carefully.
As I just said: No, I cannot say this for sure. It might be an assumption but I have neither thought it through nor is it guaranteed there is no NC app installed that has a similar assumtion and does weired things. I simply do not know.
You brought up the idea to use events. In fact, I think this might work quite smoothly. You would have to define the interaction between NC (and the app you need to create) and the existing software of yours.
You could make NC trigger some sort of webhook for example one a certain event was fired. Then, your (external) code could work on this information.
You could also do a polling approach (aka store the changes that need to be caried out and let your external app query on regular intervals for changes).
