Pictures Tagging - best ways to make use of AI-based description and tags in NextCloud?

đź’» Development
, ,
1

I’ve been working on a personal project to make my photo archive (~5 million images) fully searchable. After a bit of hacking, I now have a setup I’m really happy with - I’ve wired up my local GPU and am using mistral-small-3.1 via Ollama to generate rich descriptions and keyword-style tags for each photo.

So far, the results are great - I can describe any image and extract meaningful tags like:

"car", "a girl in a red dress", "green grass", "invoice", "kids at the beach"

And it’s freaking terrifying how accurate this thing is (I was blown away when I asked it to find a picture with a “VIN number” and it just listed it…).

The challenge?
I don’t know how to make use of these tags inside Nextcloud in a way that integrates with its native search, filtering, or media browsing experience.

What I’ve got working:

  • Full image tagging pipeline using GPU + LLM
  • Results logged to .txt and .json files per image
  • Optional EXIF embedding and/or .tags.txt creation
  • Images accessible via WebDAV and shown inside Photos/Memories

What I haven’t figured out yet:

  • How to associate these tags with the actual file in Nextcloud (via API or WebDAV)
  • How to make these tags searchable via the default Nextcloud interface
  • Whether I can inject metadata (e.g., system tags, file comments, etc.) that the platform respects

What I’ve tried:

  • OCS API endpoints for system tags (/ocs/v2.php/cloud/systemtags) — works in theory, but file ID-based tagging via API seems undocumented or broken
  • WebDAV PROPFIND and PROPPATCH for setting metadata - no success with tags
  • Looking into occ files:metadata:* commands - seems promising but not ideal for ongoing automation
  • Memories app / EXIF-based workflows - better for filtering but not for actual “tags” per se

My goal here is to be able to search in Nextcloud for “car” or “receipt” and have relevant photos show up, even if the tag came from an LLM pipeline and not from me manually assigning it.

  • Has anyone had success assigning searchable tags to files in Nextcloud programmatically?
  • Are there any best practices for embedding metadata that the core search picks up?
  • Would you recommend writing a small app to sync tags and expose a custom search experience?
  • Is it worth investing in a custom workflow engine rule or file metadata scanner?

Any input - workflows, plugins, hacks, future roadmap hints - is super appreciated!

If you’ve done something similar or know of a plugin that could help, I’d love to hear about it.

Cheers,

2

Hi,

I don’t think tags would be such a good idea. It seems that you will create a lot of tags. And the purpose of tags (in my opinion) is to help to categorize, not a keyword that should be searchable itself.

What i suggest could be to use the filesmetada (introduced with NC 28). So you could create a free text metadata “image_description”, that could be managed and edited manually or generated with AI. Then, you have to created in your app an entrypoint to the unified search that let you match these metadata content with the search. But if I consider your 5 millions entries, you should have a look on a powerful search engine like Solr or ElasticSearch that is handled by Fulltext Search app (if i remember well).

All this require an independant Nextcloud app you’ll have to create, in order to manage these capacities.

Yours,

1 Like