I’ve successfully set up the Nextcloud AI Assistant with OpenAI API, and everything is are working correctly. However, I’m encountering challenges with the context chat feature.
After reviewing the documentation and source code for both context_chat and context_chat_backend components, I’m still unclear about options for the backend.
I have limited experience with LangChain, vector databases, and text embeddings which is making it difficult for me to parse the code.
However, from my research, it appears this requires a server with an NVIDIA GPU and minimum 6GB VRAM for local embedding processing, with no option to use an external service instead.
I understand why beefy NVidia hardware is needed to run it locally, however, if this is the only option it seems inconsistent with other Nextcloud AI applications, which all support using a range of external services.
Question: Is it possible to configure the context chat indexing to use OpenAI’s text-embedding-3-small model using using local GPU hardware? If so, could someone point me to documentation or configuration examples?