I have now gained 4 days of experience to be able to use Context Chat in Nextcloud. I find the feature very useful, not having to search for everything manually. The installation cost me a lot of nerves. But now it works.
Hardware:
Dell Optiplex 7060, Intel i5 8400, 6 cores up to 4 Ghz. 16 GB RAM with nvme SSD’s.
Software:
Ubuntu 24.04, PHP 8.3, Nextcloud 31 and Apache2.
I know the hardware is not necessarily suitable, but trial and error makes perfect.
First insight: LocalAI runs more resource-efficiently than LargeLanguageModel from the NC ExtApps/AppAPI. I have tried both. With the llm2 from the NC apps, a request with context chat takes 10-15 minutes on my system. LocalAI takes 4-6 minutes for a request, depending on the model I use. I find that reasonable.
(In chat with the AI, it takes a few moments for the answer to arrive, not 5-6 minutes).
I only wanted context chat and voice transcription. I have no use for any other functions.
I tried a few models from the LocalAI Gallery and had good experiences with the following.
- Whisper-1 for transcription
- mistral-nemo-instruct-2407, meta-llama-3.1-8b-instruct, llama-3-sauerkrautlm-8b-instruct for Text2Text processing (although they sometimes answer in English as I ask in german)
For Context Chat to work, “Task: Free text to text prompt” must be activated. Also “Chat” and “Chat with Tools”. Otherwise I got no answers.
I installed the local-ai container with the “docker run …” command. I selected latest-cpu as the version. The installation with the script via https://localai.io/install.sh did not work.
I released 5 of the cores for the docker container. I first tried it with 3 cores, but that only resulted in aborted requests. Then it worked with all cores, with 5 too. (docker update --cpus 5 local-ai)
This is how I did it.
- Install LocalAI on port 8080
- Nextcloud AI Assistant was already installed.
- Install “OpenAi LocalAi integration”
- Now set up LocalAI under Artificial Intelligence in the settings. URL as localhost as previously defined, NO API key if you have not assigned one in the docker run command of LocalAI. Then define the models you want to use.
Then I first tried if the chat works, as the connection NC to LocalAi. Yes!
Then:
-
Install NC Deploy Deamon (GitHub - nextcloud/docker-socket-proxy: Nextcloud AppAPI Docker Socket Proxy)
-
Then set up the Deploy Deamon under AppAPI in the settings and test the deployment. If everything works - continue.
-
Install context_chat_backend via the command line with occ. There were always problems with the apps. Also via occ, but that was more transparent and in case of errors I could undo everything with “occ app_api:app:unregister --rm-data --force context_chat_backend” and start again.
-
Install context_chat.
Context_Chat_Backend is automatically registered on the Deploy Deamon and then appears in the options under “Artificial intelligence”.
The context chat backend must then be trained or scan the data from the NC (occ context_chat:scan – {USER}).
However, it does not capture all the data at once, but works through it piece by piece. I will probably have to wait a few more days (it has processed 2300 of 43000 files so far). Manual scan or not. The README for the apps suggests some settings that I have used, including the nextcloud-ai-worker@.service.
When testing with “docker stats”, I was able to see that it works. At first the CPU went up for context_chat_backend. Then at “local-ai” to 500%. All 5 cores full throttle. Then 4-5 minutes until the response came.
The hit rate is now around 20%. As a question to the files and meaningful answer.
I have not yet managed to query calendars, tasks and mails with Context_Agent.
I hope my report helps one or the other when using the Nextcloud Assistant with the Context Chat.
) is supposed to make email summaries as one of its tasks.