Configuration of LocalAI


Hello, I am running a Managed Nextcloud (27.1.5) and try to switch from OpenAI usage (with a working API key) to LocalAI usage.

With my configured API key, I can send a prompt with the smartpicker e.g. from a textfile and get a proper response from ChatGPT 3.5, so smart picker in general is doing fine.

To use LocalAI, I have installed LMStudio on my local PC and run a model in server mode. (localhost:4891)

I configured the AI account in Nextcloud as follows:

But there is no model available at the smart picker.
and I do not get a response from LM Studio besides the following message.

There is also no additional entry in the log of LM Studio.

Has anybody an idea, what is missing in my configuration or why LocalAI does not work for me?

1 Like

I have the same issue! Haven’t figured out how to properly integrate localai yet. I have a few more specific questions to ask, though.

The following API endpoint all work locally:

LM Studio:

# Chat with an intelligent assistant in your terminal
from openai import OpenAI

# Point to the local server
client = OpenAI(base_url="http://localhost:1234/v1", api_key="not-needed")

history = [
    {"role": "system", "content": "You are an intelligent assistant. You always provide well-reasoned answers that are both correct and helpful."},
    {"role": "user", "content": "Hello, introduce yourself to someone opening this program for the first time. Be concise."},

while True:
    completion =
        model="local-model", # this field is currently unused

    new_message = {"role": "assistant", "content": ""}
    for chunk in completion:
        if chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="", flush=True)
            new_message["content"] += chunk.choices[0].delta.content

    # Uncomment to see chat history
    # import json
    # gray_color = "\033[90m"
    # reset_color = "\033[0m"
    # print(f"{gray_color}\n{'-'*20} History dump {'-'*20}\n")
    # print(json.dumps(history, indent=2))
    # print(f"\n{'-'*55}\n{reset_color}")

    history.append({"role": "user", "content": input("> ")})


curl http://localhost:11434/api/chat -d '{
  "model": "mistral",
  "messages": [
    { "role": "user", "content": "why is the sky blue?" }


curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "llava",
    "messages": [{"role": "user", "content": [{"type":"text", "text": "What is in the image?"}, {"type": "image_url", "image_url": {"url": "" }}], "temperature": 0.9}]}'

But none of these endpoints, when set as LocalAI URL (leave empty to use, work to enable the AI functionality on Nextcloud:

  1. http://localhost:1234/v1
  2. http://localhost:11434/api/chat
  3. http://localhost:8080/v1/chat/completions

Can someone please enlighten us how to set this up properly?

Must localai be installed on the same machine as the server, or can it be run remotely on the client?

1 Like

I can add, that similar tests in my system, namely using a complete URL including the path behind server name and port were unsuccesful as well.

Your nextcloud needs to be able to connect to the path you provide. If you point to “localhost” your nextcloud is trying to connect to the server nextcloud itself is running on. If you’re running everything locally it could work. If not provide a reachable adress (like IP or URL) of the machine you’re running the LLM at.

Hello Steffen!
Thank you for your answer. Let me repeat with my own words to see, whether I understand correctly and ask you for confirmation.
Assuming that Nextcloud is running on a server with a publically available URL (e.g. and LocalAI is running on a own and personal PC (e.g. a typical desktop PC or Notebook), it is necessary to create a publically available URL for the personal PC (e.g. with a DynDNS service) to enable the Nextcloud to reach the instance where LocalAI is running. A such created DynDNS could be the URL which had to be configured in Nextcloud AI account, correct?

This would be one way to achieve this. But beware that your local machine has to be reachable on the specific port. So make sure no firewall is blocking the connection. You should also take some thoughts on security (like connected encryptions/https).

Steffen, thank you again!
The most important thing is, that I understood it is necessary to have a URL connection between Nextcloud and the LocalAI instance as long as both servers are not running on the same machine. This is obviously the reason, my setup is not working until now. And it seems that @sati-bodhi faces the same problem.
Be assured, I have possible problems like closed firewall ports or security issues like encryption or https on my radar, but at the moment it is more a try status where I try to create a test system, which is generally working.
For a more productive environment, I will address security issues more intensly.

I will work on my tests in the coming days and hopefully when it works, I will mark the thread as “solved”.


___ Update Feb, 22nd
Just received an answer to a prompt, send to my LocaiAI instance via my Managed Nextcloud smart picker… :slight_smile:.
So as promissed my confirmation, that @swindhab 's suggestions and solution leads to success. In the coming days, I will address another topic in the LocalAI configuration which are security related issues in a Managed Nextcloud environment, which is not as flexible as a self hosted system.
Bye for this evening!

I was able to gain some headway with ollama.

However, the preset Nextcloud Local URL endpoint v1/chat/completions (via the Chat Completions option) and v1/completions (via the Completions option) do not fit the ollama endpoints of:

http://localhost:11434/api/generate for text generation and
http://localhost:11434/api/chat for chat completions.

Users should be given the freedom to set their own endpoints for maximum adaptability.

Curiously, I get a:

Completion request error: API request error: model 'gpt-3.5-turbo' not found, try pulling it first

when set to chat completions. Nextcloud seems to think I am calling the OpenAI API.

Seems like the API presented isn’t a drop-in replacemet for the OpenAI-API. For me LocalAI ( is working great.

This topic was automatically closed 8 days after the last reply. New replies are no longer allowed.