Localai machine translation prompt

🔣 Translations
,
1

Is there a way to set the localai machine translation prompt? The default prompt doesn’t work:

[2024-02-22 11:44:20.620] [INFO] [LM STUDIO SERVER] Processing queued request...
[2024-02-22 11:44:20.620] [INFO] Received POST request to /v1/completions with body: {
  "model": "/home/sati/.cache/lm-studio/models/hfl/chinese-alpaca-2-7b-rlhf-gguf/ggml-model-q6_k-im.gguf",
  "prompt": "Translate from English (US) to 正體中文（臺灣）: Continue to learn and evolve",
  "max_tokens": 1000,
  "n": 1,
  "user": "admin"
}
[2024-02-22 11:44:20.620] [INFO] Provided inference configuration: {
  "n_threads": 4,
  "n_predict": 1000,
  "top_k": 40,
  "min_p": 0.05,
  "top_p": 0.95,
  "temp": 0.8,
  "repeat_penalty": 1.1,
  "input_prefix": "### Instruction:\n",
  "input_suffix": "\n### Response:\n",
  "antiprompt": [
    "### Instruction:"
  ],
  "pre_prompt": "",
  "pre_prompt_suffix": "\n",
  "pre_prompt_prefix": "",
  "seed": -1,
  "tfs_z": 1,
  "typical_p": 1,
  "repeat_last_n": 64,
  "frequency_penalty": 0,
  "presence_penalty": 0,
  "n_keep": 0,
  "logit_bias": {},
  "mirostat": 0,
  "mirostat_tau": 5,
  "mirostat_eta": 0.1,
  "memory_f16": true,
  "multiline_input": false,
  "penalize_nl": true
}

It just generates a write-up in the original language:

[2024-02-22 11:44:46.443] [INFO] Generated prediction: {
  "id": "cmpl-bjz3c12697m21cjui88ufm",
  "object": "text_completion",
  "created": 1708573460,
  "model": "/home/sati/.cache/lm-studio/models/hfl/chinese-alpaca-2-7b-rlhf-gguf/ggml-model-q6_k-im.gguf",
  "choices": [
    {
      "index": 0,
      "text": "It's very important to always be willing to learn and grow. I believe that it's never too late to start, so don't be afraid to take risks and try something new. Just keep on learning and you will find yourself developing into a better person over time. And remember, every step forward is a success!",
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 793,
    "completion_tokens": 68,
    "total_tokens": 861
  }
}

Translations work (almost) perfectly when we make use of system prompts:

[2024-02-22 11:45:47.319] [INFO] [LM STUDIO SERVER] Processing queued request...
[2024-02-22 11:45:47.319] [INFO] Received POST request to /v1/chat/completions with body: {
  "messages": [
    {
      "role": "system",
      "content": "翻譯成中文"
    },
    {
      "role": "user",
      "content": "I am solid, I am free."
    }
  ],
  "temperature": 0.7,
  "max_tokens": -1,
  "stream": false
}
[2024-02-22 11:45:47.320] [INFO] [LM STUDIO SERVER] Context Overflow Policy is: Rolling Window
[2024-02-22 11:45:47.320] [INFO] [LM STUDIO SERVER] Last message: { role: 'user', content: 'I am solid, I am free.' } (total messages = 2)
[2024-02-22 11:45:52.808] [INFO] [LM STUDIO SERVER] Accumulating tokens ... (stream = false)
[2024-02-22 11:45:52.808] [INFO] Accumulated 1 tokens: 我
[2024-02-22 11:45:53.065] [INFO] Accumulated 2 tokens: 我坚固
[2024-02-22 11:45:53.307] [INFO] Accumulated 3 tokens: 我坚固，
[2024-02-22 11:45:53.551] [INFO] Accumulated 4 tokens: 我坚固，我是
[2024-02-22 11:45:53.796] [INFO] Accumulated 5 tokens: 我坚固，我是自由
[2024-02-22 11:45:54.040] [INFO] Accumulated 6 tokens: 我坚固，我是自由。
[2024-02-22 11:45:54.293] [INFO] [LM STUDIO SERVER] Generated prediction: {
  "id": "chatcmpl-duxs5tbzqs651fk0eo9dt",
  "object": "chat.completion",
  "created": 1708573547,
  "model": "/home/sati/.cache/lm-studio/models/hfl/chinese-alpaca-2-7b-rlhf-gguf/ggml-model-q6_k-im.gguf",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "我坚固，我是自由。"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 799,
    "completion_tokens": 6,
    "total_tokens": 805
  }
}

I believe these can be set through the localai model yaml file. This would make the settings specific to a certain model. However, we don’t only use our LLMs for translation.

Ideally, we should be given to freedom to set different prompts for different tasks. Can this be done with the NC AI Assistant?

Currently, Machine Translation AI settings only allows us to rank the different avenues to perform our translations, one of which is local AI.