mirror of
https://github.com/imartinez/privateGPT.git
synced 2025-04-28 03:32:18 +00:00
4523a30c8f
* docs: add missing configurations * docs: change HF embeddings by ollama * docs: add disclaimer about Gradio UI * docs: improve readability in concepts * docs: reorder `Fully Local Setups` * docs: improve setup instructions * docs: prevent have duplicate documentation and use table to show different options * docs: rename privateGpt to PrivateGPT * docs: update ui image * docs: remove useless header * docs: convert to alerts ingestion disclaimers * docs: add UI alternatives * docs: reference UI alternatives in disclaimers * docs: fix table * chore: update doc preview version * chore: add permissions * chore: remove useless line * docs: fixes ...
123 lines
3.7 KiB
Plaintext
123 lines
3.7 KiB
Plaintext
# List of working LLM
|
|
|
|
**Do you have any working combination of LLM and embeddings?**
|
|
|
|
Please open a PR to add it to the list, and come on our Discord to tell us about it!
|
|
|
|
## Prompt style
|
|
|
|
LLMs might have been trained with different prompt styles.
|
|
The prompt style is the way the prompt is written, and how the system message is injected in the prompt.
|
|
|
|
For example, `llama2` looks like this:
|
|
```text
|
|
<s>[INST] <<SYS>>
|
|
{{ system_prompt }}
|
|
<</SYS>>
|
|
|
|
{{ user_message }} [/INST]
|
|
```
|
|
|
|
While `default` (the `llama_index` default) looks like this:
|
|
```text
|
|
system: {{ system_prompt }}
|
|
user: {{ user_message }}
|
|
assistant: {{ assistant_message }}
|
|
```
|
|
|
|
The "`tag`" style looks like this:
|
|
|
|
```text
|
|
<|system|>: {{ system_prompt }}
|
|
<|user|>: {{ user_message }}
|
|
<|assistant|>: {{ assistant_message }}
|
|
```
|
|
|
|
The "`mistral`" style looks like this:
|
|
|
|
```text
|
|
<s>[INST] You are an AI assistant. [/INST]</s>[INST] Hello, how are you doing? [/INST]
|
|
```
|
|
|
|
The "`chatml`" style looks like this:
|
|
```text
|
|
<|im_start|>system
|
|
{{ system_prompt }}<|im_end|>
|
|
<|im_start|>user"
|
|
{{ user_message }}<|im_end|>
|
|
<|im_start|>assistant
|
|
{{ assistant_message }}
|
|
```
|
|
|
|
Some LLMs will not understand these prompt styles, and will not work (returning nothing).
|
|
You can try to change the prompt style to `default` (or `tag`) in the settings, and it will
|
|
change the way the messages are formatted to be passed to the LLM.
|
|
|
|
## Example of configuration
|
|
|
|
You might want to change the prompt depending on the language and model you are using.
|
|
|
|
### English, with instructions
|
|
|
|
`settings-en.yaml`:
|
|
```yml
|
|
local:
|
|
llm_hf_repo_id: TheBloke/Mistral-7B-Instruct-v0.1-GGUF
|
|
llm_hf_model_file: mistral-7b-instruct-v0.1.Q4_K_M.gguf
|
|
embedding_hf_model_name: BAAI/bge-small-en-v1.5
|
|
prompt_style: "llama2"
|
|
```
|
|
|
|
### French, with instructions
|
|
|
|
`settings-fr.yaml`:
|
|
```yml
|
|
local:
|
|
llm_hf_repo_id: TheBloke/Vigogne-2-7B-Instruct-GGUF
|
|
llm_hf_model_file: vigogne-2-7b-instruct.Q4_K_M.gguf
|
|
embedding_hf_model_name: dangvantuan/sentence-camembert-base
|
|
prompt_style: "default"
|
|
# prompt_style: "tag" # also works
|
|
# The default system prompt is injected only when the `prompt_style` != default, and there are no system message in the discussion
|
|
# default_system_prompt: Vous êtes un assistant IA qui répond à la question posée à la fin en utilisant le contexte suivant. Si vous ne connaissez pas la réponse, dites simplement que vous ne savez pas, n'essayez pas d'inventer une réponse. Veuillez répondre exclusivement en français.
|
|
```
|
|
|
|
You might want to change the prompt as the one above might not directly answer your question.
|
|
You can read online about how to write a good prompt, but in a nutshell, make it (extremely) directive.
|
|
|
|
You can try and troubleshot your prompt by writing multiline requests in the UI, while
|
|
writing your interaction with the model, for example:
|
|
|
|
```text
|
|
Tu es un programmeur senior qui programme en python et utilise le framework fastapi. Ecrit moi un serveur qui retourne "hello world".
|
|
```
|
|
|
|
Another example:
|
|
```text
|
|
Context: None
|
|
Situation: tu es au milieu d'un champ.
|
|
Tache: va a la rivière, en bas du champ.
|
|
Décrit comment aller a la rivière.
|
|
```
|
|
|
|
### Optimised Models
|
|
GodziLLa2-70B LLM (English, rank 2 on HuggingFace OpenLLM Leaderboard), bge large Embedding Model (rank 1 on HuggingFace MTEB Leaderboard)
|
|
`settings-optimised.yaml`:
|
|
```yml
|
|
local:
|
|
llm_hf_repo_id: TheBloke/GodziLLa2-70B-GGUF
|
|
llm_hf_model_file: godzilla2-70b.Q4_K_M.gguf
|
|
embedding_hf_model_name: BAAI/bge-large-en
|
|
prompt_style: "llama2"
|
|
```
|
|
### German speaking model
|
|
`settings-de.yaml`:
|
|
```yml
|
|
local:
|
|
llm_hf_repo_id: TheBloke/em_german_leo_mistral-GGUF
|
|
llm_hf_model_file: em_german_leo_mistral.Q4_K_M.gguf
|
|
embedding_hf_model_name: T-Systems-onsite/german-roberta-sentence-transformer-v2
|
|
#llama, default or tag
|
|
prompt_style: "default"
|
|
```
|