mirror of
https://github.com/imartinez/privateGPT.git
synced 2025-06-22 05:30:34 +00:00
Adding docs about embeddings settings + adding the embedding.mode: local in mock profile (#1316)
This commit is contained in:
parent
bafdd3baf1
commit
baf29f06fa
@ -33,16 +33,20 @@ Are you running out of memory when ingesting files?
|
|||||||
To do not run out of memory, you should ingest your documents without the LLM loaded in your (video) memory.
|
To do not run out of memory, you should ingest your documents without the LLM loaded in your (video) memory.
|
||||||
To do so, you should change your configuration to set `llm.mode: mock`.
|
To do so, you should change your configuration to set `llm.mode: mock`.
|
||||||
|
|
||||||
In other words, you should update your `settings.yaml` (or your custom configuration file) to set the
|
You can also use the existing `PGPT_PROFILES=mock` that will set the following configuration for you:
|
||||||
following **before** ingesting your documents:
|
|
||||||
```yaml
|
```yaml
|
||||||
llm:
|
llm:
|
||||||
mode: mock
|
mode: mock
|
||||||
|
embedding:
|
||||||
|
mode: local
|
||||||
```
|
```
|
||||||
|
|
||||||
|
This configuration allows you to use hardware acceleration for creating embeddings while avoiding loading the full LLM into (video) memory.
|
||||||
|
|
||||||
Once your documents are ingested, you can set the `llm.mode` value back to `local` (or your previous custom value).
|
Once your documents are ingested, you can set the `llm.mode` value back to `local` (or your previous custom value).
|
||||||
|
|
||||||
You can also use the existing `PGPT_PROFILES=mock` that will set the `llm.mode` to `mock` for you.
|
|
||||||
|
|
||||||
## Supported file formats
|
## Supported file formats
|
||||||
|
|
||||||
|
@ -1,5 +1,8 @@
|
|||||||
server:
|
server:
|
||||||
env_name: ${APP_ENV:mock}
|
env_name: ${APP_ENV:mock}
|
||||||
|
|
||||||
|
# This configuration allows you to use GPU for creating embeddings while avoiding loading LLM into vRAM
|
||||||
llm:
|
llm:
|
||||||
mode: mock
|
mode: mock
|
||||||
|
embedding:
|
||||||
|
mode: local
|
||||||
|
Loading…
Reference in New Issue
Block a user