github/privateGPT
1
0
Fork 1
mirror of https://github.com/imartinez/privateGPT.git synced 2025-04-28 03:32:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

Adding docs about embeddings settings + adding the embedding.mode: local in mock profile (#1316)

Browse Source
This commit is contained in:
Gianni Acquisto 2023-11-26 17:32:11 +01:00 committed by GitHub
parent bafdd3baf1
commit baf29f06fa
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 10 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
fern/docs/pages/manual/ingestion.mdx
View File

@ -33,16 +33,20 @@ Are you running out of memory when ingesting files?
To do not run out of memory, you should ingest your documents without the LLM loaded in your (video) memory.
To do so, you should change your configuration to set `llm.mode: mock`.
In other words, you should update your `settings.yaml` (or your custom configuration file) to set the
following **before** ingesting your documents:
You can also use the existing `PGPT_PROFILES=mock` that will set the following configuration for you:
```yaml
llm:
mode: mock
embedding:
mode: local
```
This configuration allows you to use hardware acceleration for creating embeddings while avoiding loading the full LLM into (video) memory.
Once your documents are ingested, you can set the `llm.mode` value back to `local` (or your previous custom value).
You can also use the existing `PGPT_PROFILES=mock` that will set the `llm.mode` to `mock` for you.
## Supported file formats

3
settings-mock.yaml
View File

@ -1,5 +1,8 @@
server:
env_name: ${APP_ENV:mock}
# This configuration allows you to use GPU for creating embeddings while avoiding loading LLM into vRAM
llm:
mode: mock
embedding:
mode: local