github/privateGPT
1
0
Fork 1
mirror of https://github.com/imartinez/privateGPT.git synced 2025-06-27 15:58:56 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update description.md (#1107)

Browse Source 
Added a section on how to customize low level args, proposing people to stick to suggested models.
This commit is contained in:
César García 2023-11-11 09:23:46 +01:00 committed by GitHub
parent 8487440a6f
commit dbd99e7b4b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/description.md
View File

@ -174,6 +174,10 @@ is used.
> It's highly encouraged that you fully read llama-cpp and llama-cpp-python documentation relevant to your platform.
> Running into installation issues is very likely, and you'll need to troubleshoot them yourself.
#### Customizing low level parameters
Currently not all the parameters of llama-cpp and llama-cpp-python are available at PrivateGPT's `settings.yaml` file. In case you need to customize parameters such as the number of layers loaded into the GPU, you might change these at the `llm_component.py` file under the `private_gpt/components/llm/llm_component.py`. If you are getting an out of memory error, you might also try a smaller model or stick to the proposed recommended models, instead of custom tuning the parameters.
#### OSX GPU support
You will need to build [llama.cpp](https://github.com/ggerganov/llama.cpp) with