github/privateGPT
1
0
Fork 1
mirror of https://github.com/imartinez/privateGPT.git synced 2025-09-05 17:21:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: add new cuda profile

Browse Source
This commit is contained in:
Javier Martinez
2024-08-05 17:48:14 +02:00
parent 1e82ac9d6f
commit 23704d23ad
2 changed files with 35 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
fern/docs/pages/quickstart/quickstart.mdx
View File

@@ -82,6 +82,21 @@ HF_TOKEN=<your_hf_token> docker-compose --profile llamacpp-cpu up
```
Replace `<your_hf_token>` with your actual Hugging Face token.
#### 2. LlamaCPP CUDA
**Description:**
This profile runs the Private-GPT services locally using `llama-cpp` and Hugging Face models.
**Requirements:**
A **Hugging Face Token (HF_TOKEN)** is required for accessing Hugging Face models. Obtain your token following [this guide](/installation/getting-started/troubleshooting#downloading-gated-and-private-models).
**Run:**
Start the services with your Hugging Face token using pre-built images:
```sh
HF_TOKEN=<your_hf_token> docker-compose --profile llamacpp-cuda up
```
Replace `<your_hf_token>` with your actual Hugging Face token.
## Building Locally
If you prefer to build Docker images locally, which is useful when making changes to the codebase or the Dockerfiles, follow these steps: