github/privateGPT
1
0
Fork 1
mirror of https://github.com/imartinez/privateGPT.git synced 2025-09-11 20:20:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

Stop using the fake tensorrt-llm package. Update documentation.

Browse Source
This commit is contained in:
imartinez
2024-03-08 00:52:01 +01:00
parent 937c52354b
commit a93db2850c
3 changed files with 5 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
fern/docs/pages/installation/installation.mdx
View File

@@ -137,6 +137,8 @@ Follow these steps to set up a local TensorRT-powered PrivateGPT:
- Nvidia Cuda 12.2 or higher is currently required to run TensorRT-LLM.
- Install tensorrt_llm via pip with pip install --no-cache-dir --extra-index-url https://pypi.nvidia.com tensorrt-llm as explained [here](https://pypi.org/project/tensorrt-llm/)
- For this example we will use Llama2. The Llama2 model files need to be created via scripts following the instructions [here](https://github.com/NVIDIA/trt-llm-rag-windows/blob/release/1.0/README.md#building-trt-engine).
The following files will be created from following the steps in the link: