github/privateGPT
1
0
Fork 1
mirror of https://github.com/imartinez/privateGPT.git synced 2025-08-09 19:37:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Added embedded model option for fireworks \n Added documentation for fireworks

Browse Source
This commit is contained in:
Somashekar B R 2024-09-24 23:27:40 +05:30
parent 519c48b70b
commit 9c3590e555
4 changed files with 154 additions and 79 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

119
fern/docs/pages/installation/installation.mdx
View File

@ -3,45 +3,63 @@ It is important that you review the [Main Concepts](../concepts) section to unde
## Base requirements to run PrivateGPT ## Base requirements to run PrivateGPT
### 1. Clone the PrivateGPT Repository ### 1. Clone the PrivateGPT Repository
Clone the repository and navigate to it: Clone the repository and navigate to it:
```bash ```bash
git clone https://github.com/zylon-ai/private-gpt git clone https://github.com/zylon-ai/private-gpt
cd private-gpt cd private-gpt
``` ```
### 2. Install Python 3.11 ### 2. Install Python 3.11
If you do not have Python 3.11 installed, install it using a Python version manager like `pyenv`. Earlier Python versions are not supported. If you do not have Python 3.11 installed, install it using a Python version manager like `pyenv`. Earlier Python versions are not supported.
#### macOS/Linux #### macOS/Linux
Install and set Python 3.11 using [pyenv](https://github.com/pyenv/pyenv): Install and set Python 3.11 using [pyenv](https://github.com/pyenv/pyenv):
```bash ```bash
pyenv install 3.11 pyenv install 3.11
pyenv local 3.11 pyenv local 3.11
``` ```
#### Windows #### Windows
Install and set Python 3.11 using [pyenv-win](https://github.com/pyenv-win/pyenv-win): Install and set Python 3.11 using [pyenv-win](https://github.com/pyenv-win/pyenv-win):
```bash ```bash
pyenv install 3.11 pyenv install 3.11
pyenv local 3.11 pyenv local 3.11
``` ```
### 3. Install `Poetry` ### 3. Install `Poetry`
Install [Poetry](https://python-poetry.org/docs/#installing-with-the-official-installer) for dependency management: Install [Poetry](https://python-poetry.org/docs/#installing-with-the-official-installer) for dependency management:
Follow the instructions on the official Poetry website to install it. Follow the instructions on the official Poetry website to install it.
<Callout intent="warning"> <Callout intent="warning">
A bug exists in Poetry versions 1.7.0 and earlier. We strongly recommend upgrading to a tested version. A bug exists in Poetry versions 1.7.0 and earlier. We strongly recommend
To upgrade Poetry to latest tested version, run `poetry self update 1.8.3` after installing it. upgrading to a tested version. To upgrade Poetry to latest tested version, run
`poetry self update 1.8.3` after installing it.
</Callout> </Callout>
### 4. Optional: Install `make` ### 4. Optional: Install `make`
To run various scripts, you need to install `make`. Follow the instructions for your operating system: To run various scripts, you need to install `make`. Follow the instructions for your operating system:
#### macOS #### macOS
(Using Homebrew): (Using Homebrew):
```bash ```bash
brew install make brew install make
``` ```
#### Windows #### Windows
(Using Chocolatey): (Using Chocolatey):
```bash ```bash
choco install make choco install make
``` ```
@ -53,6 +71,7 @@ PrivateGPT allows customization of the setup, from fully local to cloud-based, b
```bash ```bash
poetry install --extras "<extra1> <extra2>..." poetry install --extras "<extra1> <extra2>..."
``` ```
Where `<extra>` can be any of the following options described below. Where `<extra>` can be any of the following options described below.
### Available Modules ### Available Modules
@ -62,7 +81,7 @@ You need to choose one option per category (LLM, Embeddings, Vector Stores, UI).
#### LLM #### LLM
| **Option** | **Description** | **Extra** | | **Option** | **Description** | **Extra** |
|--------------|------------------------------------------------------------------------|---------------------| | ---------- | --------------------------------------------------------------------- | ---------------- |
| **ollama** | Adds support for Ollama LLM, requires Ollama running locally | llms-ollama | | **ollama** | Adds support for Ollama LLM, requires Ollama running locally | llms-ollama |
| llama-cpp | Adds support for local LLM using LlamaCPP | llms-llama-cpp | | llama-cpp | Adds support for local LLM using LlamaCPP | llms-llama-cpp |
| sagemaker | Adds support for Amazon Sagemaker LLM, requires Sagemaker endpoints | llms-sagemaker | | sagemaker | Adds support for Amazon Sagemaker LLM, requires Sagemaker endpoints | llms-sagemaker |
@ -74,7 +93,7 @@ You need to choose one option per category (LLM, Embeddings, Vector Stores, UI).
#### Embeddings #### Embeddings
| **Option** | **Description** | **Extra** | | **Option** | **Description** | **Extra** |
|------------------|--------------------------------------------------------------------------------|-------------------------| | ----------- | -------------------------------------------------------------------------- | ---------------------- |
| **ollama** | Adds support for Ollama Embeddings, requires Ollama running locally | embeddings-ollama | | **ollama** | Adds support for Ollama Embeddings, requires Ollama running locally | embeddings-ollama |
| huggingface | Adds support for local Embeddings using HuggingFace | embeddings-huggingface | | huggingface | Adds support for local Embeddings using HuggingFace | embeddings-huggingface |
| openai | Adds support for OpenAI Embeddings, requires OpenAI API key | embeddings-openai | | openai | Adds support for OpenAI Embeddings, requires OpenAI API key | embeddings-openai |
@ -85,22 +104,25 @@ You need to choose one option per category (LLM, Embeddings, Vector Stores, UI).
#### Vector Stores #### Vector Stores
| **Option** | **Description** | **Extra** | | **Option** | **Description** | **Extra** |
|------------------|-----------------------------------------|-------------------------| | ---------- | ---------------------------------------- | ------------------------ |
| **qdrant** | Adds support for Qdrant vector store | vector-stores-qdrant | | **qdrant** | Adds support for Qdrant vector store | vector-stores-qdrant |
| milvus | Adds support for Milvus vector store | vector-stores-milvus | | milvus | Adds support for Milvus vector store | vector-stores-milvus |
| chroma | Adds support for Chroma DB vector store | vector-stores-chroma | | chroma | Adds support for Chroma DB vector store | vector-stores-chroma |
| postgres | Adds support for Postgres vector store | vector-stores-postgres | | postgres | Adds support for Postgres vector store | vector-stores-postgres |
| clickhouse | Adds support for Clickhouse vector store| vector-stores-clickhouse| | clickhouse | Adds support for Clickhouse vector store | vector-stores-clickhouse |
#### UI #### UI
| **Option** | **Description** | **Extra** | | **Option** | **Description** | **Extra** |
|--------------|------------------------------------------|-----------| | ---------- | -------------------------------- | --------- |
| Gradio | Adds support for UI using Gradio | ui | | Gradio | Adds support for UI using Gradio | ui |
<Callout intent = "warning"> <Callout intent="warning">
A working **Gradio UI client** is provided to test the API, together with a set of useful tools such as bulk A working **Gradio UI client** is provided to test the API, together with a
model download script, ingestion script, documents folder watch, etc. Please refer to the [UI alternatives](/manual/user-interface/alternatives) page for more UI alternatives. set of useful tools such as bulk model download script, ingestion script,
documents folder watch, etc. Please refer to the [UI
alternatives](/manual/user-interface/alternatives) page for more UI
alternatives.
</Callout> </Callout>
## Recommended Setups ## Recommended Setups
@ -109,7 +131,7 @@ There are just some examples of recommended setups. You can mix and match the di
You'll find more information in the Manual section of the documentation. You'll find more information in the Manual section of the documentation.
> **Important for Windows**: In the examples below or how to run PrivateGPT with `make run`, `PGPT_PROFILES` env var is being set inline following Unix command line syntax (works on MacOS and Linux). > **Important for Windows**: In the examples below or how to run PrivateGPT with `make run`, `PGPT_PROFILES` env var is being set inline following Unix command line syntax (works on MacOS and Linux).
If you are using Windows, you'll need to set the env var in a different way, for example: > If you are using Windows, you'll need to set the env var in a different way, for example:
```powershell ```powershell
# Powershell # Powershell
@ -136,6 +158,7 @@ Go to [ollama.ai](https://ollama.ai/) and follow the instructions to install Oll
After the installation, make sure the Ollama desktop app is closed. After the installation, make sure the Ollama desktop app is closed.
Now, start Ollama service (it will start a local inference server, serving both the LLM and the Embeddings): Now, start Ollama service (it will start a local inference server, serving both the LLM and the Embeddings):
```bash ```bash
ollama serve ollama serve
``` ```
@ -152,6 +175,7 @@ ollama pull nomic-embed-text
``` ```
Once done, on a different terminal, you can install PrivateGPT with the following command: Once done, on a different terminal, you can install PrivateGPT with the following command:
```bash ```bash
poetry install --extras "ui llms-ollama embeddings-ollama vector-stores-qdrant" poetry install --extras "ui llms-ollama embeddings-ollama vector-stores-qdrant"
``` ```
@ -175,6 +199,7 @@ You need to have access to sagemaker inference endpoints for the LLM and / or th
Edit the `settings-sagemaker.yaml` file to include the correct Sagemaker endpoints. Edit the `settings-sagemaker.yaml` file to include the correct Sagemaker endpoints.
Then, install PrivateGPT with the following command: Then, install PrivateGPT with the following command:
```bash ```bash
poetry install --extras "ui llms-sagemaker embeddings-sagemaker vector-stores-qdrant" poetry install --extras "ui llms-sagemaker embeddings-sagemaker vector-stores-qdrant"
``` ```
@ -198,6 +223,7 @@ You need an OPENAI API key to run this setup.
Edit the `settings-openai.yaml` file to include the correct API KEY. Never commit it! It's a secret! As an alternative to editing `settings-openai.yaml`, you can just set the env var OPENAI_API_KEY. Edit the `settings-openai.yaml` file to include the correct API KEY. Never commit it! It's a secret! As an alternative to editing `settings-openai.yaml`, you can just set the env var OPENAI_API_KEY.
Then, install PrivateGPT with the following command: Then, install PrivateGPT with the following command:
```bash ```bash
poetry install --extras "ui llms-openai embeddings-openai vector-stores-qdrant" poetry install --extras "ui llms-openai embeddings-openai vector-stores-qdrant"
``` ```
@ -221,6 +247,7 @@ You need to have access to Azure OpenAI inference endpoints for the LLM and / or
Edit the `settings-azopenai.yaml` file to include the correct Azure OpenAI endpoints. Edit the `settings-azopenai.yaml` file to include the correct Azure OpenAI endpoints.
Then, install PrivateGPT with the following command: Then, install PrivateGPT with the following command:
```bash ```bash
poetry install --extras "ui llms-azopenai embeddings-azopenai vector-stores-qdrant" poetry install --extras "ui llms-azopenai embeddings-azopenai vector-stores-qdrant"
``` ```
@ -235,6 +262,30 @@ PrivateGPT will use the already existing `settings-azopenai.yaml` settings file,
The UI will be available at http://localhost:8001 The UI will be available at http://localhost:8001
### Non-Private, FIREWORKS-powered test setup
If you want to test PrivateGPT with FIREWORKS's LLM and Embeddings -taking into account your data is going to FIREWORKS!- you can run the following command:
You need an FIREWORKS API key to run this setup.
Edit the `settings-fireworks.yaml` file to include the correct API KEY. Never commit it! It's a secret! As an alternative to editing `settings-fireworks.yaml`, you can just set the env var FIREWORKS_API_KEY.
Then, install PrivateGPT with the following command:
```bash
poetry install --extras "ui llms-fireworks embeddings-fireworks vector-stores-qdrant embeddings-openai"
```
Once installed, you can run PrivateGPT.
```bash
PGPT_PROFILES=fireworks make run
```
PrivateGPT will use the already existing `settings-fireworks.yaml` settings file, which is already configured to use FIREWORKS LLM and Embeddings endpoints, and Qdrant.
The UI will be available at http://localhost:8001
### Local, Llama-CPP powered setup ### Local, Llama-CPP powered setup
If you want to run PrivateGPT fully locally without relying on Ollama, you can run the following command: If you want to run PrivateGPT fully locally without relying on Ollama, you can run the following command:
@ -244,6 +295,7 @@ poetry install --extras "ui llms-llama-cpp embeddings-huggingface vector-stores-
``` ```
In order for local LLM and embeddings to work, you need to download the models to the `models` folder. You can do so by running the `setup` script: In order for local LLM and embeddings to work, you need to download the models to the `models` folder. You can do so by running the `setup` script:
```bash ```bash
poetry run python scripts/setup poetry run python scripts/setup
``` ```
@ -277,6 +329,7 @@ To do that, you need to install `llama.cpp` python's binding `llama-cpp-python`
that activate `METAL`: you have to pass `-DLLAMA_METAL=on` to the CMake command tha `pip` runs for you (see below). that activate `METAL`: you have to pass `-DLLAMA_METAL=on` to the CMake command tha `pip` runs for you (see below).
In other words, one should simply run: In other words, one should simply run:
```bash ```bash
CMAKE_ARGS="-DLLAMA_METAL=on" pip install --force-reinstall --no-cache-dir llama-cpp-python CMAKE_ARGS="-DLLAMA_METAL=on" pip install --force-reinstall --no-cache-dir llama-cpp-python
``` ```
@ -285,9 +338,10 @@ The above command will force the re-installation of `llama-cpp-python` with `MET
`llama.cpp` locally with your `METAL` libraries (shipped by default with your macOS). `llama.cpp` locally with your `METAL` libraries (shipped by default with your macOS).
More information is available in the documentation of the libraries themselves: More information is available in the documentation of the libraries themselves:
* [llama-cpp-python](https://github.com/abetlen/llama-cpp-python#installation-with-hardware-acceleration)
* [llama-cpp-python's documentation](https://llama-cpp-python.readthedocs.io/en/latest/#installation-with-hardware-acceleration) - [llama-cpp-python](https://github.com/abetlen/llama-cpp-python#installation-with-hardware-acceleration)
* [llama.cpp](https://github.com/ggerganov/llama.cpp#build) - [llama-cpp-python's documentation](https://llama-cpp-python.readthedocs.io/en/latest/#installation-with-hardware-acceleration)
- [llama.cpp](https://github.com/ggerganov/llama.cpp#build)
##### Llama-CPP Windows NVIDIA GPU support ##### Llama-CPP Windows NVIDIA GPU support
@ -297,11 +351,11 @@ dependencies.
Some tips to get it working with an NVIDIA card and CUDA (Tested on Windows 10 with CUDA 11.5 RTX 3070): Some tips to get it working with an NVIDIA card and CUDA (Tested on Windows 10 with CUDA 11.5 RTX 3070):
* Install latest VS2022 (and build tools) https://visualstudio.microsoft.com/vs/community/ - Install latest VS2022 (and build tools) https://visualstudio.microsoft.com/vs/community/
* Install CUDA toolkit https://developer.nvidia.com/cuda-downloads - Install CUDA toolkit https://developer.nvidia.com/cuda-downloads
* Verify your installation is correct by running `nvcc --version` and `nvidia-smi`, ensure your CUDA version is up to - Verify your installation is correct by running `nvcc --version` and `nvidia-smi`, ensure your CUDA version is up to
date and your GPU is detected. date and your GPU is detected.
* [Optional] Install CMake to troubleshoot building issues by compiling llama.cpp directly https://cmake.org/download/ - [Optional] Install CMake to troubleshoot building issues by compiling llama.cpp directly https://cmake.org/download/
If you have all required dependencies properly configured running the If you have all required dependencies properly configured running the
following powershell command should succeed. following powershell command should succeed.
@ -332,9 +386,9 @@ dependencies.
Some tips: Some tips:
* Make sure you have an up-to-date C++ compiler - Make sure you have an up-to-date C++ compiler
* Install CUDA toolkit https://developer.nvidia.com/cuda-downloads - Install CUDA toolkit https://developer.nvidia.com/cuda-downloads
* Verify your installation is correct by running `nvcc --version` and `nvidia-smi`, ensure your CUDA version is up to - Verify your installation is correct by running `nvcc --version` and `nvidia-smi`, ensure your CUDA version is up to
date and your GPU is detected. date and your GPU is detected.
After that running the following command in the repository will install llama.cpp with GPU support: After that running the following command in the repository will install llama.cpp with GPU support:
@ -356,13 +410,17 @@ AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 |
Linux GPU support is done through ROCm. Linux GPU support is done through ROCm.
Some tips: Some tips:
* Install ROCm from [quick-start install guide](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/tutorial/quick-start.html)
* [Install PyTorch for ROCm](https://rocm.docs.amd.com/projects/radeon/en/latest/docs/install/install-pytorch.html) - Install ROCm from [quick-start install guide](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/tutorial/quick-start.html)
- [Install PyTorch for ROCm](https://rocm.docs.amd.com/projects/radeon/en/latest/docs/install/install-pytorch.html)
```bash ```bash
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-6.0/torch-2.1.1%2Brocm6.0-cp311-cp311-linux_x86_64.whl wget https://repo.radeon.com/rocm/manylinux/rocm-rel-6.0/torch-2.1.1%2Brocm6.0-cp311-cp311-linux_x86_64.whl
poetry run pip install --force-reinstall --no-cache-dir torch-2.1.1+rocm6.0-cp311-cp311-linux_x86_64.whl poetry run pip install --force-reinstall --no-cache-dir torch-2.1.1+rocm6.0-cp311-cp311-linux_x86_64.whl
``` ```
* Install bitsandbytes for ROCm
- Install bitsandbytes for ROCm
```bash ```bash
PYTORCH_ROCM_ARCH=gfx900,gfx906,gfx908,gfx90a,gfx1030,gfx1100,gfx1101,gfx940,gfx941,gfx942 PYTORCH_ROCM_ARCH=gfx900,gfx906,gfx908,gfx90a,gfx1030,gfx1100,gfx1101,gfx940,gfx941,gfx942
BITSANDBYTES_VERSION=62353b0200b8557026c176e74ac48b84b953a854 BITSANDBYTES_VERSION=62353b0200b8557026c176e74ac48b84b953a854
@ -374,6 +432,7 @@ pip install . --extra-index-url https://download.pytorch.org/whl/nightly
``` ```
After that running the following command in the repository will install llama.cpp with GPU support: After that running the following command in the repository will install llama.cpp with GPU support:
```bash ```bash
LLAMA_CPP_PYTHON_VERSION=0.2.56 LLAMA_CPP_PYTHON_VERSION=0.2.56
DAMDGPU_TARGETS=gfx900;gfx906;gfx908;gfx90a;gfx1030;gfx1100;gfx1101;gfx940;gfx941;gfx942 DAMDGPU_TARGETS=gfx900;gfx906;gfx908;gfx90a;gfx1030;gfx1100;gfx1101;gfx940;gfx941;gfx942
@ -391,15 +450,15 @@ AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI =
Execution of LLMs locally still has a lot of sharp edges, specially when running on non Linux platforms. Execution of LLMs locally still has a lot of sharp edges, specially when running on non Linux platforms.
You might encounter several issues: You might encounter several issues:
* Performance: RAM or VRAM usage is very high, your computer might experience slowdowns or even crashes. - Performance: RAM or VRAM usage is very high, your computer might experience slowdowns or even crashes.
* GPU Virtualization on Windows and OSX: Simply not possible with docker desktop, you have to run the server directly on - GPU Virtualization on Windows and OSX: Simply not possible with docker desktop, you have to run the server directly on
the host. the host.
* Building errors: Some of PrivateGPT dependencies need to build native code, and they might fail on some platforms. - Building errors: Some of PrivateGPT dependencies need to build native code, and they might fail on some platforms.
Most likely you are missing some dev tools in your machine (updated C++ compiler, CUDA is not on PATH, etc.). Most likely you are missing some dev tools in your machine (updated C++ compiler, CUDA is not on PATH, etc.).
If you encounter any of these issues, please open an issue and we'll try to help. If you encounter any of these issues, please open an issue and we'll try to help.
One of the first reflex to adopt is: get more information. One of the first reflex to adopt is: get more information.
If, during your installation, something does not go as planned, retry in *verbose* mode, and see what goes wrong. If, during your installation, something does not go as planned, retry in _verbose_ mode, and see what goes wrong.
For example, when installing packages with `pip install`, you can add the option `-vvv` to show the details of the installation. For example, when installing packages with `pip install`, you can add the option `-vvv` to show the details of the installation.
@ -414,8 +473,8 @@ To install a C++ compiler on Windows 10/11, follow these steps:
1. Install Visual Studio 2022. 1. Install Visual Studio 2022.
2. Make sure the following components are selected: 2. Make sure the following components are selected:
* Universal Windows Platform development - Universal Windows Platform development
* C++ CMake tools for Windows - C++ CMake tools for Windows
3. Download the MinGW installer from the [MinGW website](https://sourceforge.net/projects/mingw/). 3. Download the MinGW installer from the [MinGW website](https://sourceforge.net/projects/mingw/).
4. Run the installer and select the `gcc` component. 4. Run the installer and select the `gcc` component.

49
poetry.lock generated
View File

@ -1303,6 +1303,20 @@ django = ["dj-database-url", "dj-email-url", "django-cache-url"]
lint = ["flake8 (==4.0.1)", "flake8-bugbear (==21.9.2)", "mypy (==0.910)", "pre-commit (>=2.4,<3.0)"] lint = ["flake8 (==4.0.1)", "flake8-bugbear (==21.9.2)", "mypy (==0.910)", "pre-commit (>=2.4,<3.0)"]
tests = ["dj-database-url", "dj-email-url", "django-cache-url", "pytest"] tests = ["dj-database-url", "dj-email-url", "django-cache-url", "pytest"]
[[package]]
name = "eval-type-backport"
version = "0.2.0"
description = "Like `typing._eval_type`, but lets older Python versions use newer typing features."
optional = true
python-versions = ">=3.8"
files = [
{file = "eval_type_backport-0.2.0-py3-none-any.whl", hash = "sha256:ac2f73d30d40c5a30a80b8739a789d6bb5e49fdffa66d7912667e2015d9c9933"},
{file = "eval_type_backport-0.2.0.tar.gz", hash = "sha256:68796cfbc7371ebf923f03bdf7bef415f3ec098aeced24e054b253a0e78f7b37"},
]
[package.extras]
tests = ["pytest"]
[[package]] [[package]]
name = "fastapi" name = "fastapi"
version = "0.111.1" version = "0.111.1"
@ -3247,32 +3261,33 @@ description = "An experimental Python binding of the Rust MiniJinja template eng
optional = true optional = true
python-versions = ">=3.8" python-versions = ">=3.8"
files = [ files = [
{file = "minijinja-2.0.1-cp38-abi3-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl", hash = "sha256:063b291cb31f5c33eb77bb4cb457f67f14426ca1418232b8ae9f267155d330cc"}, {file = "minijinja-2.2.0-cp38-abi3-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl", hash = "sha256:e4154fcf72e81be01c2733b770e6cb3e584851cb2fa73c58e347b04967d3d7c0"},
{file = "minijinja-2.0.1-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4a4e9d639dd89ce7fef86e82147082ab3c248a36950fa3fbe793685ba322c1b7"}, {file = "minijinja-2.2.0-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4b05e0070c08b550fa9a09ff9c051f47424674332dd56cc54b997dd602887907"},
{file = "minijinja-2.0.1-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:a20373af4ee5430356c196c7fe5f19e3261a4fa16c944542b4de7a2349bac7a6"}, {file = "minijinja-2.2.0-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:360ea4a93fdf1fe327f3e70eed20ecb29f324ca28fae177de0605dcc29869300"},
{file = "minijinja-2.0.1-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ade637bf4826258811a785ccc4e5d41cd2bdf4ec317b1ed3daa4dbbdd020f37d"}, {file = "minijinja-2.2.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c9cad5ccb021ef25b6a271158f4d6636474edb08cd1dd49355aac6b68a48aebb"},
{file = "minijinja-2.0.1-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:d5ec956d777e0fee8e214af48363334c04f098e986038a9e8cb92a0564f81943"}, {file = "minijinja-2.2.0-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:7a85c67c519b413fc4892854782927e1244a24cbbb3a3cb0ac5e57d9fdb1868c"},
{file = "minijinja-2.0.1-cp38-abi3-win32.whl", hash = "sha256:039f4d1a1a73f90917cff1ed7c617eb56e2b2f91bbbdc551adaa448e1673e5c2"}, {file = "minijinja-2.2.0-cp38-abi3-win32.whl", hash = "sha256:e431a2467dd6e1bcb7c511e9fbad012b02c6e5453acdd9fbd4c4af0d34a3d1c5"},
{file = "minijinja-2.0.1-cp38-abi3-win_amd64.whl", hash = "sha256:dca5d7689905dce340e36e47348b505c788daf297253b85a1aff506ea63ad1b8"}, {file = "minijinja-2.2.0-cp38-abi3-win_amd64.whl", hash = "sha256:d4df7e4a09be4249c8243207fa89e6f4d22b853c2b565a99f48e478a30713822"},
{file = "minijinja-2.0.1.tar.gz", hash = "sha256:e774beffebfb8a1ad17e638ef70917cf5e94593f79acb8a8fff7d983169f3a4e"}, {file = "minijinja-2.2.0.tar.gz", hash = "sha256:4411052c7a60f8d56468cc6d17d45d72be3d5e89e9578a04f8336cc56601523c"},
] ]
[[package]] [[package]]
name = "mistralai" name = "mistralai"
version = "1.0.3" version = "1.1.0"
description = "Python Client SDK for the Mistral AI API." description = "Python Client SDK for the Mistral AI API."
optional = true optional = true
python-versions = "<4.0,>=3.8" python-versions = "<4.0,>=3.8"
files = [ files = [
{file = "mistralai-1.0.3-py3-none-any.whl", hash = "sha256:64af7c9192e64dc66b2da6d1c4d54a1324a881c21665a2f93d6b35d9de9f87c8"}, {file = "mistralai-1.1.0-py3-none-any.whl", hash = "sha256:eea0938975195f331d0ded12d14e3c982f09f1b68210200ed4ff0c6b9b22d0fb"},
{file = "mistralai-1.0.3.tar.gz", hash = "sha256:84f1a217666c76fec9d477ae266399b813c3ac32a4a348d2ecd5fe1c039b0667"}, {file = "mistralai-1.1.0.tar.gz", hash = "sha256:9d1fe778e0e8c6ddab714e6a64c6096bd39cfe119ff38ceb5019d8e089df08ba"},
] ]
[package.dependencies] [package.dependencies]
eval-type-backport = ">=0.2.0,<0.3.0"
httpx = ">=0.27.0,<0.28.0" httpx = ">=0.27.0,<0.28.0"
jsonpath-python = ">=1.0.6,<2.0.0" jsonpath-python = ">=1.0.6,<2.0.0"
pydantic = ">=2.8.2,<2.9.0" pydantic = ">=2.9.0,<3.0.0"
python-dateutil = ">=2.9.0.post0,<3.0.0" python-dateutil = "2.8.2"
typing-inspect = ">=0.9.0,<0.10.0" typing-inspect = ">=0.9.0,<0.10.0"
[package.extras] [package.extras]
@ -5005,13 +5020,13 @@ testing = ["fields", "hunter", "process-tests", "pytest-xdist", "six", "virtuale
[[package]] [[package]]
name = "python-dateutil" name = "python-dateutil"
version = "2.9.0.post0" version = "2.8.2"
description = "Extensions to the standard Python datetime module" description = "Extensions to the standard Python datetime module"
optional = false optional = false
python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7" python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7"
files = [ files = [
{file = "python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3"}, {file = "python-dateutil-2.8.2.tar.gz", hash = "sha256:0123cacc1627ae19ddf3c27a5de5bd67ee4586fbdd6440d9748f8abb483d3e86"},
{file = "python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427"}, {file = "python_dateutil-2.8.2-py2.py3-none-any.whl", hash = "sha256:961d03dc3453ebbc59dbdea9e4e11c5651520a876d0f4db161e8674aae935da9"},
] ]
[package.dependencies] [package.dependencies]
@ -7081,4 +7096,4 @@ vector-stores-qdrant = ["llama-index-vector-stores-qdrant"]
[metadata] [metadata]
lock-version = "2.0" lock-version = "2.0"
python-versions = ">=3.11,<3.12" python-versions = ">=3.11,<3.12"
content-hash = "3fa6ef447847895b1a16b8b0422dd9e4fda1aaaadef3af71971eb412da89bf67" content-hash = "df2024950dd0e9c55ae48020af1a71510cf878d696d2aa076e65cd7ee30b08fe"

3
private_gpt/components/embedding/embedding_component.py
View File

@ -76,9 +76,10 @@ class EmbeddingComponent:
) from e ) from e
api_key = settings.fireworks.embedding_api_key or settings.fireworks.api_key api_key = settings.fireworks.embedding_api_key or settings.fireworks.api_key
model = settings.openai.embedding_model
self.embedding_model = FireworksEmbedding( self.embedding_model = FireworksEmbedding(
api_key=api_key, api_key=api_key,
model=model,
) )
case "ollama": case "ollama":
try: try:

16
private_gpt/settings/settings.py
View File

@ -275,15 +275,15 @@ class FireWorksSettings(BaseModel):
"accounts/fireworks/models/llama-v3p1-70b-instruct", "accounts/fireworks/models/llama-v3p1-70b-instruct",
description="FireWorks Model to use. Example: 'accounts/fireworks/models/llama-v3p1-70b-instruct'.", description="FireWorks Model to use. Example: 'accounts/fireworks/models/llama-v3p1-70b-instruct'.",
) )
# embedding_api_base: str = Field( embedding_api_base: str = Field(
# None, None,
# description="Base URL of OpenAI API. Example: 'https://api.openai.com/v1'.", description="Base URL of FIREWORKS API. Example: 'https://api.fireworks.ai/inference/v1'.",
# ) )
embedding_api_key: str embedding_api_key: str
# embedding_model: str = Field( embedding_model: str = Field(
# "text-embedding-ada-002", "nomic-ai/nomic-embed-text-v1.5",
# description="OpenAI embedding Model to use. Example: 'text-embedding-3-large'.", description="FIREWORKS embedding Model to use. Example: 'nomic-ai/nomic-embed-text-v1.5'.",
# ) )
class GeminiSettings(BaseModel): class GeminiSettings(BaseModel):
api_key: str api_key: str