mirror of
https://github.com/imartinez/privateGPT.git
synced 2025-04-28 03:32:18 +00:00
Refactor documentation architecture (#1264)
* Refactor documentation architecture Split into several `tab` and sections * Fix Fern's docs.yml after PR review Thank you Danny! Co-authored-by: dannysheridan <danny@buildwithfern.com> * Re-add quickstart in the overview tab It went missing after a refactoring of the doc architecture * Documentation writing * Adapt Makefile to fern documentation * Do not create overlapping page names in fern documentation This is causing 500. Thank you to @dsinghvi for the troubleshooting and the help! * Add a readme to help to understand how fern documentation work and how to add new pages * Rework the welcome view Redirects directly users to installation guide with links for people that are not familiar with documentation browsing. * Simplify the quickstart guide * PR feedback on installation guide A ton of refactoring can still be made there * PR feedback on ingestion * PR feedback on ingestion splitting * Rename section on LLM * Fix missing word in list of LLMs --------- Co-authored-by: dannysheridan <danny@buildwithfern.com>
This commit is contained in:
lopagela
GitHub
parent
57a829a8e8
commit
36f69eed0f
2
Makefile
2
Makefile
@ -46,7 +46,7 @@ dev:
|
||||
########################################################################################################################
|
||||
|
||||
api-docs:
|
||||
poetry run python scripts/extract_openapi.py private_gpt.main:app --out docs/openapi.json
|
||||
PGPT_PROFILES=mock poetry run python scripts/extract_openapi.py private_gpt.main:app --out fern/openapi/openapi.json
|
||||
|
||||
ingest:
|
||||
@poetry run python scripts/ingest_folder.py $(call args)
|
||||
|
||||
39
fern/README.md
Normal file
39
fern/README.md
Normal file
@ -0,0 +1,39 @@
|
||||
# Documentation of privateGPT
|
||||
|
||||
The documentation of this project is being rendered thanks to [fern](https://github.com/fern-api/fern).
|
||||
|
||||
Fern is basically transforming your `.md` and `.mdx` files into a static website: your documentation.
|
||||
|
||||
The configuration of your documentation is done in the `./docs.yml` file.
|
||||
There, you can configure the navbar, tabs, sections and pages being rendered.
|
||||
|
||||
The documentation of fern (and the syntax of its configuration `docs.yml`) is
|
||||
available there [docs.buildwithfern.com](https://docs.buildwithfern.com/).
|
||||
|
||||
## How to run fern
|
||||
|
||||
**You cannot render your documentation locally without fern credentials.**
|
||||
|
||||
To see how your documentation looks like, you **have to** use the CICD of this
|
||||
repository (by opening a PR, CICD job will be executed, and a preview of
|
||||
your PR's documentation will be deployed in vercel automatically, through fern).
|
||||
|
||||
The only thing you can do locally, is to run `fern check`, which check the syntax of
|
||||
your `docs.yml` file.
|
||||
|
||||
## How to add a new page
|
||||
Add in the `docs.yml` a new `page`, with the following syntax:
|
||||
|
||||
```yml
|
||||
navigation:
|
||||
# ...
|
||||
- tab: my-existing-tab
|
||||
layout:
|
||||
# ...
|
||||
- section: My Existing Section
|
||||
contents:
|
||||
# ...
|
||||
- page: My new page display name
|
||||
# The path of the page, relative to `fern/`
|
||||
path: ./docs/pages/my-existing-tab/new-page-content.mdx
|
||||
```
|
||||
105
fern/docs.yml
105
fern/docs.yml
@ -1,33 +1,92 @@
|
||||
# Main Fern configuration file
|
||||
instances:
|
||||
- url: privategpt.docs.buildwithfern.com
|
||||
custom-domain: docs.privategpt.dev
|
||||
|
||||
title: PrivateGPT | Docs
|
||||
|
||||
navigation:
|
||||
- section: Getting started
|
||||
contents:
|
||||
- page: Welcome
|
||||
path: ./docs/pages/welcome.mdx
|
||||
- page: Quickstart
|
||||
path: ./docs/pages/quickstart.mdx
|
||||
- page: Installation
|
||||
path: ./docs/pages/installation.mdx
|
||||
- page: LLMs
|
||||
path: ./docs/pages/llms.mdx
|
||||
- page: Vector Stores
|
||||
path: ./docs/pages/vectordb.mdx
|
||||
- section: Usage
|
||||
contents:
|
||||
- page: UI manual
|
||||
path: ./docs/pages/ui.mdx
|
||||
- page: Ingestion & Doc mgmt
|
||||
path: ./docs/pages/ingestion.mdx
|
||||
- page: SDKs
|
||||
path: ./docs/pages/sdks.mdx
|
||||
- api: API Reference
|
||||
# The tabs definition, in the top left corner
|
||||
tabs:
|
||||
overview:
|
||||
display-name: Overview
|
||||
icon: "fa-solid fa-home"
|
||||
installation:
|
||||
display-name: Installation
|
||||
icon: "fa-solid fa-download"
|
||||
manual:
|
||||
display-name: Manual
|
||||
icon: "fa-solid fa-book"
|
||||
recipes:
|
||||
display-name: Recipes
|
||||
icon: "fa-solid fa-flask"
|
||||
api-reference:
|
||||
display-name: API Reference
|
||||
icon: "fa-solid fa-file-contract"
|
||||
|
||||
navbar-links:
|
||||
# Definition of tabs contents, will be displayed on the left side of the page, below all tabs
|
||||
navigation:
|
||||
# The default tab
|
||||
- tab: overview
|
||||
layout:
|
||||
- section: Welcome
|
||||
contents:
|
||||
- page: Welcome
|
||||
path: ./docs/pages/overview/welcome.mdx
|
||||
- page: Quickstart
|
||||
path: ./docs/pages/overview/quickstart.mdx
|
||||
# How to install privateGPT, with FAQ and troubleshooting
|
||||
- tab: installation
|
||||
layout:
|
||||
- section: Getting started
|
||||
contents:
|
||||
- page: Installation
|
||||
path: ./docs/pages/installation/installation.mdx
|
||||
# Manual of privateGPT: how to use it and configure it
|
||||
- tab: manual
|
||||
layout:
|
||||
- section: Document management
|
||||
contents:
|
||||
- page: Ingestion
|
||||
path: ./docs/pages/manual/ingestion.mdx
|
||||
- page: Deletion
|
||||
path: ./docs/pages/manual/ingestion-reset.mdx
|
||||
- section: Storage
|
||||
contents:
|
||||
- page: Vector Stores
|
||||
path: ./docs/pages/manual/vectordb.mdx
|
||||
- section: Advanced Setup
|
||||
contents:
|
||||
- page: LLM Backends
|
||||
path: ./docs/pages/manual/llms.mdx
|
||||
- section: User Interface
|
||||
contents:
|
||||
- page: User interface (Gradio) Manual
|
||||
path: ./docs/pages/manual/ui.mdx
|
||||
# Small code snippet or example of usage to help users
|
||||
- tab: recipes
|
||||
layout:
|
||||
- section: Choice of LLM
|
||||
contents:
|
||||
# TODO: add recipes
|
||||
- page: List of LLMs
|
||||
path: ./docs/pages/recipes/list-llm.mdx
|
||||
# More advanced usage of privateGPT, by API
|
||||
- tab: api-reference
|
||||
layout:
|
||||
- section: Overview
|
||||
contents:
|
||||
- page : API Reference overview
|
||||
path: ./docs/pages/api-reference/api-reference.mdx
|
||||
- page: SDKs
|
||||
path: ./docs/pages/api-reference/sdks.mdx
|
||||
- api: API Reference
|
||||
|
||||
# Definition of the navbar, will be displayed in the top right corner.
|
||||
# `type:primary` is always displayed at the most right side of the navbar
|
||||
navbar-links:
|
||||
- type: secondary
|
||||
text: Github
|
||||
url: "https://github.com/imartinez/privateGPT"
|
||||
- type: secondary
|
||||
text: Contact us
|
||||
url: "mailto:hello@zylon.ai"
|
||||
|
||||
1
fern/docs/pages/api-reference/api-reference.mdx
Normal file
1
fern/docs/pages/api-reference/api-reference.mdx
Normal file
@ -0,0 +1 @@
|
||||
# API Reference
|
||||
@ -1,4 +1,6 @@
|
||||
We use [Fern](www.buildwithfern.com) to offer API clients for Node.js, Python, Go, and Java. We recommend using these clients to interact with our endpoints. The clients are kept up to date automatically, so we encourage you to use the latest version.
|
||||
We use [Fern](www.buildwithfern.com) to offer API clients for Node.js, Python, Go, and Java.
|
||||
We recommend using these clients to interact with our endpoints.
|
||||
The clients are kept up to date automatically, so we encourage you to use the latest version.
|
||||
|
||||
## SDKs
|
||||
|
||||
@ -1,37 +0,0 @@
|
||||
## Ingesting & Managing Documents
|
||||
|
||||
The ingestion of documents can be done in different ways:
|
||||
|
||||
* Using the `/ingest` API
|
||||
* Using the Gradio UI
|
||||
* Using the Bulk Local Ingestion functionality (check next section)
|
||||
|
||||
### Bulk Local Ingestion
|
||||
|
||||
When you are running PrivateGPT in a fully local setup, you can ingest a complete folder for convenience (containing
|
||||
pdf, text files, etc.)
|
||||
and optionally watch changes on it with the command:
|
||||
|
||||
```bash
|
||||
make ingest /path/to/folder -- --watch
|
||||
```
|
||||
|
||||
To log the processed and failed files to an additional file, use:
|
||||
|
||||
```bash
|
||||
make ingest /path/to/folder -- --watch --log-file /path/to/log/file.log
|
||||
```
|
||||
|
||||
After ingestion is complete, you should be able to chat with your documents
|
||||
by navigating to http://localhost:8001 and using the option `Query documents`,
|
||||
or using the completions / chat API.
|
||||
|
||||
### Reset Local documents database
|
||||
|
||||
When running in a local setup, you can remove all ingested documents by simply
|
||||
deleting all contents of `local_data` folder (except .gitignore).
|
||||
|
||||
To simplify this process, you can use the command:
|
||||
```bash
|
||||
make wipe
|
||||
```
|
||||
@ -9,9 +9,8 @@
|
||||
cd privateGPT
|
||||
```
|
||||
|
||||
* Install Python 3.11. Ideally through a python version manager like `pyenv`.
|
||||
Python 3.12
|
||||
should work too. Earlier python versions are not supported.
|
||||
* Install Python `3.11` (*if you do not have it already*). Ideally through a python version manager like `pyenv`.
|
||||
Python 3.12 should work too. Earlier python versions are not supported.
|
||||
* osx/linux: [pyenv](https://github.com/pyenv/pyenv)
|
||||
* windows: [pyenv-win](https://github.com/pyenv-win/pyenv-win)
|
||||
|
||||
@ -38,53 +37,17 @@ poetry install --with ui
|
||||
|
||||
Verify everything is working by running `make run` (or `poetry run python -m private_gpt`) and navigate to
|
||||
http://localhost:8001. You should see a [Gradio UI](https://gradio.app/) **configured with a mock LLM** that will
|
||||
echo back the input. Later we'll see how to configure a real LLM.
|
||||
echo back the input. Below we'll see how to configure a real LLM.
|
||||
|
||||
### Settings
|
||||
|
||||
<Callout intent="info">
|
||||
The default settings of PrivateGPT work out-of-the-box for a 100% local setup. Skip this section if you just want to test PrivateGPT locally, and come back later to learn about more configuration options.
|
||||
The default settings of PrivateGPT should work out-of-the-box for a 100% local setup. **However**, as is, it runs exclusively on your CPU.
|
||||
Skip this section if you just want to test PrivateGPT locally, and come back later to learn about more configuration options (and have better performances).
|
||||
</Callout>
|
||||
|
||||
<br />
|
||||
|
||||
PrivateGPT is configured through *profiles* that are defined using yaml files, and selected through env variables. The full list of properties configurable can be found in `settings.yaml`
|
||||
|
||||
#### env var `PGPT_SETTINGS_FOLDER`
|
||||
|
||||
The location of the settings folder. Defaults to the root of the project.
|
||||
Should contain the default `settings.yaml` and any other `settings-{profile}.yaml`.
|
||||
|
||||
#### env var `PGPT_PROFILES`
|
||||
|
||||
By default, the profile definition in `settings.yaml` is loaded.
|
||||
Using this env var you can load additional profiles; format is a comma separated list of profile names.
|
||||
This will merge `settings-{profile}.yaml` on top of the base settings file.
|
||||
|
||||
For example:
|
||||
`PGPT_PROFILES=local,cuda` will load `settings-local.yaml`
|
||||
and `settings-cuda.yaml`, their contents will be merged with
|
||||
later profiles properties overriding values of earlier ones like `settings.yaml`.
|
||||
|
||||
During testing, the `test` profile will be active along with the default, therefore `settings-test.yaml`
|
||||
file is required.
|
||||
|
||||
#### Environment variables expansion
|
||||
|
||||
Configuration files can contain environment variables,
|
||||
they will be expanded at runtime.
|
||||
|
||||
Expansion must follow the pattern `${VARIABLE_NAME:default_value}`.
|
||||
|
||||
For example, the following configuration will use the value of the `PORT`
|
||||
environment variable or `8001` if it's not set.
|
||||
Missing variables with no default will produce an error.
|
||||
|
||||
```yaml
|
||||
server:
|
||||
port: ${PORT:8001}
|
||||
```
|
||||
|
||||
### Local LLM requirements
|
||||
|
||||
Install extra dependencies for local execution:
|
||||
@ -122,17 +85,33 @@ is used.
|
||||
|
||||
#### Customizing low level parameters
|
||||
|
||||
Currently not all the parameters of llama-cpp and llama-cpp-python are available at PrivateGPT's `settings.yaml` file. In case you need to customize parameters such as the number of layers loaded into the GPU, you might change these at the `llm_component.py` file under the `private_gpt/components/llm/llm_component.py`. If you are getting an out of memory error, you might also try a smaller model or stick to the proposed recommended models, instead of custom tuning the parameters.
|
||||
Currently, not all the parameters of `llama.cpp` and `llama-cpp-python` are available at PrivateGPT's `settings.yaml` file.
|
||||
In case you need to customize parameters such as the number of layers loaded into the GPU, you might change
|
||||
these at the `llm_component.py` file under the `private_gpt/components/llm/llm_component.py`.
|
||||
|
||||
If you are getting an out of memory error, you might also try a smaller model or stick to the proposed
|
||||
recommended models, instead of custom tuning the parameters.
|
||||
|
||||
#### OSX GPU support
|
||||
|
||||
You will need to build [llama.cpp](https://github.com/ggerganov/llama.cpp) with
|
||||
metal support. To do that run:
|
||||
You will need to build [llama.cpp](https://github.com/ggerganov/llama.cpp) with metal support.
|
||||
|
||||
To do that, you need to install `llama.cpp` python's binding `llama-cpp-python` through pip, with the compilation flag
|
||||
that activate `METAL`: you have to pass `-DLLAMA_METAL=on` to the CMake command tha `pip` runs for you (see below).
|
||||
|
||||
In other words, one should simply run:
|
||||
```bash
|
||||
CMAKE_ARGS="-DLLAMA_METAL=on" pip install --force-reinstall --no-cache-dir llama-cpp-python
|
||||
```
|
||||
|
||||
The above command will force the re-installation of `llama-cpp-python` with `METAL` support by compiling
|
||||
`llama.cpp` locally with your `METAL` libraries (shipped by default with your macOS).
|
||||
|
||||
More information is available in the documentation of the libraries themselves:
|
||||
* [llama-cpp-python](https://github.com/abetlen/llama-cpp-python#installation-with-hardware-acceleration)
|
||||
* [llama-cpp-python's documentation](https://llama-cpp-python.readthedocs.io/en/latest/#installation-with-hardware-acceleration)
|
||||
* [llama.cpp](https://github.com/ggerganov/llama.cpp#build)
|
||||
|
||||
#### Windows NVIDIA GPU support
|
||||
|
||||
Windows GPU support is done through CUDA.
|
||||
@ -157,7 +136,7 @@ $env:CMAKE_ARGS='-DLLAMA_CUBLAS=on'; poetry run pip install --force-reinstall --
|
||||
If your installation was correct, you should see a message similar to the following next
|
||||
time you start the server `BLAS = 1`.
|
||||
|
||||
```
|
||||
```console
|
||||
llama_new_context_with_model: total VRAM used: 4857.93 MB (model: 4095.05 MB, context: 762.87 MB)
|
||||
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 0 | VSX = 0 |
|
||||
```
|
||||
@ -194,7 +173,7 @@ llama_new_context_with_model: total VRAM used: 4857.93 MB (model: 4095.05 MB, co
|
||||
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 0 | VSX = 0 |
|
||||
```
|
||||
|
||||
#### Known issues and Troubleshooting
|
||||
### Known issues and Troubleshooting
|
||||
|
||||
Execution of LLMs locally still has a lot of sharp edges, specially when running on non Linux platforms.
|
||||
You might encounter several issues:
|
||||
@ -206,6 +185,11 @@ You might encounter several issues:
|
||||
Most likely you are missing some dev tools in your machine (updated C++ compiler, CUDA is not on PATH, etc.).
|
||||
If you encounter any of these issues, please open an issue and we'll try to help.
|
||||
|
||||
One of the first reflex to adopt is: get more information.
|
||||
If, during your installation, something does not go as planned, retry in *verbose* mode, and see what goes wrong.
|
||||
|
||||
For example, when installing packages with `pip install`, you can add the option `-vvv` to show the details of the installation.
|
||||
|
||||
#### Troubleshooting: C++ Compiler
|
||||
|
||||
If you encounter an error while building a wheel during the `pip install` process, you may need to install a C++
|
||||
@ -222,9 +206,10 @@ To install a C++ compiler on Windows 10/11, follow these steps:
|
||||
3. Download the MinGW installer from the [MinGW website](https://sourceforge.net/projects/mingw/).
|
||||
4. Run the installer and select the `gcc` component.
|
||||
|
||||
** For OSX **
|
||||
**For OSX**
|
||||
|
||||
1. Check if you have a C++ compiler installed, Xcode might have done it for you. for example running `gcc`.
|
||||
1. Check if you have a C++ compiler installed, `Xcode` should have done it for you. To install Xcode, go to the App
|
||||
Store and search for Xcode and install it. **Or** you can install the command line tools by running `xcode-select --install`.
|
||||
2. If not, you can install clang or gcc with homebrew `brew install gcc`
|
||||
|
||||
#### Troubleshooting: Mac Running Intel
|
||||
80
fern/docs/pages/manual/configuration.mdx
Normal file
80
fern/docs/pages/manual/configuration.mdx
Normal file
@ -0,0 +1,80 @@
|
||||
# Configure your private GPT
|
||||
|
||||
The configuration of your private GPT server is done thanks to `settings` files (more precisely `settings.yaml`).
|
||||
These text files are written using the [YAML](https://en.wikipedia.org/wiki/YAML) syntax.
|
||||
|
||||
While privateGPT is distributing safe and universal configuration files, you might want to quickly customize your
|
||||
privateGPT, and this can be done using the `settings` files.
|
||||
|
||||
This project is defining the concept of **profiles** (or configuration profiles).
|
||||
This mechanism, using your environment variables, is giving you the ability to easily switch between
|
||||
configuration you've made.
|
||||
|
||||
A typical use case of profile is to easily switch between LLM and embeddings.
|
||||
To be a bit more precise, you can change the language (to French, Spanish, Italian, English, etc) by simply changing
|
||||
the profile you've selected; no code changes required!
|
||||
|
||||
PrivateGPT is configured through *profiles* that are defined using yaml files, and selected through env variables.
|
||||
The full list of properties configurable can be found in `settings.yaml`.
|
||||
|
||||
## How to know which profiles exist
|
||||
Given that a profile `foo_bar` points to the file `settings-foo_bar.yaml` and vice-versa, you simply have to look
|
||||
at the files starting with `settings` and ending in `.yaml`.
|
||||
|
||||
## How to use an existing profiles
|
||||
**Please note that the syntax to set the value of an environment variables depends on your OS**.
|
||||
You have to set environment variable `PGPT_PROFILES` to the name of the profile you want to use.
|
||||
|
||||
For example, on **linux and macOS**, this gives:
|
||||
```bash
|
||||
export PGPT_PROFILES=my_profile_name_here
|
||||
```
|
||||
|
||||
Windows Powershell(s) have a different syntax, one of them being:
|
||||
```shell
|
||||
set PGPT_PROFILES=my_profile_name_here
|
||||
```
|
||||
If the above is not working, you might want to try other ways to set an env variable in your window's terminal.
|
||||
|
||||
---
|
||||
|
||||
Once you've set this environment variable to the desired profile, you can simply launch your privateGPT,
|
||||
and it will run using your profile on top of the default configuration.
|
||||
|
||||
## Reference
|
||||
Additional details on the profiles are described in this section
|
||||
|
||||
### Environment variable `PGPT_SETTINGS_FOLDER`
|
||||
|
||||
The location of the settings folder. Defaults to the root of the project.
|
||||
Should contain the default `settings.yaml` and any other `settings-{profile}.yaml`.
|
||||
|
||||
### Environment variable `PGPT_PROFILES`
|
||||
|
||||
By default, the profile definition in `settings.yaml` is loaded.
|
||||
Using this env var you can load additional profiles; format is a comma separated list of profile names.
|
||||
This will merge `settings-{profile}.yaml` on top of the base settings file.
|
||||
|
||||
For example:
|
||||
`PGPT_PROFILES=local,cuda` will load `settings-local.yaml`
|
||||
and `settings-cuda.yaml`, their contents will be merged with
|
||||
later profiles properties overriding values of earlier ones like `settings.yaml`.
|
||||
|
||||
During testing, the `test` profile will be active along with the default, therefore `settings-test.yaml`
|
||||
file is required.
|
||||
|
||||
### Environment variables expansion
|
||||
|
||||
Configuration files can contain environment variables,
|
||||
they will be expanded at runtime.
|
||||
|
||||
Expansion must follow the pattern `${VARIABLE_NAME:default_value}`.
|
||||
|
||||
For example, the following configuration will use the value of the `PORT`
|
||||
environment variable or `8001` if it's not set.
|
||||
Missing variables with no default will produce an error.
|
||||
|
||||
```yaml
|
||||
server:
|
||||
port: ${PORT:8001}
|
||||
```
|
||||
14
fern/docs/pages/manual/ingestion-reset.mdx
Normal file
14
fern/docs/pages/manual/ingestion-reset.mdx
Normal file
@ -0,0 +1,14 @@
|
||||
# Reset Local documents database
|
||||
|
||||
When running in a local setup, you can remove all ingested documents by simply
|
||||
deleting all contents of `local_data` folder (except .gitignore).
|
||||
|
||||
To simplify this process, you can use the command:
|
||||
```bash
|
||||
make wipe
|
||||
```
|
||||
|
||||
# Advanced usage
|
||||
|
||||
You can actually delete your documents from your storage by using the
|
||||
API endpoint `DELETE` in the Ingestion API.
|
||||
78
fern/docs/pages/manual/ingestion.mdx
Normal file
78
fern/docs/pages/manual/ingestion.mdx
Normal file
@ -0,0 +1,78 @@
|
||||
# Ingesting & Managing Documents
|
||||
|
||||
The ingestion of documents can be done in different ways:
|
||||
|
||||
* Using the `/ingest` API
|
||||
* Using the Gradio UI
|
||||
* Using the Bulk Local Ingestion functionality (check next section)
|
||||
|
||||
## Bulk Local Ingestion
|
||||
|
||||
When you are running PrivateGPT in a fully local setup, you can ingest a complete folder for convenience (containing
|
||||
pdf, text files, etc.)
|
||||
and optionally watch changes on it with the command:
|
||||
|
||||
```bash
|
||||
make ingest /path/to/folder -- --watch
|
||||
```
|
||||
|
||||
To log the processed and failed files to an additional file, use:
|
||||
|
||||
```bash
|
||||
make ingest /path/to/folder -- --watch --log-file /path/to/log/file.log
|
||||
```
|
||||
|
||||
After ingestion is complete, you should be able to chat with your documents
|
||||
by navigating to http://localhost:8001 and using the option `Query documents`,
|
||||
or using the completions / chat API.
|
||||
|
||||
## Ingestion troubleshooting
|
||||
|
||||
Are you running out of memory when ingesting files?
|
||||
|
||||
To do not run out of memory, you should ingest your documents without the LLM loaded in your (video) memory.
|
||||
To do so, you should change your configuration to set `llm.mode: mock`.
|
||||
|
||||
In other words, you should update your `settings.yaml` (or your custom configuration file) to set the
|
||||
following **before** ingesting your documents:
|
||||
```yaml
|
||||
llm:
|
||||
mode: mock
|
||||
```
|
||||
|
||||
Once your documents are ingested, you can set the `llm.mode` value back to `local` (or your previous custom value).
|
||||
|
||||
You can also use the existing `PGPT_PROFILES=mock` that will set the `llm.mode` to `mock` for you.
|
||||
|
||||
## Supported file formats
|
||||
|
||||
privateGPT by default supports all the file formats that contains clear text (for example, `.txt` files, `.html`, etc.).
|
||||
However, these text based file formats as only considered as text files, and are not pre-processed in any other way.
|
||||
|
||||
It also supports the following file formats:
|
||||
* `.hwp`
|
||||
* `.pdf`
|
||||
* `.docx`
|
||||
* `.pptx`
|
||||
* `.ppt`
|
||||
* `.pptm`
|
||||
* `.jpg`
|
||||
* `.png`
|
||||
* `.jpeg`
|
||||
* `.mp3`
|
||||
* `.mp4`
|
||||
* `.csv`
|
||||
* `.epub`
|
||||
* `.md`
|
||||
* `.mbox`
|
||||
* `.ipynb`
|
||||
* `.json`
|
||||
|
||||
**Please note the following nuance**: while `privateGPT` supports these file formats, it **might** require additional
|
||||
dependencies to be installed in your python's virtual environment.
|
||||
For example, if you try to ingest `.epub` files, `privateGPT` might fail to do it, and will instead display an
|
||||
explanatory error asking you to download the necessary dependencies to install this file format.
|
||||
|
||||
|
||||
**Other file formats might work**, but they will be considered as plain text
|
||||
files (in other words, they will be ingested as `.txt` files).
|
||||
21
fern/docs/pages/overview/quickstart.mdx
Normal file
21
fern/docs/pages/overview/quickstart.mdx
Normal file
@ -0,0 +1,21 @@
|
||||
## Local Installation steps
|
||||
|
||||
The steps in [Installation](/installation) section are better explained and cover more
|
||||
setup scenarios (macOS, Windows, Linux).
|
||||
But if you like one-liners, have python3.11 installed, and you are running a UNIX (macOS or Linux)
|
||||
system, you can get up and running on CPU in few lines:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/imartinez/privateGPT && cd privateGPT && \
|
||||
python3.11 -m venv .venv && source .venv/bin/activate && \
|
||||
pip install --upgrade pip poetry && poetry install --with ui,local && ./scripts/setup
|
||||
|
||||
# Launch the privateGPT API server **and** the gradio UI
|
||||
python3.11 -m private_gpt
|
||||
|
||||
# In another terminal, create a new browser window on your private GPT!
|
||||
open http:////127.0.0.1:8001/
|
||||
```
|
||||
|
||||
The above is not working, or it is too slow, so **you want to run it on GPU(s)**?
|
||||
Please check the more detailed [installation guide](/installation).
|
||||
@ -1,6 +1,16 @@
|
||||
## Introduction 👋
|
||||
|
||||
PrivateGPT provides an **API** containing all the building blocks required to build **private, context-aware AI applications**. The API follows and extends OpenAI API standard, and supports both normal and streaming responses.
|
||||
PrivateGPT provides an **API** containing all the building blocks required to
|
||||
build **private, context-aware AI applications**.
|
||||
The API follows and extends OpenAI API standard, and supports both normal and streaming responses.
|
||||
That means that, if you can use OpenAI API in one of your tools, you can use your own PrivateGPT API instead,
|
||||
with no code changes, **and for free** if you are running privateGPT in `local` mode.
|
||||
|
||||
Looking for the installation quickstart? [Quickstart installation guide for Linux and macOS](/overview/welcome/quickstart).
|
||||
|
||||
Do you want to install it on Windows? Or do you want to take full advantage of your hardware for better performances?
|
||||
The installation guide will help you in the [Installation section](/installation).
|
||||
|
||||
|
||||
## Frequently Visited Resources
|
||||
|
||||
@ -38,5 +48,6 @@ The API is divided in two logical blocks:
|
||||
documents.
|
||||
|
||||
<Callout intent = "info">
|
||||
A working **Gradio UI client** is provided to test the API, together with a set of useful tools such as bulk model download script, ingestion script, documents folder watch, etc.
|
||||
A working **Gradio UI client** is provided to test the API, together with a set of useful tools such as bulk
|
||||
model download script, ingestion script, documents folder watch, etc.
|
||||
</Callout>
|
||||
@ -1,38 +0,0 @@
|
||||
## Local Installation steps
|
||||
|
||||
The steps in `Installation and Settings` section are better explained and cover more
|
||||
setup scenarios. But if you are looking for a quick setup guide, here it is:
|
||||
|
||||
```bash
|
||||
# Clone the repo
|
||||
git clone https://github.com/imartinez/privateGPT
|
||||
cd privateGPT
|
||||
|
||||
# Install Python 3.11
|
||||
pyenv install 3.11
|
||||
pyenv local 3.11
|
||||
|
||||
# Install dependencies
|
||||
poetry install --with ui,local
|
||||
|
||||
# Download Embedding and LLM models
|
||||
poetry run python scripts/setup
|
||||
|
||||
# (Optional) For Mac with Metal GPU, enable it. Check Installation and Settings section
|
||||
to know how to enable GPU on other platforms
|
||||
CMAKE_ARGS="-DLLAMA_METAL=on" pip install --force-reinstall --no-cache-dir llama-cpp-python
|
||||
|
||||
# Run the local server
|
||||
PGPT_PROFILES=local make run
|
||||
|
||||
# Note: on Mac with Metal you should see a ggml_metal_add_buffer log, stating GPU is
|
||||
being used
|
||||
|
||||
# Navigate to the UI and try it out!
|
||||
http://localhost:8001/
|
||||
```
|
||||
|
||||
## API
|
||||
|
||||
As explained in the introduction, the API contains high level APIs (ingestion and chat/completions) and low level APIs
|
||||
(embeddings and chunk retrieval). In this section the different specific API calls are explained.
|
||||
28
fern/docs/pages/recipes/list-llm.mdx
Normal file
28
fern/docs/pages/recipes/list-llm.mdx
Normal file
@ -0,0 +1,28 @@
|
||||
## List of working LLM
|
||||
|
||||
**Do you have any working combination of LLM and embeddings?**
|
||||
Please open a PR to add it to the list, and come on our Discord to tell us about it!
|
||||
|
||||
## Example of configuration
|
||||
|
||||
You might want to change the prompt depending on the language and model you are using.
|
||||
|
||||
### English, with instructions
|
||||
|
||||
`settings-en.yaml`:
|
||||
```yml
|
||||
local:
|
||||
llm_hf_repo_id: TheBloke/Mistral-7B-Instruct-v0.1-GGUF
|
||||
llm_hf_model_file: mistral-7b-instruct-v0.1.Q4_K_M.gguf
|
||||
embedding_hf_model_name: BAAI/bge-small-en-v1.5
|
||||
```
|
||||
|
||||
### French, with instructions
|
||||
|
||||
`settings-fr.yaml`:
|
||||
```yml
|
||||
local:
|
||||
llm_hf_repo_id: TheBloke/Vigogne-2-7B-Instruct-GGUF
|
||||
llm_hf_model_file: vigogne-2-7b-instruct.Q4_K_M.gguf
|
||||
embedding_hf_model_name: dangvantuan/sentence-camembert-base
|
||||
```
|
||||
5
settings-mock.yaml
Normal file
5
settings-mock.yaml
Normal file
@ -0,0 +1,5 @@
|
||||
server:
|
||||
env_name: ${APP_ENV:mock}
|
||||
|
||||
llm:
|
||||
mode: mock
|
||||
Loading…
Reference in New Issue
Block a user