github/DB-GPT
1
0
Fork 0
mirror of https://github.com/csunny/DB-GPT.git synced 2025-11-04 01:17:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(model): Update the default model to glm-4-9b-chat (#1629)

Browse Source
This commit is contained in:
Fangyin Cheng
2024-06-13 09:43:34 +08:00
committed by GitHub
parent e11087aa7e
commit 162e2c9b1c
15 changed files with 157 additions and 132 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.env.template
View File

@@ -17,7 +17,7 @@
#** LLM MODELS **#
#*******************************************************************#
# LLM_MODEL, see dbgpt/configs/model_config.LLM_MODEL_CONFIG
LLM_MODEL=vicuna-13b-v1.5
LLM_MODEL=glm-4-9b-chat
## LLM model path, by default, DB-GPT will read the model path from LLM_MODEL_CONFIG based on the LLM_MODEL.
## Of course you can specify your model path according to LLM_MODEL_PATH
## In DB-GPT, the priority from high to low to read model path:
@@ -25,7 +25,7 @@ LLM_MODEL=vicuna-13b-v1.5
## 2. environment variable with key: MODEL_PATH
## 3. environment variable with key: LLM_MODEL_PATH
## 4. the config in dbgpt/configs/model_config.LLM_MODEL_CONFIG
# LLM_MODEL_PATH=/app/models/vicuna-13b-v1.5
# LLM_MODEL_PATH=/app/models/glm-4-9b-chat
# LLM_PROMPT_TEMPLATE=vicuna_v1.1
MODEL_SERVER=http://127.0.0.1:8000
LIMIT_MODEL_CONCURRENCY=5

2
dbgpt/_private/config.py
View File

@@ -194,7 +194,7 @@ class Config(metaclass=Singleton):
self.CHAT_HISTORY_STORE_TYPE = os.getenv("CHAT_HISTORY_STORE_TYPE", "db")
### LLM Model Service Configuration
self.LLM_MODEL = os.getenv("LLM_MODEL", "vicuna-13b-v1.5")
self.LLM_MODEL = os.getenv("LLM_MODEL", "glm-4-9b-chat")
self.LLM_MODEL_PATH = os.getenv("LLM_MODEL_PATH")
### Proxy llm backend, this configuration is only valid when "LLM_MODEL=proxyllm"

2
dbgpt/agent/expand/resources/search_tool.py
View File

@@ -26,6 +26,8 @@ def baidu_search(
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:112.0) "
"Gecko/20100101 Firefox/112.0"
}
if num_results < 8:
num_results = 8
url = f"https://www.baidu.com/s?wd={query}&rn={num_results}"
response = requests.get(url, headers=headers)
response.encoding = "utf-8"

2
docker-compose.yml
View File

@@ -24,7 +24,7 @@ services:
- LOCAL_DB_HOST=db
- LOCAL_DB_PASSWORD=aa123456
- ALLOWLISTED_PLUGINS=db_dashboard
- LLM_MODEL=vicuna-13b-v1.5
- LLM_MODEL=glm-4-9b-chat
depends_on:
- db
volumes:

2
docker/base/run_sqlite.sh
View File

@@ -4,7 +4,7 @@ docker run --ipc host --gpus all -d \
-p 5000:5000 \
-e LOCAL_DB_TYPE=sqlite \
-e LOCAL_DB_PATH=data/default_sqlite.db \
-e LLM_MODEL=vicuna-13b-v1.5 \
-e LLM_MODEL=glm-4-9b-chat \
-e LANGUAGE=zh \
-v /data:/data \
-v /data/models:/app/models \

4
docker/compose_examples/cluster-docker-compose.yml
View File

@@ -19,7 +19,7 @@ services:
- 8100:8100/tcp
llm-worker:
image: eosphorosai/dbgpt:latest
command: dbgpt start worker --model_name vicuna-13b-v1.5 --model_path /app/models/vicuna-13b-v1.5 --port 8001 --controller_addr http://controller:8000
command: dbgpt start worker --model_name glm-4-9b-chat --model_path /app/models/glm-4-9b-chat --port 8001 --controller_addr http://controller:8000
environment:
- DBGPT_LOG_LEVEL=DEBUG
depends_on:
@@ -66,7 +66,7 @@ services:
- LOCAL_DB_PATH=data/default_sqlite.db
- LOCAL_DB_TYPE=sqlite
- ALLOWLISTED_PLUGINS=db_dashboard
- LLM_MODEL=vicuna-13b-v1.5
- LLM_MODEL=glm-4-9b-chat
- MODEL_SERVER=http://controller:8000
depends_on:
- controller

4
docs/docs/installation/advanced_usage/OpenAI_SDK_call.md
View File

@@ -38,7 +38,7 @@ Chat
curl http://127.0.0.1:8100/api/v1/chat/completions \
-H "Authorization: Bearer EMPTY" \
-H "Content-Type: application/json" \
-d '{"model": "vicuna-13b-v1.5", "messages": [{"role": "user", "content": "hello"}]}'
-d '{"model": "glm-4-9b-chat", "messages": [{"role": "user", "content": "hello"}]}'
```
:::tip
@@ -61,7 +61,7 @@ curl http://127.0.0.1:8100/api/v1/embeddings \
import openai
openai.api_key = "EMPTY"
openai.api_base = "http://127.0.0.1:8100/api/v1"
model = "vicuna-13b-v1.5"
model = "glm-4-9b-chat"
completion = openai.ChatCompletion.create(
model=model,

10
docs/docs/installation/advanced_usage/vLLM_inference.md
View File

@@ -4,15 +4,17 @@ DB-GPT supports [vLLM](https://github.com/vllm-project/vllm) inference, a fast a
## Install dependencies
`vLLM` is an optional dependency in DB-GPT. You can install it manually through the following command.
```python
$ pip install -e ".[vllm]"
```bash
pip install -e ".[vllm]"
```
## Modify configuration file
In the `.env` configuration file, modify the inference type of the model to start `vllm` inference.
```python
LLM_MODEL=vicuna-13b-v1.5
```bash
LLM_MODEL=glm-4-9b-chat
MODEL_TYPE=vllm
# modify the following configuration if you possess GPU resources
# gpu_memory_utilization=0.8
```
For more information about the list of models supported by `vLLM`, please refer to the [vLLM supported model document](https://docs.vllm.ai/en/latest/models/supported_models.html#supported-models).

28
docs/docs/installation/docker.md
View File

@@ -5,15 +5,15 @@
There are two ways to prepare a Docker image. 1. Pull from the official image 2. Build locally. You can **choose any one** during actual use.
1.Pulled from the official image repository, [Eosphoros AI Docker Hub](https://hub.docker.com/u/eosphorosai)
```python
```bash
docker pull eosphorosai/dbgpt:latest
```
2.local build(optional)
```python
```bash
bash docker/build_all_images.sh
```
Check the Docker image
```python
```bash
# command
docker images | grep "eosphorosai/dbgpt"
@@ -24,12 +24,12 @@ eosphorosai/dbgpt latest eb3cdc5b4ead About a minute ago 1
```
`eosphorosai/dbgpt` is the base image, which contains project dependencies and the sqlite database. The `eosphorosai/dbgpt-allinone` image is built from `eosphorosai/dbgpt`, which contains a MySQL database. Of course, in addition to pulling the Docker image, the project also provides Dockerfile files, which can be built directly through scripts in DB-GPT. Here are the build commands:
```python
```bash
bash docker/build_all_images.sh
```
When using it, you need to specify specific parameters. The following is an example of specifying parameter construction:
```python
```bash
bash docker/build_all_images.sh \
--base-image nvidia/cuda:11.8.0-runtime-ubuntu22.04 \
--pip-index-url https://pypi.tuna.tsinghua.edu.cn/simple \
@@ -42,12 +42,12 @@ You can view the specific usage through the command `bash docker/build_all_image
### Run through Sqlite database
```python
```bash
docker run --ipc host --gpus all -d \
-p 5670:5670 \
-e LOCAL_DB_TYPE=sqlite \
-e LOCAL_DB_PATH=data/default_sqlite.db \
-e LLM_MODEL=vicuna-13b-v1.5 \
-e LLM_MODEL=glm-4-9b-chat \
-e LANGUAGE=zh \
-v /data/models:/app/models \
--name dbgpt \
@@ -55,23 +55,23 @@ eosphorosai/dbgpt
```
Open the browser and visit [http://localhost:5670](http://localhost:5670)
- `-e LLM_MODEL=vicuna-13b-v1.5`, which means the base model uses `vicuna-13b-v1.5`. For more model usage, you can view the configuration in `/pilot/configs/model_config.LLM_MODEL_CONFIG`.
- `-e LLM_MODEL=glm-4-9b-chat`, which means the base model uses `glm-4-9b-chat`. For more model usage, you can view the configuration in `/pilot/configs/model_config.LLM_MODEL_CONFIG`.
- `-v /data/models:/app/models`, specifies the model file to be mounted. The directory `/data/models` is mounted in `/app/models` of the container. Of course, it can be replaced with other paths.
After the container is started, you can view the logs through the following command
```python
```bash
docker logs dbgpt -f
```
### Run through MySQL database
```python
```bash
docker run --ipc host --gpus all -d -p 3306:3306 \
-p 5670:5670 \
-e LOCAL_DB_HOST=127.0.0.1 \
-e LOCAL_DB_PASSWORD=aa123456 \
-e MYSQL_ROOT_PASSWORD=aa123456 \
-e LLM_MODEL=vicuna-13b-v1.5 \
-e LLM_MODEL=glm-4-9b-chat \
-e LANGUAGE=zh \
-v /data/models:/app/models \
--name db-gpt-allinone \
@@ -79,16 +79,16 @@ db-gpt-allinone
```
Open the browser and visit [http://localhost:5670](http://localhost:5670)
- `-e LLM_MODEL=vicuna-13b-v1.5`, which means the base model uses `vicuna-13b-v1.5`. For more model usage, you can view the configuration in `/pilot/configs/model_config.LLM_MODEL_CONFIG`.
- `-e LLM_MODEL=glm-4-9b-chat`, which means the base model uses `glm-4-9b-chat`. For more model usage, you can view the configuration in `/pilot/configs/model_config.LLM_MODEL_CONFIG`.
- `-v /data/models:/app/models`, specifies the model file to be mounted. The directory `/data/models` is mounted in `/app/models` of the container. Of course, it can be replaced with other paths.
After the container is started, you can view the logs through the following command
```python
```bash
docker logs db-gpt-allinone -f
```
### Run through the OpenAI proxy model
```python
```bash
PROXY_API_KEY="You api key"
PROXY_SERVER_URL="https://api.openai.com/v1/chat/completions"
docker run --gpus all -d -p 3306:3306 \

10
docs/docs/installation/model_service/cluster.md
View File

@@ -25,12 +25,12 @@ By default, `Model Server` will start on port `8000`
## Start Model Worker
:::tip
Start `chatglm2-6b` model Worker
Start `glm-4-9b-chat` model Worker
:::
```shell
dbgpt start worker --model_name chatglm2-6b \
--model_path /app/models/chatglm2-6b \
dbgpt start worker --model_name glm-4-9b-chat \
--model_path /app/models/glm-4-9b-chat \
--port 8001 \
--controller_addr http://127.0.0.1:8000
```
@@ -92,7 +92,7 @@ $ dbgpt model list
+-------------------+------------+------------+------+---------+---------+-----------------+----------------------------+
| Model Name | Model Type | Host | Port | Healthy | Enabled | Prompt Template | Last Heartbeat |
+-------------------+------------+------------+------+---------+---------+-----------------+----------------------------+
| chatglm2-6b | llm | 172.17.0.2 | 8001 | True | True | | 2023-09-12T23:04:31.287654 |
| glm-4-9b-chat | llm | 172.17.0.2 | 8001 | True | True | | 2023-09-12T23:04:31.287654 |
| WorkerManager | service | 172.17.0.2 | 8001 | True | True | | 2023-09-12T23:04:31.286668 |
| WorkerManager | service | 172.17.0.2 | 8003 | True | True | | 2023-09-12T23:04:29.845617 |
| WorkerManager | service | 172.17.0.2 | 8002 | True | True | | 2023-09-12T23:04:24.598439 |
@@ -124,7 +124,7 @@ MODEL_SERVER=http://127.0.0.1:8000
Or it can be started directly by command to formulate the model.
```shell
LLM_MODEL=chatglm2-6b dbgpt start webserver --light
LLM_MODEL=glm-4-9b-chat dbgpt start webserver --light
```
## Command line usage

52
docs/docs/installation/model_service/stand_alone.md
View File

@@ -1,26 +1,26 @@
# Stand-alone Deployment
## Preparation
```python
```bash
# download source code
$ git clone https://github.com/eosphoros-ai/DB-GPT.git
git clone https://github.com/eosphoros-ai/DB-GPT.git
$ cd DB-GPT
cd DB-GPT
```
## Environment installation
```python
```bash
# create a virtual environment
$ conda create -n dbgpt_env python=3.10
conda create -n dbgpt_env python=3.10
# activate virtual environment
$ conda activate dbgpt_env
conda activate dbgpt_env
```
## Install dependencies
```python
```bash
pip install -e ".[default]"
```
@@ -34,11 +34,11 @@ Download LLM and Embedding model
:::
```python
$ mkdir models && cd models
```bash
mkdir models && cd models
# download embedding model, eg: text2vec-large-chinese
$ git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
```
:::tip
@@ -46,7 +46,7 @@ $ git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
Set up proxy API and modify `.env`configuration
:::
```python
```bash
#set LLM_MODEL TYPE
LLM_MODEL=proxyllm
#set your Proxy Api key and Proxy Server url
@@ -58,23 +58,23 @@ PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions
⚠️ If you have GPU resources, you can use local models to deploy
:::
```python
$ mkdir models && cd models
```bash
mkdir models && cd models
# # download embedding model, eg: vicuna-13b-v1.5 or
$ git clone https://huggingface.co/lmsys/vicuna-13b-v1.5
# # download embedding model, eg: glm-4-9b-chat or
git clone https://huggingface.co/THUDM/glm-4-9b-chat
# download embedding model, eg: text2vec-large-chinese
$ git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
$ popd
popd
```
## Command line startup
```python
LLM_MODEL=vicuna-13b-v1.5
```bash
LLM_MODEL=glm-4-9b-chat
dbgpt start webserver --port 6006
```
By default, the `dbgpt start webserver command` will start the `webserver`, `model controller`, and `model worker` through a single Python process. In the above command, port `6006` is specified.
@@ -86,16 +86,16 @@ By default, the `dbgpt start webserver command` will start the `webserver`, `mod
:::tip
view and display all model services
:::
```python
```bash
dbgpt model list
```
```python
```bash
# result
+-----------------+------------+------------+------+---------+---------+-----------------+----------------------------+
| Model Name | Model Type | Host | Port | Healthy | Enabled | Prompt Template | Last Heartbeat |
+-----------------+------------+------------+------+---------+---------+-----------------+----------------------------+
| vicuna-13b-v1.5 | llm | 172.17.0.9 | 6006 | True | True | | 2023-10-16T19:49:59.201313 |
| glm-4-9b-chat | llm | 172.17.0.9 | 6006 | True | True | | 2023-10-16T19:49:59.201313 |
| WorkerManager | service | 172.17.0.9 | 6006 | True | True | | 2023-10-16T19:49:59.246756 |
+-----------------+------------+------------+------+---------+---------+-----------------+----------------------------+
@@ -105,14 +105,14 @@ Where `WorkerManager` is the management process of `Model Workers`
:::tip
check and verify model serving
:::
```python
dbgpt model chat --model_name vicuna-13b-v1.5
```bash
dbgpt model chat --model_name glm-4-9b-chat
```
The above command will launch an interactive page that allows you to talk to the model through the terminal.
```python
Chatbot started with model vicuna-13b-v1.5. Type 'exit' to leave the chat.
```bash
Chatbot started with model glm-4-9b-chat. Type 'exit' to leave the chat.
You: Hello

85
docs/docs/installation/sourcecode.md
View File

@@ -20,7 +20,7 @@ Download DB-GPT
```python
```bash
git clone https://github.com/eosphoros-ai/DB-GPT.git
```
@@ -32,7 +32,7 @@ git clone https://github.com/eosphoros-ai/DB-GPT.git
Create a Python virtual environment
:::
```python
```bash
python >= 3.10
conda create -n dbgpt_env python=3.10
conda activate dbgpt_env
@@ -44,7 +44,7 @@ pip install -e ".[default]"
:::tip
Copy environment variables
:::
```python
```bash
cp .env.template .env
```
@@ -56,7 +56,7 @@ DB-GPT can be deployed on servers with lower hardware through proxy model, or as
:::info note
⚠️ You need to ensure that git-lfs is installed
```python
```bash
● CentOS installation: yum install git-lfs
● Ubuntu installation: apt-get install git-lfs
● MacOS installation: brew install git-lfs
@@ -79,13 +79,13 @@ import TabItem from '@theme/TabItem';
<TabItem value="openai" label="open ai">
Install dependencies
```python
```bash
pip install -e ".[openai]"
```
Download embedding model
```python
```bash
cd DB-GPT
mkdir models and cd models
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
@@ -93,7 +93,7 @@ git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
Configure the proxy and modify LLM_MODEL, PROXY_API_URL and API_KEY in the `.env`file
```python
```bash
# .env
LLM_MODEL=chatgpt_proxyllm
PROXY_API_KEY={your-openai-sk}
@@ -105,13 +105,13 @@ PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions
<TabItem value="qwen" label="通义千问">
Install dependencies
```python
```bash
pip install dashscope
```
Download embedding model
```python
```bash
cd DB-GPT
mkdir models and cd models
@@ -123,7 +123,7 @@ git clone https://huggingface.co/moka-ai/m3e-large
Configure the proxy and modify LLM_MODEL, PROXY_API_URL and API_KEY in the `.env`file
```python
```bash
# .env
# Aliyun tongyiqianwen
LLM_MODEL=tongyi_proxyllm
@@ -134,13 +134,13 @@ PROXY_SERVER_URL={your_service_url}
<TabItem value="chatglm" label="chatglm" >
Install dependencies
```python
```bash
pip install zhipuai
```
Download embedding model
```python
```bash
cd DB-GPT
mkdir models and cd models
@@ -152,7 +152,7 @@ git clone https://huggingface.co/moka-ai/m3e-large
Configure the proxy and modify LLM_MODEL, PROXY_API_URL and API_KEY in the `.env`file
```python
```bash
# .env
LLM_MODEL=zhipu_proxyllm
PROXY_SERVER_URL={your_service_url}
@@ -165,7 +165,7 @@ ZHIPU_PROXY_API_KEY={your-zhipu-sk}
Download embedding model
```python
```bash
cd DB-GPT
mkdir models and cd models
@@ -177,7 +177,7 @@ git clone https://huggingface.co/moka-ai/m3e-large
Configure the proxy and modify LLM_MODEL, MODEL_VERSION, API_KEY and API_SECRET in the `.env`file
```python
```bash
# .env
LLM_MODEL=wenxin_proxyllm
WEN_XIN_MODEL_VERSION={version} # ERNIE-Bot or ERNIE-Bot-turbo
@@ -190,7 +190,7 @@ WEN_XIN_API_SECRET={your-wenxin-sct}
Yi's API is compatible with OpenAI's API, so you can use the same dependencies as OpenAI's API.
```python
```bash
pip install -e ".[openai]"
```
@@ -225,9 +225,9 @@ YI_API_KEY={your-yi-api-key}
<Tabs
defaultValue="vicuna"
values={[
{label: 'ChatGLM', value: 'chatglm'},
{label: 'Vicuna', value: 'vicuna'},
{label: 'Baichuan', value: 'baichuan'},
{label: 'ChatGLM', value: 'chatglm'},
]}>
<TabItem value="vicuna" label="vicuna">
@@ -241,7 +241,7 @@ YI_API_KEY={your-yi-api-key}
##### Download LLM
```python
```bash
cd DB-GPT
mkdir models and cd models
@@ -255,7 +255,7 @@ git clone https://huggingface.co/lmsys/vicuna-13b-v1.5
```
##### Environment variable configuration, configure the LLM_MODEL parameter in the `.env` file
```python
```bash
# .env
LLM_MODEL=vicuna-13b-v1.5
```
@@ -274,7 +274,7 @@ LLM_MODEL=vicuna-13b-v1.5
##### Download LLM
```python
```bash
cd DB-GPT
mkdir models and cd models
@@ -290,7 +290,7 @@ git clone https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat
```
##### Environment variable configuration, configure the LLM_MODEL parameter in the `.env` file
```python
```bash
# .env
LLM_MODEL=baichuan2-13b
```
@@ -299,16 +299,17 @@ LLM_MODEL=baichuan2-13b
<TabItem value="chatglm" label="chatglm">
##### Hardware requirements description
| Model | Quantize | VRAM Size |
|------------------ |--------------|----------------|
|ChatGLM-6b | 4-bit | 7GB |
|ChatGLM-6b | 8-bit | 9GB |
|ChatGLM-6b | FP16 | 14GB |
| Model | Quantize | VRAM Size |
|--------------------|-------------|----------------|
| glm-4-9b-chat | Not support | 16GB |
| ChatGLM-6b | 4-bit | 7GB |
| ChatGLM-6b | 8-bit | 9GB |
| ChatGLM-6b | FP16 | 14GB |
##### Download LLM
```python
```bash
cd DB-GPT
mkdir models and cd models
@@ -318,13 +319,13 @@ or
git clone https://huggingface.co/moka-ai/m3e-large
# llm model
git clone https://huggingface.co/THUDM/chatglm2-6b
git clone https://huggingface.co/THUDM/glm-4-9b-chat
```
##### Environment variable configuration, configure the LLM_MODEL parameter in the `.env` file
```python
```bash
# .env
LLM_MODEL=chatglm2-6b
LLM_MODEL=glm-4-9b-chat
```
</TabItem>
@@ -347,7 +348,7 @@ Method 1: Download the converted model
:::
If you want to use [Vicuna-13b-v1.5](https://huggingface.co/lmsys/vicuna-13b-v1.5), you can download the converted file [TheBloke/vicuna-13B-v1.5-GGUF](https://huggingface.co/TheBloke/vicuna-13B-v1.5-GGUF), only this one file is needed. Download the file and put it in the model path. You need to rename the model to: `ggml-model-q4_0.gguf`.
```python
```bash
wget https://huggingface.co/TheBloke/vicuna-13B-v1.5-GGUF/resolve/main/vicuna-13b-v1.5.Q4_K_M.gguf -O models/ggml-model-q4_0.gguf
```
@@ -360,7 +361,7 @@ During use, you can also convert the model file yourself according to the instru
#### Install dependencies
llama.cpp is an optional installation item in DB-GPT. You can install it with the following command.
```python
```bash
pip install -e ".[llama_cpp]"
```
@@ -410,13 +411,13 @@ After version 0.4.7, we removed the automatic generation of MySQL database Schem
1. Frist, execute MySQL script to create database and tables.
```python
```bash
$ mysql -h127.0.0.1 -uroot -p{your_password} < ./assets/schema/dbgpt.sql
```
2. Second, set DB-GPT MySQL database settings in `.env` file.
```python
```bash
LOCAL_DB_TYPE=mysql
LOCAL_DB_USER= {your username}
LOCAL_DB_PASSWORD={your_password}
@@ -432,19 +433,19 @@ LOCAL_DB_PORT=3306
The DB-GPT project has a part of test data built-in by default, which can be loaded into the local database for testing through the following command
- **Linux**
```python
```bash
bash ./scripts/examples/load_examples.sh
```
- **Windows**
```python
```bash
.\scripts\examples\load_examples.bat
```
## Run service
The DB-GPT service is packaged into a server, and the entire DB-GPT service can be started through the following command.
```python
```bash
python dbgpt/app/dbgpt_server.py
```
:::info NOTE
@@ -452,9 +453,17 @@ python dbgpt/app/dbgpt_server.py
If you are running version v0.4.3 or earlier, please start with the following command:
```python
```bash
python pilot/server/dbgpt_server.py
```
### Run DB-GPT with command `dbgpt`
If you want to run DB-GPT with the command `dbgpt`:
```bash
dbgpt start webserver
```
:::
## Visit website

72
docs/docs/quickstart.md
View File

@@ -7,7 +7,7 @@ DB-GPT supports the installation and use of a variety of open source and closed
:::info note
- Detailed installation and deployment tutorials can be found in [Installation](/docs/installation).
- This page only introduces deployment based on ChatGPT proxy and local Vicuna model.
- This page only introduces deployment based on ChatGPT proxy and local glm model.
:::
## Environmental preparation
@@ -20,7 +20,7 @@ Download DB-GPT
```python
```bash
git clone https://github.com/eosphoros-ai/DB-GPT.git
```
@@ -32,7 +32,7 @@ git clone https://github.com/eosphoros-ai/DB-GPT.git
Create a Python virtual environment
:::
```python
```bash
python >= 3.10
conda create -n dbgpt_env python=3.10
conda activate dbgpt_env
@@ -44,7 +44,7 @@ pip install -e ".[default]"
:::tip
Copy environment variables
:::
```python
```bash
cp .env.template .env
```
@@ -61,8 +61,8 @@ import TabItem from '@theme/TabItem';
<Tabs
defaultValue="openai"
values={[
{label: 'Open AI', value: 'openai'},
{label: 'Vicuna', value: 'vicuna'},
{label: 'Open AI(Proxy LLM)', value: 'openai'},
{label: 'glm-4(Local LLM)', value: 'glm-4'},
]}>
<TabItem value="openai" label="openai">
@@ -70,7 +70,7 @@ import TabItem from '@theme/TabItem';
:::info note
⚠️ You need to ensure that git-lfs is installed
```python
```bash
● CentOS installation: yum install git-lfs
● Ubuntu installation: apt-get install git-lfs
● MacOS installation: brew install git-lfs
@@ -79,13 +79,13 @@ import TabItem from '@theme/TabItem';
#### Install dependencies
```python
```bash
pip install -e ".[openai]"
```
#### Download embedding model
```python
```bash
cd DB-GPT
mkdir models and cd models
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
@@ -93,7 +93,7 @@ git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
#### Configure the proxy and modify LLM_MODEL, PROXY_API_URL and API_KEY in the `.env`file
```python
```bash
# .env
LLM_MODEL=chatgpt_proxyllm
PROXY_API_KEY={your-openai-sk}
@@ -101,35 +101,32 @@ PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions
```
</TabItem>
<TabItem value="vicuna" label="vicuna">
<TabItem value="glm-4" label="glm-4">
#### Hardware requirements description
| Model | Quantize | VRAM Size |
|:----------------------------------------:|--------------:|---------------|
|Vicuna-7b | 4-bit | 8GB |
|Vicuna-7b | 8-bit | 12GB |
|Vicuna-13b | 4-bit | 12GB |
|Vicuna-13b | 8-bit | 20GB |
| Model | GPU VRAM Size |
|:--------------:|-------------------|
| glm-4-9b | 16GB |
#### Download LLM
```python
```bash
cd DB-GPT
mkdir models and cd models
# embedding model
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
or
git clone https://huggingface.co/moka-ai/m3e-large
# also you can use m3e-large model, you can choose one of them according to your needs
# git clone https://huggingface.co/moka-ai/m3e-large
# llm model, if you use openai or Azure or tongyi llm api service, you don't need to download llm model
git clone https://huggingface.co/lmsys/vicuna-13b-v1.5
# LLM model, if you use openai or Azure or tongyi llm api service, you don't need to download llm model
git clone https://huggingface.co/THUDM/glm-4-9b-chat
```
#### Environment variable configuration, configure the LLM_MODEL parameter in the `.env` file
```python
```bash
# .env
LLM_MODEL=vicuna-13b-v1.5
LLM_MODEL=glm-4-9b-chat
```
</TabItem>
@@ -140,38 +137,49 @@ LLM_MODEL=vicuna-13b-v1.5
Load default test data into SQLite database
- **Linux**
```python
```bash
bash ./scripts/examples/load_examples.sh
```
- **Windows**
```python
```bash
.\scripts\examples\load_examples.bat
```
## Run service
```python
```bash
python dbgpt/app/dbgpt_server.py
```
:::info NOTE
### Run service
### Run old service
If you are running version v0.4.3 or earlier, please start with the following command:
```python
```bash
python pilot/server/dbgpt_server.py
```
### Run DB-GPT with command `dbgpt`
If you want to run DB-GPT with the command `dbgpt`:
```bash
dbgpt start webserver
```
:::
## Visit website
#### 1. Production model:
Open the browser and visit [`http://localhost:5670`](http://localhost:5670)
#### 2. Development mode:
```
### (Optional) Run web front-end separately
On the other hand, you can also run the web front-end separately.
```bash
cd web & npm install
cp .env.template .env
// set the API_BASE_URL to your DB-GPT server address, it usually is http://localhost:5670

4
examples/agents/custom_tool_agent_example.py
View File

@@ -19,6 +19,10 @@ logging.basicConfig(
@tool
def simple_calculator(first_number: int, second_number: int, operator: str) -> float:
"""Simple calculator tool. Just support +, -, *, /."""
if isinstance(first_number, str):
first_number = int(first_number)
if isinstance(second_number, str):
second_number = int(second_number)
if operator == "+":
return first_number + second_number
elif operator == "-":

8
scripts/setup_autodl_env.sh
View File

@@ -34,15 +34,15 @@ install_sys_packages() {
clone_repositories() {
cd /root && git clone https://github.com/eosphoros-ai/DB-GPT.git
mkdir -p /root/DB-GPT/models && cd /root/DB-GPT/models
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
git clone https://huggingface.co/Qwen/Qwen-1_8B-Chat
git clone https://www.modelscope.cn/Jerry0/text2vec-large-chinese.git
git clone https://www.modelscope.cn/qwen/Qwen2-0.5B-Instruct.git
rm -rf /root/DB-GPT/models/text2vec-large-chinese/.git
rm -rf /root/DB-GPT/models/Qwen-1_8B-Chat/.git
rm -rf /root/DB-GPT/models/Qwen2-0.5B-Instruct/.git
}
install_dbgpt_packages() {
conda activate dbgpt && cd /root/DB-GPT && pip install -e ".[default]" && pip install transformers_stream_generator einops
cp .env.template .env && sed -i 's/LLM_MODEL=vicuna-13b-v1.5/LLM_MODEL=qwen-1.8b-chat/' .env
cp .env.template .env && sed -i 's/LLM_MODEL=glm-4-9b-chat/LLM_MODEL=qwen2-0.5b-instruct/' .env
}
clean_up() {