feat:merge main branch

This commit is contained in:
aries_ckt
2023-11-03 10:27:48 +08:00
37 changed files with 1550 additions and 166 deletions

View File

@@ -77,3 +77,4 @@ By analyzing this information, we can identify performance bottlenecks in model
./vms/standalone.md
./vms/index.md
./openai.md

View File

@@ -0,0 +1,51 @@
OpenAI-Compatible RESTful APIs
==================================
(openai-apis-index)=
### Install Prepare
You must [deploy DB-GPT cluster](https://db-gpt.readthedocs.io/en/latest/getting_started/install/cluster/vms/index.html) first.
### Launch Model API Server
```bash
dbgpt start apiserver --controller_addr http://127.0.0.1:8000 --api_keys EMPTY
```
By default, the Model API Server starts on port 8100.
### Validate with cURL
#### List models
```bash
curl http://127.0.0.1:8100/api/v1/models \
-H "Authorization: Bearer EMPTY" \
-H "Content-Type: application/json"
```
#### Chat completions
```bash
curl http://127.0.0.1:8100/api/v1/chat/completions \
-H "Authorization: Bearer EMPTY" \
-H "Content-Type: application/json" \
-d '{"model": "vicuna-13b-v1.5", "messages": [{"role": "user", "content": "hello"}]}'
```
### Validate with OpenAI Official SDK
#### Chat completions
```python
import openai
openai.api_key = "EMPTY"
openai.api_base = "http://127.0.0.1:8100/api/v1"
model = "vicuna-13b-v1.5"
completion = openai.ChatCompletion.create(
model=model,
messages=[{"role": "user", "content": "hello"}]
)
# print the completion
print(completion.choices[0].message.content)
```

View File

@@ -24,9 +24,12 @@ PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions
#Azure
LLM_MODEL=chatgpt_proxyllm
OPENAI_API_TYPE=azure
PROXY_API_KEY={your-openai-sk}
PROXY_SERVER_URL=https://xx.openai.azure.com/v1/chat/completions
PROXY_API_KEY={your-azure-sk}
PROXY_API_BASE=https://{your domain}.openai.azure.com/
PROXY_API_TYPE=azure
PROXY_SERVER_URL=xxxx
PROXY_API_VERSION=2023-05-15
PROXYLLM_BACKEND=gpt-35-turbo
#Aliyun tongyi
LLM_MODEL=tongyi_proxyllm