feat: APIServer supports embeddings (#1256)

This commit is contained in:
Fangyin Cheng
2024-03-05 20:21:37 +08:00
committed by GitHub
parent 5f3ee35804
commit 74ec8e52cd
9 changed files with 414 additions and 40 deletions

View File

@@ -10,7 +10,7 @@ The call of multi-model services is compatible with the OpenAI interface, and th
## Start apiserver
After deploying the model service, you need to start the API Server. By default, the model API Server uses port `8100` to start.
```python
```bash
dbgpt start apiserver --controller_addr http://127.0.0.1:8000 --api_keys EMPTY
```
@@ -25,7 +25,7 @@ After the apiserver is started, the service call can be verified. First, let's l
:::tip
List models
:::
```python
```bash
curl http://127.0.0.1:8100/api/v1/models \
-H "Authorization: Bearer EMPTY" \
-H "Content-Type: application/json"
@@ -34,17 +34,31 @@ curl http://127.0.0.1:8100/api/v1/models \
:::tip
Chat
:::
```python
```bash
curl http://127.0.0.1:8100/api/v1/chat/completions \
-H "Authorization: Bearer EMPTY" \
-H "Content-Type: application/json" \
-d '{"model": "vicuna-13b-v1.5", "messages": [{"role": "user", "content": "hello"}]}'
```
:::tip
Embedding
:::
```bash
curl http://127.0.0.1:8100/api/v1/embeddings \
-H "Authorization: Bearer EMPTY" \
-H "Content-Type: application/json" \
-d '{
"model": "text2vec",
"input": "Hello world!"
}'
```
## Verify via OpenAI SDK
```python
```bash
import openai
openai.api_key = "EMPTY"
openai.api_base = "http://127.0.0.1:8100/api/v1"