DB-GPT/docs/getting_started/install/cluster/openai.md

OpenAI-Compatible RESTful APIs

(openai-apis-index)=

Install Prepare

You must deploy DB-GPT cluster first.

Launch Model API Server

dbgpt start apiserver --controller_addr http://127.0.0.1:8000 --api_keys EMPTY

By default, the Model API Server starts on port 8100.

Validate with cURL

List models

curl http://127.0.0.1:8100/api/v1/models \
-H "Authorization: Bearer EMPTY" \
-H "Content-Type: application/json"

Chat completions

curl http://127.0.0.1:8100/api/v1/chat/completions \
-H "Authorization: Bearer EMPTY" \
-H "Content-Type: application/json" \
-d '{"model": "vicuna-13b-v1.5", "messages": [{"role": "user", "content": "hello"}]}'

Validate with OpenAI Official SDK

Chat completions

import openai
openai.api_key = "EMPTY"
openai.api_base = "http://127.0.0.1:8100/api/v1"
model = "vicuna-13b-v1.5"

completion = openai.ChatCompletion.create(
  model=model,
  messages=[{"role": "user", "content": "hello"}]
)
# print the completion
print(completion.choices[0].message.content)