mirror of
https://github.com/csunny/DB-GPT.git
synced 2026-08-08 23:54:27 +00:00
chore: update 0.7.4 version (#2919)
Co-authored-by: alan.cl <alan.cl@antgroup.com>
This commit is contained in:
alanchen
GitHub
@@ -47,29 +47,37 @@ Core capabilities of the benchmarking module:
|
||||
# Dataset Structure
|
||||
|
||||
## Standard Benchmark Structure
|
||||
| Field | Description | example |
|
||||
|---------|--------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------|
|
||||
| 编号 | Question serial number | 1, 2... |
|
||||
| 数据集ID | Dataset ID | D2025050900161503000025249569, ... |
|
||||
| 用户问题 | Question title | 各性别的平均年龄是多少，并按年龄顺序显示结果？ |
|
||||
| 自定义标签 | Question source, SQL type | KAGGLE_DS_1, CTE1 |
|
||||
| 知识 | Knowledge context required | 暂无 |
|
||||
| Field | Description | example |
|
||||
|---------|--------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------|
|
||||
| 编号 | Question serial number | 1, 2... |
|
||||
| 数据集ID | Dataset ID | D2025050900161503000025249569, ... |
|
||||
| 用户问题 | Question title | 各性别的平均年龄是多少，并按年龄顺序显示结果？ |
|
||||
| 自定义标签 | Question source, SQL type | KAGGLE_DS_1, CTE1 |
|
||||
| 知识 | Knowledge context required | 暂无 |
|
||||
| 标准答案SQL | Correct SQL for the question(based on Alibaba Cloud MaxCompute syntax) | SELECT gender, AVG(age) AS avg_age FROM users GROUP BY gender ORDER BY avg_age |
|
||||
| 标准结果 | Correct SQL query result on the Alibaba Cloud MaxCompute engine (some questions have multiple answers) | {"性别":["Female","Male"],"平均年龄":["27.73","27.84"]} |
|
||||
| 是否排序 | Whether the question involves sorting | {"性别":["Female","Male"],"平均年龄":[27.73,27.84]} |
|
||||
| 标准结果 | Correct SQL query result on the Alibaba Cloud MaxCompute engine (some questions have multiple answers) | `{"性别":["Female","Male"],"平均年龄":["27.73","27.84"]}` |
|
||||
| 是否排序 | Whether the question involves sorting | `{"性别":["Female","Male"],"平均年龄":[27.73,27.84]}` |
|
||||
| prompt | Model conversation prompt | 已知以下数据集，包含了字段名及其采样信息：... |
|
||||
|
||||
|
||||
# How To Use
|
||||
|
||||
## Environment Setup
|
||||
- Start the DB-GPT service, and wait for the benchmark dataset to load automatically. When you see the log line, the dataset has finished loading (about 1~3 minute).
|
||||
- Step1: Upgrade to V0.7.4 and upgrade the metadata database
|
||||
|
||||
For SQLite, the table schema is upgraded automatically by default. For MySQL, you need to run the DDL manually. The file assets/schema/dbgpt.sql contains the complete DDL for the current version. Version-specific DDL changes can be found under assets/schema/upgrade. For example, if you are upgrading from v0.7.1 to v0.7.4, you can run the following DDL:
|
||||
|
||||
```
|
||||
mysql -h127.0.0.1 -uroot -p{your_password} < assets/schema/upgrade/v0_7_4/upgrade_to_v0.7.4.sql
|
||||
```
|
||||
|
||||
- Step2: Start the DB-GPT service, and wait for the benchmark dataset to load automatically. When you see the log line, the dataset has finished loading (about 1~5 minute).
|
||||
|
||||
<p align="left">
|
||||
<img src={'/img/module/benchmark/env_load.png'} width="1000px"/>
|
||||
</p>
|
||||
|
||||
- Register LLM on the DB-GPT platform
|
||||
- Step3: Register LLM on the DB-GPT platform
|
||||
- Method 1: Configure via configuration file. Reference: [ProxyModel Configuration](http://docs.dbgpt.cn/docs/next/installation/advanced_usage/More_proxyllms)
|
||||
- Method 2: Configure via product page. Reference: [Models](http://docs.dbgpt.cn/docs/next/application/llms)
|
||||
|
||||
@@ -144,7 +152,7 @@ Core capabilities of the benchmarking module:
|
||||
| prompt | Model conversation prompt | 已知以下数据集，包含了字段名及其采样信息：... |
|
||||
| Cot长度 | CoT tokens consumed | 100 |
|
||||
| LLM输出结果 | SQL generated by the LLM | select gender as `gender`, avg(cast(age as real)) as `average_age` from di_finance_data group by gender order by avg(cast(age as real)) |
|
||||
| 结果执行 | Query result of the LLM-generated SQL | {"性别":["Female","Male"],"平均年龄":[27.73,27.84]} |
|
||||
| 结果执行 | Query result of the LLM-generated SQL | `{"性别":["Female","Male"],"平均年龄":[27.73,27.84]}` |
|
||||
| 执行结果的报错信息 | Error message if the SQL fails | |
|
||||
| traceId | Log ID | 暂无 |
|
||||
| 耗时（秒） | Time consumed | 10 |
|
||||
@@ -169,10 +177,10 @@ Core capabilities of the benchmarking module:
|
||||
| selfDefineTags | Question source, SQL type | KAGGLE_DS_1, CTE1 |
|
||||
| prompt | Model conversation prompt | 已知以下数据集，包含了字段名及其采样信息：... |
|
||||
| standardAnswerSql | Correct SQL for the question(based on Alibaba Cloud MaxCompute syntax) | select gender as `gender`, avg(cast(age as real)) as `average_age` from di_finance_data group by gender order by avg(cast(age as real)) |
|
||||
| standardAnswer | Correct SQL query result on the Alibaba Cloud MaxCompute engine (some questions have multiple answers) | {"性别": ["Female", "Male"], "平均年龄": ["27.73", "27.84"]} |
|
||||
| standardAnswer | Correct SQL query result on the Alibaba Cloud MaxCompute engine (some questions have multiple answers) | `{"性别": ["Female", "Male"], "平均年龄": ["27.73", "27.84"]}` |
|
||||
| llmCode | Evaluated model name | DeepSeek-V3.1 |
|
||||
| llmOutput | SQL generated by the LLM | select gender as `gender`, avg(cast(age as real)) as `average_age` from di_finance_data group by gender order by avg(cast(age as real)) |
|
||||
| executeResult | Query result of the LLM-generated SQL | {"性别":["Female","Male"],"平均年龄":[27.73,27.84]} |
|
||||
| executeResult | Query result of the LLM-generated SQL | `{"性别":["Female","Male"],"平均年龄":[27.73,27.84]}` |
|
||||
| errorMsg | Comparison error message | |
|
||||
| compareResult | Comparison result between the reference answer and the LLM output | RIGHT: correct; WRONG: incorrect; FAILED: failed (usually the SQL has issues) |
|
||||
|
||||
|
||||
@@ -581,6 +581,10 @@ const sidebars = {
|
||||
type: 'doc',
|
||||
id: 'modules/visual',
|
||||
},
|
||||
{
|
||||
type: 'doc',
|
||||
id: 'modules/benchmark',
|
||||
},
|
||||
|
||||
],
|
||||
link: {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "dbgpt-acc-auto"
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
description = "Add your description here"
|
||||
authors = [
|
||||
{ name = "csunny", email = "cfqcsunny@gmail.com" }
|
||||
|
||||
@@ -1 +1 @@
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
# https://github.com/astral-sh/uv/issues/2252#issuecomment-2624150395
|
||||
[project]
|
||||
name = "dbgpt-acc-flash-attn"
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
description = "Add your description here"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
|
||||
@@ -1 +1 @@
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "dbgpt-app"
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
description = "Add your description here"
|
||||
authors = [
|
||||
{ name = "csunny", email = "cfqcsunny@gmail.com" }
|
||||
|
||||
@@ -1 +1 @@
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "dbgpt-client"
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
description = "Add your description here"
|
||||
authors = [
|
||||
{ name = "csunny", email = "cfqcsunny@gmail.com" }
|
||||
|
||||
@@ -1 +1 @@
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "dbgpt"
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
description = """DB-GPT is an experimental open-source project that uses localized GPT \
|
||||
large models to interact with your data and environment. With this solution, you can be\
|
||||
assured that there is no risk of data leakage, and your data is 100% private and secure.\
|
||||
|
||||
@@ -1 +1 @@
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "dbgpt-ext"
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
description = "Add your description here"
|
||||
authors = [
|
||||
{ name = "csunny", email = "cfqcsunny@gmail.com" }
|
||||
|
||||
@@ -1 +1 @@
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "dbgpt-serve"
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
description = "Add your description here"
|
||||
authors = [
|
||||
{ name = "csunny", email = "cfqcsunny@gmail.com" }
|
||||
|
||||
@@ -1 +1 @@
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
|
||||
@@ -75,6 +75,10 @@ class BenchmarkLLMTask:
|
||||
)
|
||||
response = await self._llm_client.generate(request=request)
|
||||
|
||||
if not response:
|
||||
logger.error("[benchmarkLLMTask] request llm failed, response is None")
|
||||
return None
|
||||
|
||||
if not response.success:
|
||||
code = str(response.error_code)
|
||||
reason = response.text
|
||||
@@ -82,8 +86,12 @@ class BenchmarkLLMTask:
|
||||
return None
|
||||
|
||||
if response.has_text:
|
||||
cot_tokens = 0
|
||||
if response.usage and isinstance(response.usage, dict):
|
||||
cot_tokens = response.usage.get("total_tokens", 0)
|
||||
|
||||
return ReasoningResponse(
|
||||
cot_tokens=response.usage.get("total_tokens", 0),
|
||||
cot_tokens=cot_tokens,
|
||||
think=response.thinking_text if response.has_thinking else None,
|
||||
content=self._get_answer(response.text),
|
||||
)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "dbgpt-mono"
|
||||
version = "0.7.3"
|
||||
version = "0.7.4"
|
||||
description = """DB-GPT is an experimental open-source project that uses localized GPT \
|
||||
large models to interact with your data and environment. With this solution, you can be\
|
||||
assured that there is no risk of data leakage, and your data is 100% private and secure.\
|
||||
|
||||
Reference in New Issue
Block a user