Commit Graph

3351 Commits

Author SHA1 Message Date
alan.cl
597b1e63b8 fix(connector): 修复 CI 风格检测与陈旧单测
CI build-web 失败 + make fmt-check 报错的修复,均不影响功能逻辑:

- side-bar.tsx: 修复两处超长 className 键值对的 prettier 换行(CI 构建失败根因)
- agentic_data_api.py: BaseTool 加入 TYPE_CHECKING(F821);循环变量 tool->sub_tool 避免遮蔽 import(F402)
- catalog.py/task_executor.py/models.py: 超长字符串隐式拼接换行(E501),拼接结果不变
- manager.py: 精简过长的 docstring 与模块注释(E501)
- test_manager.py: 同步 4 个陈旧测试断言以匹配当前 connector 鉴权模型
  (所有类型强制 server_uri、headers 由 auth_type+token 派生、tool 摘要含 original_name)
- test_service.py: 精简过长的测试 docstring(E501)

验证: make fmt-check 全绿;connector 测试 53 passed

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 21:35:49 +08:00
alan.cl
d554c6c4b9 fix(web): repair merge conflicts in chat input area + clean up ManusLeftPanel
- index.tsx: 修复合并导致的 JSX 结构错乱(重复的 Voice/Send 按钮、孤儿块、
  错位的 )}),恢复单一聊天输入区工具栏,融合 main 的 ContextUsageBar/taskPlan
  与本分支的 connector 选择器
- ManusLeftPanel.tsx: 清理未使用变量(hasObservations/hasContent/contentExpanded/
  未用的 t),给保留接口的 onShare/setDownloading 加 _ 前缀,修复 getTodoStep*
  的 TFunction 类型断言,prettier 格式化 import
- en/chat.ts: use_connector 文案统一为 'Select MCP'
- tongyi.toml: 新增 agent_context 上下文窗口预算配置
- scheduled_task/serve.py: 增加 DBGPT_CHAT_TASK_SCHEDULER_ENABLED 开关与内存 jobstore

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 20:12:44 +08:00
alan.cl
e58777768d Merge branch 'main' into feat/ai_connector
# Conflicts:
#	assets/schema/dbgpt.sql
#	assets/schema/upgrade/v0_8_1/upgrade_to_v0.8.1.sql
#	packages/dbgpt-app/src/dbgpt_app/openapi/api_v1/agentic_data_api.py
#	web/locales/en/chat.ts
#	web/locales/zh/chat.ts
#	web/new-components/chat/content/ManusLeftPanel.tsx
#	web/pages/index.tsx
2026-06-04 16:02:22 +08:00
alan.cl
de9e59500d Merge remote-tracking branch 'origin/main' 2026-06-04 09:50:08 +08:00
alan.cl
3cf8cbaddc docs(application): add MCP connectors & scheduled tasks guides + i18n
- Add EN/ZH docs for MCP Connectors and Scheduled Tasks under Application
- Wire two new sidebar groups (above App Guides), collapse existing groups
- Add zh-CN sidebar label translations in current.json
- Bundle product screenshots (mcp/, schedule/) for both locales
- Frontend i18n: extract hardcoded Chinese in connector/scheduled-task UI
  and home composer pickers into en/zh locale keys

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 19:33:37 +08:00
alan.cl
910d928917 fix(i18n): 保存为定时任务按钮支持国际化 2026-06-03 17:58:49 +08:00
Anna Tao
5c0c25e8f1 feat: add CLIENT SETNAME to Valkey vector store client (#3090)
Signed-off-by: Anna Tao <annatao@amazon.com>
2026-06-03 17:43:42 +08:00
Octopus
82408793f5 feat: upgrade MiniMax default model to M3 (#3093) 2026-06-03 17:42:55 +08:00
alan.cl
e2fee36b13 feat(scheduled-task): add cron-based chat replay scheduled tasks
后端:新增独立 scheduled_task serve 模块(API/DAO/models/service/
chat_replay_runner)与 util/scheduler 单例调度器,并在 serve_initialization
注册 ScheduledTaskServe;将原 connector 下的 scheduled_task 旧实现迁移至此。

前端:新增定时任务列表/详情页、保存与编辑抽屉、CronInput、执行历史表
及 use-scheduled-task hook 与类型定义。

修复:主页保存定时任务时,buildSnapshot 改为复用发送瞬间的真实 payload,
使快照忠实携带 file_path / database_name / knowledge_space 等执行上下文;
详情页"任务环境"展示数据库 / 文件 / 知识库,并优先显示技能名。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 21:22:30 +08:00
alan.cl
dae085b6df fix ui 2026-05-31 12:06:54 +08:00
alan.cl
9f54958429 chore(schema): add v0.8.1 schema for MCP connector + scheduled task tables
Adds two MySQL tables introduced by the feat/ai_connector branch:
- connector_instance: stores MCP connector instances (encrypted credentials,
  transport / extra config JSON, lifecycle status)
- dbgpt_serve_scheduled_task: scheduled cron tasks bound to a connector
  (APScheduler expression + tool invocation snapshot)

Files:
- assets/schema/dbgpt.sql: append both tables to keep the baseline current
- assets/schema/upgrade/v0_8_1/upgrade_to_v0.8.1.sql: 0.8.0 -> 0.8.1 delta
- assets/schema/upgrade/v0_8_1/v0.8.1.sql: full baseline snapshot at v0.8.1

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-31 11:50:26 +08:00
alan.cl
ad7b21c4c5 feat(connector): support mcp streamablehttp protocol 2026-05-30 15:54:25 +08:00
Artoriasn
1c9e320521 fix(model): support DeepSeek V4 Pro (#3079) 2026-05-27 21:09:36 +08:00
alan.cl
0868c23988 fix(connector): route order + frontend list URL trailing slash
Two backend integration bugs surfaced when /construct/connectors loaded:

1. GET /pending-confirms returned {Connector 'pending-confirms' not found}
   because the route was defined AFTER GET /{connector_id}, so FastAPI
   matched the catch-all first and treated 'pending-confirms' as a
   connector_id. HITL polling never reached the queue endpoint.

   Move /pending-confirms, /confirm, and ConfirmRequest BEFORE the
   /{connector_id} routes so FastAPI matches the fixed paths first.

2. GET /api/v2/serve/connectors (no trailing slash) hit DB-GPT's static
   Next-export fallback (404 HTML), because FastAPI registers the list
   route as GET / under api_prefix and the auto-redirect was intercepted.
   Frontend useConnectors() used the bare base URL.

   Add API_BASE_LIST = `${API_BASE}/` constant and use it for the list
   (GET /) and create (POST /) calls. Single-resource calls keep the
   bare base + "/{id}" form.
2026-05-27 11:31:47 +08:00
alan.cl
bfb8ba6a09 fix(connector): register ConnectorServe in register_serve_apps
The connector module had scan_serve_configs() listed it (line 32) so its
config schema was discovered, but register_serve_apps() never called
system_app.register(ConnectorServe, ...). That left the FastAPI router
unmounted, so all /api/v2/serve/connectors/* requests returned 404 and
the /construct/connectors page was non-functional.

Register the Serve component alongside the other 13 serves (file/flow/
rag/...) after Model Serve.
2026-05-27 11:24:34 +08:00
alan.cl
9041733342 fix(connector): wire frontend through ctx-axios + add sidebar entry + normalize field names
Pre-existing integration gaps surfaced when running the dev server:

1. use-connector-api.ts used bare fetch() instead of ctx-axios, so all
   /api/v2/serve/connectors/* requests hit the Next.js dev server (3000)
   instead of the backend (5670) and returned the Next 404 HTML page.
   Migrate every helper to axios via @/utils/ctx-axios.

2. useConfirmPolling.ts had the same bare-fetch problem; HITL confirm
   polling and resolve calls would never reach the backend.

3. Backend ConnectorResponse exposes connector_id while the frontend
   ConnectorInstance type uses id, leaving every c.id undefined. Add a
   normalizeConnector boundary in use-connector-api.ts so the rest of
   the frontend can keep using id.

4. /construct/connectors had no entry in the side-bar settings menu —
   users had to type the URL by hand. Add an "ApiOutlined / 连接器管理"
   item next to prompts in the management group.

5. Add the new `connectors` translation key to zh/en common.ts so the
   sidebar label resolves and TS strict-key check passes.
2026-05-27 10:51:32 +08:00
alan.cl
6add267298 feat(connector): support user-defined custom MCP servers (custom_mcp)
- Manager: branch on connector_type == 'custom_mcp', build pack from
  extra_config (server_uri + auth_type) instead of catalog. Add
  connector_id parameter for restore-from-DB id preservation.
- Service: pass extra_config and connector_id through to manager on both
  create and after_start rehydration paths. Rehydration now reads
  config_json so custom_mcp connectors survive restart.
- Frontend: align catalog type to backend (auth_fields), add select +
  default support, conditional token/header_name fields driven by
  auth_type, and submit a config payload alongside credentials for
  custom_mcp.
- Confirmation: custom_mcp tools are NOT in any catalog.confirm_actions,
  so the existing ConfirmationInterceptor naturally returns False --
  custom_mcp writes execute without confirmation (intentional MVP scope).
2026-05-26 13:56:09 +08:00
alan.cl
6eb5f1b961 feat(connector): wire HITL ConfirmDialog into chat composer
- useConfirmPolling activated when a connector is attached and Agent is streaming
- Render ConfirmDialog at bottom of Playground; backend pending-confirms now
  reach the user as a bottom-right modal
- 300s backend timeout matches dialog countdown — auto-deny on lapse
2026-05-26 13:47:39 +08:00
alan.cl
5da697034f feat(connector): multi-select connectors in chat composer
- Replace single selectedConnector with selectedConnectors array
- Send ext_info.connector_ids (array) to backend (Task C contract)
- Render one tag per selected connector with individual onClose
- Popover stays open after pick; click toggles membership
- Badge shows count; tooltip shows comma-joined names
- ChatMessage.attachedConnectors[] for replay
2026-05-26 13:43:32 +08:00
alan.cl
babd975465 feat(connector): inject only user-selected connectors per request
- Read ext_info.connector_ids (list) at start of _react_agent_stream;
  fall back to legacy ext_info.connector_id (string) for older clients
- Filter pack injection to selected ids via get_connector_tools()
- Empty/missing selection no longer leaks all active connectors into prompt
- Warn (not error) on stale ids so users can recover gracefully
2026-05-26 12:51:55 +08:00
alan.cl
beb9e6244b feat(connector): auto-prefix MCP tool names per connector instance
- compute_tool_prefix: type-based for built-ins, slug-based for custom, slug
  with type-prefix fallback for multi-instance built-ins, id-prefix fallback
- _apply_tool_prefix: rename tools in pack.sub_resources, sync tool_server_map,
  annotate description with "(via {display_name})"
- Wire into create_connector after preload_resource
2026-05-26 11:58:41 +08:00
alan.cl
3875b900ec feat(connector): wire /types endpoint to catalog with custom_mcp option
- Add ConnectorManager.get_catalog() public accessor
- Replace hardcoded /types response with catalog-driven ConnectorTypeOption list
- Append custom_mcp placeholder for future user-defined MCP servers
2026-05-26 11:31:09 +08:00
Honglei
5dc365682a fix(rag): use size chunking by default for markdown knowledge (Fixes #3030) (#3033)
Co-authored-by: honglei <honglei@users.noreply.github.com>
2026-05-25 23:46:57 +08:00
Jaeyoung Yun
b855cd0d54 feat: Harden ~/.dbgpt/configs/<profile>.toml file permissions to 0o600 (#3077)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:44:58 +08:00
Cc
3a07e8dc33 fix: restrict personal skill script execution (#3071)
Co-authored-by: wolfkill <27876473+wolfkill@users.noreply.github.com>
2026-05-21 19:57:01 +08:00
Cc
79ea62cd4e fix: validate example file names (#3066)
Co-authored-by: wolfkill <27876473+wolfkill@users.noreply.github.com>
2026-05-21 19:17:44 +08:00
Cc
aecad1a9c0 fix: validate skill upload filenames (#3065)
Co-authored-by: wolfkill <27876473+wolfkill@users.noreply.github.com>
2026-05-21 19:17:22 +08:00
Cc
51c4e8cdaa fix: handle knowledge space id responses (#3070)
Co-authored-by: wolfkill <27876473+wolfkill@users.noreply.github.com>
2026-05-21 19:16:57 +08:00
Cc
72639e849b fix: constrain python upload filenames (#3064)
Co-authored-by: wolfkill <27876473+wolfkill@users.noreply.github.com>
2026-05-21 19:16:22 +08:00
Artoriasn
cc47cf0cc9 fix(agent): tolerate multi-step ReAct output (#3074) 2026-05-21 09:57:16 +08:00
Aries-ckt
9470ea03ff feat(agent):adds context management, task-plan tracking, and the corresponding frontend UI. (#3053) 2026-05-18 11:17:04 +08:00
Aalron
d6e1581dcd feat: Implement VARBINARY and BINARY types for StarRocks (#3062) 2026-05-17 23:13:13 +08:00
hankchorome
fa1a8d2f70 fix(chat_db_qa): tell LLM the table list is a TOP-K subset, not the full DB (#3045)
Co-authored-by: hankchrome <hankchrome@github.com>
2026-05-13 11:42:39 +08:00
alan.cl
0c22d4780d feat: dbgpt support connector 2026-05-13 10:01:24 +08:00
Aalron
8a4940ded8 fix: fix the insufficient length of the content column in the gpts_messages table (#3055) 2026-05-13 09:51:50 +08:00
Daria Korenieva
d60c91c752 feat(ext): add Valkey cache integration (#3057)
Signed-off-by: Daria Korenieva <daric2612@gmail.com>
2026-05-12 20:34:43 +08:00
Mohamed Arbi
c1f9c63af6 feat(qdrant): make distance metric configurable (#3044) 2026-05-10 16:05:45 +08:00
Daria Korenieva
ca126edfab feat(ext): add Valkey vector store integration (#3051)
Signed-off-by: Daria Korenieva <daric2612@gmail.com>
2026-05-09 19:13:52 +08:00
hankchorome
22e87ece6e fix(milvus): make MilvusStore compatible with milvus 2.5+ (#3042)
Co-authored-by: hankchrome <hankchrome@users.noreply.github.com>
Co-authored-by: hankchrome <hankchrome@github.com>
2026-05-09 13:33:14 +08:00
hankchorome
c78fa1a10f fix(storage,datasource): per-chunk error tolerance + per-db indexing lock (#3040)
Co-authored-by: hankchrome <hankchrome@users.noreply.github.com>
Co-authored-by: hankchrome <hankchrome@github.com>
2026-05-05 22:31:56 +08:00
Aarish Alam
000a653d91 feat: add LiteLLM as an embedded AI gateway provider (#3043) 2026-05-05 22:30:45 +08:00
hankchorome
8ca27d21f9 perf(datasource): cache ConnectorManager.get_connector with TTL (~63s -> <10ms on warm cache) (#3046)
Co-authored-by: hankchrome <hankchrome@github.com>
2026-05-05 20:36:28 +08:00
hankchorome
71784ce226 fix(datasource): override get_fields() for MSSQL with SQL Server-correct INFORMATION_SCHEMA (#3039)
Co-authored-by: hankchrome <hankchrome@users.noreply.github.com>
2026-04-30 22:34:38 +08:00
github0070070
e46db3053b fix: add authentication dependency to knowledge module endpoints (#3038) 2026-04-29 13:40:26 +08:00
Octopus
9ed8671f4c fix(awel): BranchOperator incorrectly skips shared downstream nodes (fixes #2935) (#3035)
Co-authored-by: octo-patch <octo-patch@github.com>
2026-04-26 20:37:20 +08:00
Anush
e6abfa26aa feat: Adds support for vector search with Qdrant (#3034) 2026-04-25 11:26:11 +08:00
Honglei
83d306e8ad fix(rag): honor configured Tongyi embedding model name (Fixes #3029) (#3032)
Co-authored-by: honglei <honglei@users.noreply.github.com>
2026-04-22 18:53:58 +08:00
Octopus
116388f931 fix: use explicit UTF-8 encoding when writing code_interpreter temp script (#3023)
Co-authored-by: octo-patch <octo-patch@github.com>
2026-04-20 21:43:47 +08:00
alan.cl
0f133f20eb docs: update external connectors design for SSE-only transport, connector-scoped confirmation, and scheduled task auto-confirm 2026-04-09 15:05:33 +08:00
yanzhicong
b4b0a53d1b feat: add openGauss datasource support (#3007) 2026-04-03 17:18:54 +08:00