Update OceanBaseStore to be compatible with oceanbase-4.3.3.0 (#2074)

Signed-off-by: shanhaikang.shk <shanhaikang.shk@oceanbase.com>
This commit is contained in:
GITHUBear 2024-10-18 16:14:37 +08:00 committed by GitHub
parent 53ba6259d2
commit c398fb64c4
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 359 additions and 686 deletions

View File

@ -232,11 +232,9 @@ class Config(metaclass=Singleton):
self.OB_USER = os.getenv("OB_USER", "root")
self.OB_PASSWORD = os.getenv("OB_PASSWORD", "")
self.OB_DATABASE = os.getenv("OB_DATABASE", "test")
self.OB_SQL_DBG_LOG_PATH = os.getenv("OB_SQL_DBG_LOG_PATH", "")
self.OB_ENABLE_NORMALIZE_VECTOR = bool(
os.getenv("OB_ENABLE_NORMALIZE_VECTOR", "")
)
self.OB_ENABLE_INDEX = bool(os.getenv("OB_ENABLE_INDEX", ""))
# QLoRA
self.QLoRA = os.getenv("QUANTIZE_QLORA", "True")

File diff suppressed because it is too large Load Diff

View File

@ -17,7 +17,6 @@ services:
- OB_PORT=2881
- OB_USER=root@test
- OB_DATABASE=test
- OB_SQL_DBG_LOG_PATH=/sql_log/sql.log
- LOCAL_DB_TYPE=sqlite
- LLM_MODEL=tongyi_proxyllm
- PROXYLLM_BACKEND=qwen-plus
@ -26,7 +25,6 @@ services:
# - TONGYI_PROXY_API_KEY={your-api-key}
- LANGUAGE=zh
# - OB_ENABLE_NORMALIZE_VECTOR=True
# - OB_ENABLE_INDEX=True
ports:
- 3306:3306
- 12345:12345

View File

@ -120,12 +120,8 @@ OB_USER=root@test
OB_DATABASE=test
## Optional
# OB_PASSWORD=
## Optional: SQL statements executed by OceanBase is recorded in the log file specified by {OB_SQL_DBG_LOG_PATH}.
# OB_SQL_DBG_LOG_PATH={your-sql-dbg-log-dir}/sql.log
## Optional: If {OB_ENABLE_NORMALIZE_VECTOR} is set, the vector stored in OceanBase is normalized.
# OB_ENABLE_NORMALIZE_VECTOR=True
## Optional: If {OB_ENABLE_INDEX} is set, OceanBase will automatically create a vector index table.
# OB_ENABLE_INDEX=True
```
</TabItem>
</Tabs>

View File

@ -23,8 +23,24 @@ If you want to change vector db, Update your .env, set your vector store type, V
If you want to use OceanBase, please first start a docker container via the following command:
```shell
docker run -p 2881:2881 --name obvec -d oceanbase/oceanbase-ce:vector
docker run --name=ob433 -e MODE=slim -p 2881:2881 -d quay.io/oceanbase/oceanbase-ce:4.3.3.0-100000142024101215
```
Donwload the partner package:
```shell
pip install --upgrade --quiet pyobvector
```
Check the connection to OceanBase and set the memory usage ratio for vector data:
```python
from pyobvector import ObVecClient
tmp_client = ObVecClient()
tmp_client.perform_raw_text_sql(
"ALTER SYSTEM ob_vector_memory_limit_percentage = 30"
)
```
Then set the following variables in the .env file:
```shell
VECTOR_STORE_TYPE=OceanBase
@ -34,12 +50,8 @@ OB_USER=root@test
OB_DATABASE=test
## Optional
# OB_PASSWORD=
## Optional: SQL statements executed by OceanBase is recorded in the log file specified by {OB_SQL_DBG_LOG_PATH}.
# OB_SQL_DBG_LOG_PATH={your-sql-dbg-log-dir}/sql.log
## Optional: If {OB_ENABLE_NORMALIZE_VECTOR} is set, the vector stored in OceanBase is normalized.
# OB_ENABLE_NORMALIZE_VECTOR=True
## Optional: If {OB_ENABLE_INDEX} is set, OceanBase will automatically create a vector index table.
# OB_ENABLE_INDEX=True
```
If you want to support more vector db, you can integrate yourself.[how to integrate](https://db-gpt.readthedocs.io/en/latest/modules/vector.html)
```commandline