github/DB-GPT
1
0
Fork 0
mirror of https://github.com/csunny/DB-GPT.git synced 2025-09-24 19:08:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add quantize_qlora support for falcon

Browse Source
This commit is contained in:
zhanghy-sketchzh
2023-06-08 13:37:48 +08:00
parent b357fd9d0c
commit bb9081e00f
3 changed files with 18 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.env.template
View File

@@ -21,7 +21,7 @@ LLM_MODEL=vicuna-13b
MODEL_SERVER=http://127.0.0.1:8000
LIMIT_MODEL_CONCURRENCY=5
MAX_POSITION_EMBEDDINGS=4096
QUANTIZE_QLORA=True
## SMART_LLM_MODEL - Smart language model (Default: vicuna-13b)
## FAST_LLM_MODEL - Fast language model (Default: chatglm-6b)
# SMART_LLM_MODEL=vicuna-13b
@@ -112,4 +112,4 @@ PROXY_SERVER_URL=http://127.0.0.1:3000/proxy_address
#*******************************************************************#
# ** SUMMARY_CONFIG
#*******************************************************************#
SUMMARY_CONFIG=FAST
SUMMARY_CONFIG=FAST