github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-17 23:18:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

[Inference]Update inference config and fix test (#5178)

Browse Source 
* unify the config setting

* fix test

* fix import

* fix test

* fix

* fix

* add logger

* revise log info

---------

Co-authored-by: CjhHa1 <cjh18671720497outlook.com>
This commit is contained in:
Jianghai
2023-12-12 17:22:41 +08:00
committed by FrankLeeeee
parent 3de2e62299
commit 93aeacca34
9 changed files with 61 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tests/test_infer/test_kvcache_manager.py
View File

@@ -3,7 +3,7 @@ import random
import torch
from transformers.models.llama import LlamaConfig
from colossalai.inference.core.config import InferenceConfig
from colossalai.inference.config import InferenceConfig
from colossalai.inference.kv_cache import CacheBlock, KVCacheManager
from colossalai.logging import disable_existing_loggers
from colossalai.testing import parameterize