mirror of
https://github.com/hpcaitech/ColossalAI.git
synced 2026-01-06 08:04:16 +00:00
17cfa57140
* add bloom inference methods and policy * enable pass BatchInferState from model forward * revise bloom infer layers/policies * add engine for inference (draft) * add test for bloom infer * fix bloom infer policy and flow * revise bloom test * fix bloom file path * remove unused codes * fix bloom modeling * fix dir typo * fix trivial * fix policy * clean pr * trivial fix
5 lines
123 B
Python
5 lines
123 B
Python
from .engine import TPInferEngine
|
|
from .kvcache_manager import MemoryManager
|
|
|
|
__all__ = ['MemoryManager', 'TPInferEngine']
|