github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-06 19:40:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

[Inference] First PR for rebuild colossal-infer (#5143)

Browse Source 
* add engine and scheduler

* add dirs

---------

Co-authored-by: CjhHa1 <cjh18671720497outlook.com>
This commit is contained in:
Jianghai
2023-12-01 17:02:44 +08:00
committed by FrankLeeeee
parent c174c4fc5f
commit 4cf4682e70
33 changed files with 86 additions and 5148 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
colossalai/inference/__init__.py
View File

@@ -1,4 +0,0 @@
from .engine import InferenceEngine
from .engine.policies import BloomModelInferPolicy, ChatGLM2InferPolicy, LlamaModelInferPolicy
__all__ = ["InferenceEngine", "LlamaModelInferPolicy", "BloomModelInferPolicy", "ChatGLM2InferPolicy"]