github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-08 20:40:34 +00:00
Code Issues Packages Projects Releases Wiki Activity

[Pipeline Inference] Merge pp with tp (#4993)

Browse Source 
* refactor pipeline into new CaiInferEngine

* updata llama modeling forward

* merge tp with pp

* update docstring

* optimize test workflow and example

* fix typo

* add assert and todo
This commit is contained in:
Bin Jia
2023-11-01 12:46:21 +08:00
committed by GitHub
parent 335cb105e2
commit b6696beb04
12 changed files with 268 additions and 203 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
colossalai/inference/__init__.py
View File

@@ -1,4 +1,4 @@
from .pipeline import PPInferEngine
from .hybridengine import CaiInferEngine
from .hybridengine.polices import LlamaModelInferPolicy
__all__ = ['PPInferEngine']
__all__ = ["CaiInferEngine", "LlamaModelInferPolicy"]