github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-06-20 20:54:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
hotfix/kernel_build_before_load
ColossalAI/colossalai/inference/engine
History
Hongxin Liu 1cd7efc520
[inference] refactor examples and fix schedule (#5077) 
* [setup] refactor infer setup

* [hotfix] fix infenrece behavior on 1 1 gpu

* [exmaple] refactor inference examples
2023-11-21 10:46:03 +08:00
..
modeling [Kernels]added flash-decoidng of triton (#5063) 2023-11-20 13:58:29 +08:00
policies [inference] Refactor inference architecture (#5057) 2023-11-19 21:05:05 +08:00
__init__.py [inference] update examples and engine (#5073) 2023-11-20 19:44:52 +08:00
engine.py [inference] refactor examples and fix schedule (#5077) 2023-11-21 10:46:03 +08:00
microbatch_manager.py [inference] Refactor inference architecture (#5057) 2023-11-19 21:05:05 +08:00