github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-17 23:18:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

[Inference/SpecDec] Add Basic Drafter Model Container (#5405)

Browse Source 
* [Infer/Fix] Fix Dependency in test - RMSNorm kernel (#5399)

fix dependency in pytest

* add drafter model container (basic ver)
This commit is contained in:
Yuanheng Zhao
2024-02-28 13:48:17 +08:00
committed by Yuanheng
parent d63c469f45
commit 5a9b05f7b2
4 changed files with 216 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
colossalai/inference/spec/__init__.py Normal file
View File

@@ -0,0 +1,4 @@
from .drafter import Drafter
from .struct import DrafterOutput
__all__ = ["Drafter", "DrafterOutput"]