github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-11 05:49:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

[Inference]Add Nopadding Llama Modeling (#5327)

Browse Source 
* add nopadding llama modeling

* add nopadding_llama.py

* rm unused codes

* fix bugs in test_xine_copy.py

* fix code style
This commit is contained in:
yuehuayingxueluo
2024-01-30 10:31:46 +08:00
committed by GitHub
parent c7c104cb7c
commit e8f0642f28
9 changed files with 386 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
colossalai/inference/modeling/policy/__init__.py
View File

@@ -1,7 +1,9 @@
from .llama import LlamaModelInferPolicy
from .nopadding_llama import NoPaddingLlamaModelInferPolicy
from .padding_llama import PaddingLlamaModelInferPolicy
model_policy_map = {
"llama": LlamaModelInferPolicy,
"padding_llama": PaddingLlamaModelInferPolicy,
"nopadding_llama": NoPaddingLlamaModelInferPolicy,
}
__all__ = ["LlamaModelInferPolicy", "model_polic_map"]
__all__ = ["PaddingLlamaModelInferPolicy", "NoPaddingLlamaModelInferPolicy", "model_polic_map"]