github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-06 19:40:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

[shardformer] supported bloom model (#4098)

Browse Source
This commit is contained in:
Frank Lee
2023-06-28 15:04:35 +08:00
parent 8af29ee47a
commit b1c2901530
20 changed files with 724 additions and 154 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tests/kit/model_zoo/transformers/gpt.py
View File

@@ -6,8 +6,6 @@ from ..registry import ModelAttribute, model_zoo
# ===============================
# Register single-sentence GPT
# ===============================
BATCH_SIZE = 1 # it can only be 1 as GPT cannot handle batch sizes > 1 if no padding token is defined.
SEQ_LENGTH = 16
def data_gen():