github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-27 20:46:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

[pipeline] move bert related pipeline components to shardformer (#4187)

Browse Source 
* move bert related pipeline components to shardformer

* fix bugs

* revision

* fix bert model tests

* fix bert_lm_head model tests

* fix tests

* fix tests

* done checks

* skip bloom
This commit is contained in:
Jianghai
2023-07-07 15:41:00 +08:00
committed by Hongxin Liu
parent c5ea728016
commit f3bcc292c8
9 changed files with 556 additions and 65 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
colossalai/shardformer/policies/auto_policy.py
View File

@@ -29,7 +29,7 @@ _POLICY_LIST = {
"transformers.models.bert.modeling_bert.BertModel":
PolicyLocation(file_name="bert", class_name="BertModelPolicy"),
"transformers.models.bert.modeling_bert.BertForPreTraining":
PolicyLocation(file_name="bert", class_name="BertForPretrainingPolicy"),
PolicyLocation(file_name="bert", class_name="BertForPreTrainingPolicy"),
"transformers.models.bert.modeling_bert.BertLMHeadModel":
PolicyLocation(file_name="bert", class_name="BertLMHeadModelPolicy"),
"transformers.models.bert.modeling_bert.BertForMaskedLM":