github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-27 20:46:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

[pipeline] refactor test pipeline and remove useless utils in pipeline (#4324)

Browse Source 
* refactor tests

* refactor bloom model

* finish policy tests

* refactor tests

* fix test pure pipeline

* remove test pipeline and cutdown launch process

* refactor tests

* refactor bloom model

* finish policy tests

* refactor tests

* fix test pure pipeline

* remove test pipeline and cutdown launch process
This commit is contained in:
Jianghai
2023-08-01 10:35:17 +08:00
committed by Hongxin Liu
parent d3c6cd66f3
commit f13954cd58
14 changed files with 138 additions and 1246 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
colossalai/shardformer/policies/bert.py
View File

@@ -315,7 +315,7 @@ class BertForMaskedLMPolicy(BertPolicy):
def module_policy(self):
policy = super().module_policy()
policy = self.add_lm_head_policy(policy)
mpolicy = self.add_lm_prediction_policy(policy)
policy = self.add_lm_prediction_policy(policy)
from transformers.models.bert.modeling_bert import BertForMaskedLM
if self.pipeline_stage_manager:
self.set_pipeline_forward(model_cls=BertForMaskedLM,