github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-01 01:06:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

[pipeline] support fp32 for HybridPlugin/merge shardformer test and pipeline test into one file (#4354)

Browse Source 
* add naive optimizer for 3DPlugin/refactor gpt2 shardformer test

* merge tests of PP/DP/TP combinations into one test file

* fix bug when sync grad for dp in HybridPlugin

* update supported precisions for 3DPlugin/fix bug when shifting tp_degree

* improve the passing of lazy_init

* modify lazy_init/use sync_shared_params
This commit is contained in:
Baizhou Zhang
2023-08-01 17:29:09 +08:00
committed by Hongxin Liu
parent f13954cd58
commit 0ceec8f9a9
8 changed files with 187 additions and 142 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
tests/kit/model_zoo/transformers/gpt.py
View File

@@ -70,7 +70,8 @@ config = transformers.GPT2Config(n_layer=2,
resid_pdrop=0,
summary_first_dropout=0,
hidden_dropout=0,
problem_type="single_label_classification")
problem_type="single_label_classification",
pad_token_id=50256)
# register the following models
model_zoo.register(name='transformers_gpt',