[pipeline] add chatglm (#4363)

Browse Source

* add pipeline policy and bert forward to be done

* add bertmodel pipeline forward and make tests

* add Bert_Policy and test for policy

* update formatting

* update formatting

* update the code

* fix bugs

* fix name confilt

* add bloom model and policy ,revise the base class of policy

* revise

* revision

* add bert_for_pretraining

* add bert_for_pretraining forward and policy

* fix typos

* cancel warning

* change the imediate output to default dict

* change the default output of get_shared_params

* add chatglm

* add

* chatglm

* chatglm

* finish chatglm

* deletes

* fix rmsnorm

* chatglm

* fix chatglm shard

* init

...

This commit is contained in:

Jianghai

2023-08-04 14:55:31 +08:00

committed by Hongxin Liu

parent b1feeced8e

commit a88e92251d

9 changed files with 1828 additions and 57 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

1373

colossalai/shardformer/modeling/chatglm2_6b/modeling_chatglm.py Normal file

File diff suppressed because it is too large Load Diff