[shardformer] add Dropout layer support different dropout pattern (#3856)

Browse Source

* add dropout layer, add dropout test

* modify seed manager as context manager

* add a copy of col_nn.layer

* add dist_crossentropy loss; separate module test

* polish the code

* fix dist crossentropy loss

...

This commit is contained in:

FoolPlayer

2023-06-01 16:21:02 +08:00

committed by Frank Lee

parent c594dc2f1c

commit ab8a47f830

14 changed files with 1413 additions and 41 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

0

colossalai/shardformer/layer/__init__.py Normal file