FoolPlayer 21a3915c98 [shardformer] add Dropout layer support different dropout pattern (#3856) ...

* add dropout layer, add dropout test

* modify seed manager as context manager

* add a copy of col_nn.layer

* add dist_crossentropy loss; separate module test

* polish the code

* fix dist crossentropy loss

2023-06-08 15:01:34 +08:00

..

colossalai_layer

fixed using zero with tp cannot access weight correctly

2023-02-28 10:52:30 +08:00

moe

[doc] Fix typo under colossalai and doc(#3618)

2023-04-26 11:38:43 +08:00

parallel_1d

[shardformer] add Dropout layer support different dropout pattern (#3856)

2023-06-08 15:01:34 +08:00

parallel_2d

[utils] refactor parallel layers checkpoint and bcast model on loading checkpoint (#1548)

2022-09-06 20:18:35 +08:00

parallel_2p5d

[utils] refactor parallel layers checkpoint and bcast model on loading checkpoint (#1548)

2022-09-06 20:18:35 +08:00

parallel_3d

improved allgather & reducescatter for 3d

2023-01-03 17:46:08 +08:00

parallel_sequence

[nfc] fix typo colossalai/nn (#3887)

2023-06-05 16:04:27 +08:00

utils

…

vanilla

added skip_bias_add for non-tp linear

2022-11-09 15:41:08 +08:00

wrapper

[NFC] polish colossalai/nn/layer/wrapper/pipeline_wrapper.py code style (#1303)

2022-07-13 19:01:07 +08:00

__init__.py

…

base_layer.py

[utils] refactor parallel layers checkpoint and bcast model on loading checkpoint (#1548)

2022-09-06 20:18:35 +08:00