ver217 ae71036cd2 [utils] refactor parallel layers checkpoint and bcast model on loading checkpoint (#1548) ...

* refactor parallel layer

* broadcast rank0 model after load ckpt

2022-09-06 20:18:35 +08:00

..

colossalai_layer

[TP] allow layernorm without bias (#750)

2022-04-14 11:43:56 +08:00

moe

[gemini] add GeminiMemoryManger (#832)

2022-04-24 13:08:48 +08:00

parallel_1d

[utils] refactor parallel layers checkpoint and bcast model on loading checkpoint (#1548)

2022-09-06 20:18:35 +08:00

parallel_2d

[utils] refactor parallel layers checkpoint and bcast model on loading checkpoint (#1548)

2022-09-06 20:18:35 +08:00

parallel_2p5d

[utils] refactor parallel layers checkpoint and bcast model on loading checkpoint (#1548)

2022-09-06 20:18:35 +08:00

parallel_3d

[utils] refactor parallel layers checkpoint and bcast model on loading checkpoint (#1548)

2022-09-06 20:18:35 +08:00

parallel_sequence

[NFC] polish colossalai/nn/layer/parallel_sequence/layers.py code style (#1280)

2022-07-13 12:08:21 +08:00

utils

[NFC] polish colossalai/nn/layer/utils/common.py code style (#983)

2022-05-17 10:25:06 +08:00

vanilla

[NFC] polish colossalai/nn/layer/vanilla/layers.py code style (#1295)

2022-07-13 12:08:21 +08:00

wrapper

[NFC] polish colossalai/nn/layer/wrapper/pipeline_wrapper.py code style (#1303)

2022-07-13 19:01:07 +08:00

__init__.py

[MOE] changed parallelmode to dist process group (#460)

2022-03-19 13:46:29 +08:00

base_layer.py

[utils] refactor parallel layers checkpoint and bcast model on loading checkpoint (#1548)

2022-09-06 20:18:35 +08:00