ver217 9492a561c3 [tensor] ColoTensor supports ZeRo (#1015) ...

* impl chunk manager

* impl param op hook

* add reduce_chunk

* add zero hook v2

* add zero dp

* fix TensorInfo

* impl load balancing when using zero without chunk

* fix zero hook

* polish chunk

* fix bugs

* ddp ok

* zero ok

* polish code

* fix bugs about load balancing

* polish code

* polish code

* add ene-to-end test

* polish code

* polish code

* polish code

* fix typo

* add test_chunk

* fix bugs

* fix bugs

* polish code

2022-05-31 12:00:12 +08:00

..

layer

[NFC] polish colossalai/nn/layer/utils/common.py code style (#983)

2022-05-17 10:25:06 +08:00

loss

[hotfix] Raise messages for indivisible batch sizes with tensor parallelism (#622)

2022-04-02 16:12:04 +08:00

lr_scheduler

Refactored docstring to google style

2022-03-29 17:17:47 +08:00

metric

[hotfix] Raise messages for indivisible batch sizes with tensor parallelism (#622)

2022-04-02 16:12:04 +08:00

model

Develop/experiments (#59)

2021-12-09 15:08:29 +08:00

optimizer

[zero] improve adaptability for not-shard parameters (#708)

2022-04-11 13:38:51 +08:00

__init__.py

Layer integration (#83)

2021-12-27 15:04:32 +08:00

init.py

Refactored docstring to google style

2022-03-29 17:17:47 +08:00

parallel.py

[tensor] ColoTensor supports ZeRo (#1015)

2022-05-31 12:00:12 +08:00