Jiarui Fang de0468c7a8 [zero] zero init context (#321) ...

* add zero init context

* add more flags for zero init context
fix bug of repeated converting param to ShardedParamV2

* polish code

2022-03-11 15:50:28 +08:00

..

init_ctx

[zero] zero init context (#321)

2022-03-11 15:50:28 +08:00

shard_utils

[zero] polish shard strategy (#310)

2022-03-11 15:50:28 +08:00

sharded_model

fix sharded param hook and unit test

2022-03-11 15:50:28 +08:00

sharded_optim

[zero] cpu adam kernel (#288)

2022-03-11 15:50:28 +08:00

sharded_param

[zero] zero init context (#321)

2022-03-11 15:50:28 +08:00

__init__.py

added buffer sync to naive amp model wrapper (#291)

2022-03-11 15:50:28 +08:00