Jiarui Fang d6abd933f2 [zero] zero init context (#321) ...

* add zero init context

* add more flags for zero init context
fix bug of repeated converting param to ShardedParamV2

* polish code

2022-03-07 16:14:40 +08:00

..

init_ctx

[zero] zero init context (#321)

2022-03-07 16:14:40 +08:00

shard_utils

[zero] polish shard strategy (#310)

2022-03-04 15:35:07 +08:00

sharded_model

fix sharded param hook and unit test

2022-03-04 15:27:39 +08:00

sharded_optim

[zero] cpu adam kernel (#288)

2022-03-04 16:05:15 +08:00

sharded_param

[zero] zero init context (#321)

2022-03-07 16:14:40 +08:00

__init__.py

added buffer sync to naive amp model wrapper (#291)

2022-03-02 16:47:17 +08:00