ver217 fb841dd5c5 [zero] optimize grad offload (#539) ...

* optimize grad offload

* polish code

* polish code

2022-03-29 12:48:00 +08:00

..

init_ctx

[zero] polish ZeroInitContext (#540)

2022-03-29 09:09:04 +08:00

shard_utils

[zero] refactor model data tracing (#537)

2022-03-28 16:38:18 +08:00

sharded_model

[zero] optimize grad offload (#539)

2022-03-29 12:48:00 +08:00

sharded_optim

[zero] optimize grad offload (#539)

2022-03-29 12:48:00 +08:00

sharded_param

[zero] get memory usage of sharded optim v2. (#542)

2022-03-29 09:08:18 +08:00

__init__.py

[refactor] remove old zero code (#517)

2022-03-25 14:54:39 +08:00