Baizhou Zhang c9625dbb63 [shardformer] support sharded optimizer checkpointIO of HybridParallelPlugin (#4540) ...

* implement sharded optimizer saving

* add more param info

* finish implementation of sharded optimizer saving

* fix bugs in optimizer sharded saving

* add pp+zero test

* param group loading

* greedy loading of optimizer

* fix bug when loading

* implement optimizer sharded saving

* add optimizer test & arrange checkpointIO utils

* fix gemini sharding state_dict

* add verbose option

* add loading of master params

* fix typehint

* fix master/working mapping in fp16 amp

2023-08-31 14:50:47 +08:00

..

gemini

[shardformer] support sharded optimizer checkpointIO of HybridParallelPlugin (#4540)

2023-08-31 14:50:47 +08:00

legacy

[nfc] fix typo colossalai/zero (#3923)

2023-06-08 00:01:29 +08:00

low_level

[shardformer] support pp+tp+zero1 tests (#4531)

2023-08-30 21:29:18 +08:00

__init__.py

[zero] reorganize zero/gemini folder structure (#3424)

2023-04-04 13:48:16 +08:00

wrapper.py

[doc] Fix typo under colossalai and doc(#3618)

2023-04-26 11:38:43 +08:00