Baizhou Zhang c9625dbb63 [shardformer] support sharded optimizer checkpointIO of HybridParallelPlugin (#4540) ... * implement sharded optimizer saving * add more param info * finish implementation of sharded optimizer saving * fix bugs in optimizer sharded saving * add pp+zero test * param group loading * greedy loading of optimizer * fix bug when loading * implement optimizer sharded saving * add optimizer test & arrange checkpointIO utils * fix gemini sharding state_dict * add verbose option * add loading of master params * fix typehint * fix master/working mapping in fp16 amp		2023-08-31 14:50:47 +08:00
..
__init__.py	[shardformer] support sharded checkpoint IO for models of HybridParallelPlugin (#4506)	2023-08-25 22:04:57 +08:00
checkpoint_io_base.py	Next commit [checkpointio] Unsharded Optimizer Checkpoint for Gemini Plugin (#4141)	2023-07-07 16:33:06 +08:00
general_checkpoint_io.py	[checkpointio] Sharded Optimizer Checkpoint for Gemini Plugin (#4302)	2023-07-21 14:39:01 +08:00
hybrid_parallel_checkpoint_io.py	[shardformer] support sharded optimizer checkpointIO of HybridParallelPlugin (#4540)	2023-08-31 14:50:47 +08:00
index_file.py	[checkpointio] General Checkpointing of Sharded Optimizers (#3984)	2023-06-15 15:21:26 +08:00
utils.py	[shardformer] support sharded optimizer checkpointIO of HybridParallelPlugin (#4540)	2023-08-31 14:50:47 +08:00