github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-10 05:20:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

Next commit [checkpointio] Unsharded Optimizer Checkpoint for Gemini Plugin (#4141)

Browse Source 
* [checkpointio] unsharded optimizer checkpoint for Gemini plugin

* [checkpointio] unsharded optimizer checkpoint for Gemini using all_gather
This commit is contained in:
Baizhou Zhang
2023-07-07 16:33:06 +08:00
committed by GitHub
parent fee32a3b78
commit 58913441a1
9 changed files with 684 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
colossalai/interface/optimizer.py
View File

@@ -119,3 +119,9 @@ class OptimizerWrapper:
"""
raise NotImplementedError(
"The method unscale_grad is only available for optimizers with mixed precision training")
def unwrap(self):
"""
Unwrap the optimizer for checkpoint saving/loading.
"""
return self.optim