github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-12 20:54:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,853 Commits 59 Branches 51 Tags
grpo-dev
Commit Graph

2 Commits

Author SHA1 Message Date
LuGY
839847b7d7 [zero]support zero2 with gradient accumulation (#4511) 
* support gradient accumulation with zero2

* fix type
 2023-08-25 13:44:07 +08:00
LuGY
1a49a5ea00 [zero] support shard optimizer state dict of zero (#4194) 
* support shard optimizer of zero

* polish code

* support sync grad manually
 2023-07-31 22:13:29 +08:00