github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-01-15 14:47:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
399 Commits 60 Branches 51 Tags
59bf2dc590d4100bf1d9eae0daafd04fcf7e2051
Commit Graph

2 Commits

Author SHA1 Message Date
HELSON
055fbf5be6 [zero] adapt zero for unsharded paramters (Optimizer part) (#601) 2022-04-01 20:10:47 +08:00
HELSON
e6d50ec107 [zero] adapt zero for unsharded parameters (#561) 
* support existing sharded and unsharded parameters in zero

* add unitest for moe-zero model init

* polish moe gradient handler
 2022-03-31 18:34:11 +08:00