github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-01-20 17:44:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,559 Commits 60 Branches 51 Tags
e94c79f15b2ee1428ccda2cdb2d00007697c578c
Commit Graph

3 Commits

Author SHA1 Message Date
HELSON
e7d3afc9cc [optimizer] add div_scale for optimizers (#2117) 
* [optimizer] add div_scale for optimizers

* [zero] use div_scale in zero optimizer

* fix testing error
 2022-12-12 17:58:57 +08:00
HELSON
63fbba3c19 [zero] add L2 gradient clipping for ZeRO (#2112) 
* [zero] add L2 gradient clipping

* [testing] add MlpModel

* [zero] add unit test for grad clipping

* fix atol
 2022-12-09 18:09:17 +08:00
Jiarui Fang
f7e276fa71 [Gemini] add GeminiAdamOptimizer (#1960) 2022-11-16 14:44:28 +08:00