github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-04-12 23:22:42 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,515 Commits 62 Branches 51 Tags
f10ce01e31342de802ff083cbd1c3a38845a02bf
Commit Graph

3 Commits

Author SHA1 Message Date
HELSON
e7d3afc9cc [optimizer] add div_scale for optimizers (#2117) 
* [optimizer] add div_scale for optimizers

* [zero] use div_scale in zero optimizer

* fix testing error
 2022-12-12 17:58:57 +08:00
HELSON
63fbba3c19 [zero] add L2 gradient clipping for ZeRO (#2112) 
* [zero] add L2 gradient clipping

* [testing] add MlpModel

* [zero] add unit test for grad clipping

* fix atol
 2022-12-09 18:09:17 +08:00
Jiarui Fang
f7e276fa71 [Gemini] add GeminiAdamOptimizer (#1960) 2022-11-16 14:44:28 +08:00