github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-28 21:17:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,863 Commits 59 Branches 51 Tags
077a5cdde409cc89b726240b4788717fba1e62c4
Commit Graph

5 Commits

Author SHA1 Message Date
HELSON
077a5cdde4 [zero] fix gradient clipping in hybrid parallelism (#2521) 
* [zero] fix gradient clipping in hybrid parallelism

* [testing] change model name to avoid pytest warning

* [hotfix] fix unit testing
 2023-01-29 15:09:57 +08:00
HELSON
a5dc4253c6 [zero] polish low level optimizer (#2473) 2023-01-13 14:56:17 +08:00
Jiarui Fang
867c8c2d3a [zero] low level optim supports ProcessGroup (#2464) 2023-01-13 10:05:58 +08:00
HELSON
a1ce02d740 [zero] test gradient accumulation (#1964) 
* [zero] fix memory leak for zero2

* [zero] test gradient accumulation

* [zero] remove grad clip test
 2022-11-29 13:00:30 +08:00
HELSON
6e51d296f0 [zero] migrate zero1&2 (#1878) 
* add zero1&2 optimizer

* rename test ditectory

* rename test files

* change tolerance in test
 2022-11-11 09:26:40 +08:00