HELSON 63fbba3c19 [zero] add L2 gradient clipping for ZeRO (#2112) ...

* [zero] add L2 gradient clipping

* [testing] add MlpModel

* [zero] add unit test for grad clipping

* fix atol

2022-12-09 18:09:17 +08:00

..

test_chunk_mgrv2.py

[feature] A new ZeRO implementation (#1644)

2022-10-09 09:18:51 +08:00

test_chunkv2.py

[hotfix] fix zero's incompatibility with checkpoint in torch-1.12 (#1786)

2022-11-02 16:11:34 +08:00

test_fwd_bwd.py

[Gemini] remove eval in gemini unittests! (#2092)

2022-12-07 11:58:37 +08:00

test_gemini_use_rmt.py

[Gemini] gemini use the runtime memory tracer (RMT) (#2099)

2022-12-07 23:04:02 +08:00

test_grad_clip.py

[zero] add L2 gradient clipping for ZeRO (#2112)

2022-12-09 18:09:17 +08:00

test_optim.py

[zero] add L2 gradient clipping for ZeRO (#2112)

2022-12-09 18:09:17 +08:00

test_search.py

[zero] add chunk init function for users (#1729)

2022-10-18 16:31:22 +08:00

test_zeroddp_state_dict.py

[Gemini] more tests for Gemini (#2038)

2022-11-29 17:13:10 +08:00

test_zerooptim_state_dict.py

[Gemini] add unitests to check gemini correctness (#2015)

2022-11-24 16:51:45 +08:00