LuGY 839847b7d7 [zero]support zero2 with gradient accumulation (#4511) ...

* support gradient accumulation with zero2

* fix type

2023-08-25 13:44:07 +08:00

..

test_grad_acc.py

[zero]support zero2 with gradient accumulation (#4511)

2023-08-25 13:44:07 +08:00

test_zero1_2.py

[hotfix] fix unsafe async comm in zero (#4404)

2023-08-11 15:09:24 +08:00

test_zero_ckpt.py

[zero] add state dict for low level zero (#4179)

2023-07-31 22:13:29 +08:00

test_zero_tp.py

[gemini] improve compatibility and add static placement policy (#4479)

2023-08-24 09:29:25 +08:00