github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-08-06 02:24:31 +00:00
Code Issues Packages Projects Releases Wiki Activity
fix-setup
ColossalAI/tests/test_zero/test_gemini
History
botbw 2fc85abf43
[gemini] async grad chunk reduce (all-reduce&reduce-scatter) (#5713) 
* [gemini] async grad chunk reduce (all-reduce&reduce-scatter)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [gemini] add test

* [gemini] rename func

* [gemini] update llama benchmark

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [gemini] use tensor counter

* [gemini] change default config in GeminiPlugin and GeminiDDP

* [chore] typo

* [gemini] fix sync issue & add test cases

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-05-24 10:31:16 +08:00
..
test_chunk_mgrv2.py [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
test_chunkv2.py [gemini] async grad chunk reduce (all-reduce&reduce-scatter) (#5713) 2024-05-24 10:31:16 +08:00
test_fwd_bwd.py [gemini] async grad chunk reduce (all-reduce&reduce-scatter) (#5713) 2024-05-24 10:31:16 +08:00
test_gemini_use_rmt.py [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
test_grad_accum.py [gemini] async grad chunk reduce (all-reduce&reduce-scatter) (#5713) 2024-05-24 10:31:16 +08:00
test_grad_clip.py [gemini] async grad chunk reduce (all-reduce&reduce-scatter) (#5713) 2024-05-24 10:31:16 +08:00
test_inference.py [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
test_optim.py [gemini] async grad chunk reduce (all-reduce&reduce-scatter) (#5713) 2024-05-24 10:31:16 +08:00
test_runtime_mem_tracer.py [test] merge old components to test to model zoo (#4945) 2023-10-20 10:35:08 +08:00
test_search.py [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
test_zeroddp_state_dict.py [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
test_zerooptim_state_dict.py [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00