github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-05-07 16:08:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
c6ab96983a
ColossalAI/tests
History
LuGY c6ab96983a [zero] refactor low level zero for shard evenly () 
* refactor low level zero

* fix zero2 and support cpu offload

* avg gradient and modify unit test

* refactor grad store, support layer drop

* refactor bucket store, support grad accumulation

* fix and update unit test of zero and ddp

* compatible with tp, ga and unit test

* fix memory leak and polish

* add zero layer drop unittest

* polish code

* fix import err in unit test

* support diffenert comm dtype, modify docstring style

* polish code

* test padding and fix

* fix unit test of low level zero

* fix pad recording in bucket store

* support some models

* polish
2023-07-31 22:13:29 +08:00
..
components_to_test [CI] fix typo with tests/ etc. () 2023-05-11 16:30:58 +08:00
kit [shardformer] added embedding gradient check () 2023-07-04 16:05:01 +08:00
test_amp [test] refactor tests with spawn () 2023-04-06 14:51:35 +08:00
test_analyzer [devops] update torch version of CI () 2023-05-15 17:20:56 +08:00
test_auto_parallel [gemini] fix argument naming during chunk configuration searching 2023-06-25 13:34:15 +08:00
test_autochunk [test] fixed tests failed due to dtensor change () 2023-07-04 16:05:01 +08:00
test_booster [zero] refactor low level zero for shard evenly () 2023-07-31 22:13:29 +08:00
test_checkpoint_io [checkpointio] Sharded Optimizer Checkpoint for Gemini Plugin () 2023-07-21 14:39:01 +08:00
test_cluster [devops] update torch version of CI () 2023-05-15 17:20:56 +08:00
test_comm [test] refactor tests with spawn () 2023-04-06 14:51:35 +08:00
test_config [pipeline] refactor the pipeline module () 2022-06-10 11:27:38 +08:00
test_context [test] refactor tests with spawn () 2023-04-06 14:51:35 +08:00
test_data [test] refactor tests with spawn () 2023-04-06 14:51:35 +08:00
test_data_pipeline_tensor_parallel [CI] fix typo with tests/ etc. () 2023-05-11 16:30:58 +08:00
test_ddp [test] refactor tests with spawn () 2023-04-06 14:51:35 +08:00
test_device [format] applied code formatting on changed files in pull request 4152 () 2023-07-04 16:07:47 +08:00
test_engine [test] refactor tests with spawn () 2023-04-06 14:51:35 +08:00
test_fx [shardformer] shardformer support opt models () 2023-07-04 16:05:01 +08:00
test_kernels [Kernels] added triton-implemented of self attention for colossal-ai () 2023-07-18 23:53:38 +08:00
test_layers [CI] fix typo with tests/ etc. () 2023-05-11 16:30:58 +08:00
test_lazy [lazy] support init on cuda () 2023-07-19 16:43:01 +08:00
test_moe [CI] fix typo with tests/ etc. () 2023-05-11 16:30:58 +08:00
test_ops [test] refactor tests with spawn () 2023-04-06 14:51:35 +08:00
test_optimizer [bf16] add bf16 support () 2023-06-05 15:58:31 +08:00
test_pipeline [test] refactor tests with spawn () 2023-04-06 14:51:35 +08:00
test_shardformer [format] applied code formatting on changed files in pull request 4152 () 2023-07-04 16:07:47 +08:00
test_tensor [test] fixed tests failed due to dtensor change () 2023-07-04 16:05:01 +08:00
test_trainer [CI] fix typo with tests/ etc. () 2023-05-11 16:30:58 +08:00
test_utils [lazy] refactor lazy init () 2023-06-05 14:20:47 +08:00
test_zero [zero] refactor low level zero for shard evenly () 2023-07-31 22:13:29 +08:00
__init__.py [zero] Update sharded model v2 using sharded param v2 () 2022-03-11 15:50:28 +08:00