LuGY c6ab96983a [zero] refactor low level zero for shard evenly (#4030) ...

* refactor low level zero

* fix zero2 and support cpu offload

* avg gradient and modify unit test

* refactor grad store, support layer drop

* refactor bucket store, support grad accumulation

* fix and update unit test of zero and ddp

* compatible with tp, ga and unit test

* fix memory leak and polish

* add zero layer drop unittest

* polish code

* fix import err in unit test

* support diffenert comm dtype, modify docstring style

* polish code

* test padding and fix

* fix unit test of low level zero

* fix pad recording in bucket store

* support some models

* polish

2023-07-31 22:13:29 +08:00

..

test_mixed_precision

[shardformer] adapted T5 and LLaMa test to use kit (#4049)

2023-07-04 16:05:01 +08:00

test_plugin

[zero] refactor low level zero for shard evenly (#4030)

2023-07-31 22:13:29 +08:00

test_accelerator.py

[CI] fix typo with tests/ etc. (#3727)

2023-05-11 16:30:58 +08:00