Baizhou Zhang 21ba89cab6 [gemini] support gradient accumulation (#4869) ... * add test * fix no_sync bug in low level zero plugin * fix test * add argument for grad accum * add grad accum in backward hook for gemini * finish implementation, rewrite tests * fix test * skip stuck model in low level zero test * update doc * optimize communication & fix gradient checkpoint * modify doc * cleaning codes * update cpu adam fp16 case		2023-10-17 14:07:21 +08:00
..
utils	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
__init__.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
albert.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
beit.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
bert.py	[gemini] support gradient accumulation (#4869)	2023-10-17 14:07:21 +08:00
gpt2.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
hanging_param_model.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
inline_op_model.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
nested_model.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
registry.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
repeated_computed_layers.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
resnet.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
simple_net.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00