HELSON 63fbba3c19 [zero] add L2 gradient clipping for ZeRO (#2112) ...

* [zero] add L2 gradient clipping

* [testing] add MlpModel

* [zero] add unit test for grad clipping

* fix atol

2022-12-09 18:09:17 +08:00

..

chunk

[zero] add L2 gradient clipping for ZeRO (#2112)

2022-12-09 18:09:17 +08:00

memory_tracer

[gemini] get the param visited order during runtime (#2108)

2022-12-09 16:13:03 +08:00

ophooks

[gemini] get the param visited order during runtime (#2108)

2022-12-09 16:13:03 +08:00

paramhooks

[hotfix] remove potiential circle import (#1307)

2022-07-14 13:44:26 +08:00

__init__.py

[hotfix] polish chunk import (#1787)

2022-11-02 12:10:52 +08:00

gemini_context.py

[hotfix] add deconstructor for stateful tensor (#848)

2022-04-24 15:03:04 +08:00

gemini_mgr.py

[Gemini] gemini use the runtime memory tracer (RMT) (#2099)

2022-12-07 23:04:02 +08:00

placement_policy.py

[Gemini] polish memstats collector (#1962)

2022-11-16 15:45:57 +08:00

stateful_tensor_mgr.py

[gemini] accelerate adjust_layout() (#878)

2022-04-26 18:08:31 +08:00

stateful_tensor.py

[hotfix] add deconstructor for stateful tensor (#848)

2022-04-24 15:03:04 +08:00

tensor_placement_policy.py

[gemini] accelerate adjust_layout() (#878)

2022-04-26 18:08:31 +08:00

tensor_utils.py

[Gemini] free and allocate cuda memory by tensor.storage, add grad hook (#2040)

2022-11-30 15:57:45 +08:00