littsk 83b52c56cd [feature] Add clip_grad_norm for hybrid_parallel_plugin (#4837) ...

* Add clip_grad_norm for hibrid_parallel_plugin

* polish code

* add unittests

* Move tp to a higher-level optimizer interface.

* bug fix

* polish code

2023-10-12 11:32:37 +08:00

..

components_to_test

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

kit

[test] modify model supporting part of low_level_zero plugin (including correspoding docs)

2023-10-05 15:10:31 +08:00

test_analyzer

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_auto_parallel

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_autochunk

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_booster

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_checkpoint_io

[gemini] support amp o3 for gemini (#4872)

2023-10-12 10:39:08 +08:00

test_cluster

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_config

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_device

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_fx

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_gptq

[feature] add gptq for inference (#4754)

2023-09-22 11:02:50 +08:00

test_infer

fix test llama (#4884)

2023-10-11 17:43:01 +08:00

test_infer_ops

[infer] fix test bug (#4838)

2023-10-04 10:01:03 +08:00

test_lazy

[lazy] support from_pretrained (#4801)

2023-09-26 11:04:11 +08:00

test_legacy

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_moe

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_optimizer

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_pipeline

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_shardformer

[feature] Add clip_grad_norm for hybrid_parallel_plugin (#4837)

2023-10-12 11:32:37 +08:00

test_tensor

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_utils

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_zero

[gemini] support amp o3 for gemini (#4872)

2023-10-12 10:39:08 +08:00

__init__.py

[zero] Update sharded model v2 using sharded param v2 (#323)

2022-03-11 15:50:28 +08:00