Baizhou Zhang c0a033700c [shardformer] fix master param sync for hybrid plugin/rewrite unwrapping logic (#4758) ...

* fix master param sync for hybrid plugin

* rewrite unwrap for ddp/fsdp

* rewrite unwrap for zero/gemini

* rewrite unwrap for hybrid plugin

* fix geemini unwrap

* fix bugs

2023-09-20 18:29:37 +08:00

..

__init__.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

dp_plugin_base.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

gemini_plugin.py

[shardformer] fix master param sync for hybrid plugin/rewrite unwrapping logic (#4758)

2023-09-20 18:29:37 +08:00

hybrid_parallel_plugin.py

[shardformer] fix master param sync for hybrid plugin/rewrite unwrapping logic (#4758)

2023-09-20 18:29:37 +08:00

low_level_zero_plugin.py

[shardformer] fix master param sync for hybrid plugin/rewrite unwrapping logic (#4758)

2023-09-20 18:29:37 +08:00

plugin_base.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

pp_plugin_base.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

torch_ddp_plugin.py

[shardformer] fix master param sync for hybrid plugin/rewrite unwrapping logic (#4758)

2023-09-20 18:29:37 +08:00

torch_fsdp_plugin.py

[shardformer] fix master param sync for hybrid plugin/rewrite unwrapping logic (#4758)

2023-09-20 18:29:37 +08:00