Xuanlei Zhao dc003c304c [moe] merge moe into main (#4978) ...

* update moe module
* support openmoe

2023-11-02 02:21:24 +00:00

..

__init__.py

[moe] merge moe into main (#4978)

2023-11-02 02:21:24 +00:00

_base_gradient_handler.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

_data_parallel_gradient_handler.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

_moe_gradient_handler.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

_pipeline_parallel_gradient_handler.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

_sequence_parallel_gradient_handler.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

_zero_gradient_handler.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

utils.py

[legacy] move engine to legacy (#4560)

2023-09-05 21:53:10 +08:00