Frank Lee e2089c5c15 adapted for sequence parallel (#163)

2022-01-20 13:44:51 +08:00

..

__init__.py

adapted for sequence parallel (#163)

2022-01-20 13:44:51 +08:00

_base_gradient_handler.py

Migrated project

2021-10-28 18:21:23 +02:00

_data_parallel_gradient_handler.py

add interleaved pipeline, fix naive amp and update pipeline model initializer (#80)

2021-12-20 23:26:19 +08:00

_moe_gradient_handler.py

Added MoE parallel (#127)

2022-01-07 15:08:36 +08:00

_pipeline_parallel_gradient_handler.py

Optimize pipeline schedule (#94)

2021-12-30 15:56:46 +08:00

_sequence_parallel_gradient_handler.py

adapted for sequence parallel (#163)

2022-01-20 13:44:51 +08:00

_zero_gradient_handler.py

Migrated project

2021-10-28 18:21:23 +02:00