Wenhao Chen 724441279b [moe]: fix ep/tp tests, add hierarchical all2all (#4982) ...

* fix: add warning for EP different behavior

* fix: use shard_data in ep & tp model

* to: add used_capacity

* fix: fix router test

* feat: add create_ep_node_group

* feat: add create_ep_hierarchical_group fn

* feat: add HierarchicalAllToAll

* test: add hierarchical all2all test

* fix: fix test errors

* fix: simplify create_ep_hierarchical_group

* fix: add hierarchical_alltoall arg

* fix: fix environ typo

* revert: revert process mesh order

* to: add todo mark

* fix: skip hierarchical_comm if torch < 1.13.1

2023-11-09 06:31:00 +00:00

..

bert

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

commons

[example] make gpt example directory more clear (#2353)

2023-01-06 11:11:26 +08:00

gpt

[bug] fix get_default_parser in examples (#4764)

2023-09-21 10:42:25 +08:00

llama2

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

openmoe

[moe]: fix ep/tp tests, add hierarchical all2all (#4982)

2023-11-09 06:31:00 +00:00

opt

[bug] fix get_default_parser in examples (#4764)

2023-09-21 10:42:25 +08:00

palm

[nfc] fix minor typo in README (#4846)

2023-10-07 17:51:11 +08:00