YeAnbang ed43a4be04 [Distributed RLHF] Integration of PP (#6257) ...

* update help information

* update style

* fix

* minor fix

* support PP training

* add pp support

* remove unused code

* address conversation

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>

2025-04-09 13:23:24 +08:00

..

chatglm2_6b

[pre-commit.ci] pre-commit autoupdate (#5572)

2024-07-01 17:16:41 +08:00

__init__.py

…

bert.py

[fp8] support hybrid parallel plugin (#5982)

2024-08-12 18:17:05 +08:00

blip2.py

…

bloom.py

[Feature] Split cross-entropy computation in SP (#5959)

2024-09-10 12:06:50 +08:00

chatglm2.py

[Device]Support npu (#6159)

2024-12-17 15:42:39 +08:00

command.py

[Feature] Split cross-entropy computation in SP (#5959)

2024-09-10 12:06:50 +08:00

deepseek_v3.py

[shardformer] support pipeline for deepseek v3 and optimize lora save (#6188)

2025-02-14 14:48:54 +08:00

deepseek.py

[moe] add parallel strategy for shared_expert && fix test for deepseek (#6063)

2024-09-18 10:09:01 +08:00

falcon.py

[shardformer] fix modeling of bloom and falcon (#5796)

2024-06-11 17:43:50 +08:00

gpt2.py

fix

2024-10-15 13:28:33 +08:00

gptj.py

[fp8] support hybrid parallel plugin (#5982)

2024-08-12 18:17:05 +08:00

jit.py

…

llama.py

[Zerobubble] merge main. (#6142)

2024-11-19 19:00:36 +08:00

mistral.py

[Feature] Split cross-entropy computation in SP (#5959)

2024-09-10 12:06:50 +08:00

mixtral.py

[Zerobubble] merge main. (#6142)

2024-11-19 19:00:36 +08:00

opt.py

[Feature] Split cross-entropy computation in SP (#5959)

2024-09-10 12:06:50 +08:00

qwen2.py

[Distributed RLHF] Integration of PP (#6257)

2025-04-09 13:23:24 +08:00

sam.py

[shardformer]delete xformers (#5859)

2024-06-28 11:20:04 +08:00

t5.py

[shardformer] Support the T5ForTokenClassification model (#5816)

2024-06-27 16:40:38 +08:00

vit.py

…

whisper.py

[shardformer] upgrade transformers to 4.39.3 (#5815)

2024-06-14 10:59:33 +08:00