Haze188 22ce873c3f [Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702) ...

* [pre-commit.ci] auto fixes from pre-commit.com hooks

* add parallel cross entropy output for falcon model & fix some typos in bloom.py

* fix module name error, self.model -> self.transformers in bloom, falcon model

* Fix the overflow bug of distributed cross entropy loss function when training with fp16

* add dtype to parallel cross entropy loss function

* fix dtype related typos adn prettify the loss.py

* fix grad dtype and update dtype mismatch error

* fix typo bugs

2024-05-21 11:07:13 +08:00

..

chatglm2_6b

[hotfix] fix typo change enabel to enable under colossalai/shardformer/ (#5317)

2024-03-05 21:48:46 +08:00

__init__.py

[shardformer] added development protocol for standardization (#4149)

2023-07-04 16:05:01 +08:00

bert.py

[shardformer] support bias_gelu_jit_fused for models (#5647)

2024-04-29 15:33:51 +08:00

blip2.py

[shardformer] support bias_gelu_jit_fused for models (#5647)

2024-04-29 15:33:51 +08:00

bloom.py

[Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702)

2024-05-21 11:07:13 +08:00

chatglm2.py

[shardformer] fix chatglm implementation (#5644)

2024-04-25 14:41:17 +08:00

falcon.py

[Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702)

2024-05-21 11:07:13 +08:00

gpt2.py

[Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702)

2024-05-21 11:07:13 +08:00

gptj.py

[shardformer] update transformers (#5583)

2024-04-24 22:51:50 +08:00

jit.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

llama.py

[Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702)

2024-05-21 11:07:13 +08:00

mistral.py

[Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702)

2024-05-21 11:07:13 +08:00

opt.py

[Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702)

2024-05-21 11:07:13 +08:00

qwen2.py

[Shardformer]fix the num_heads assert for llama model and qwen model (#5704)

2024-05-10 15:33:39 +08:00

sam.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

t5.py

[shardformer] update transformers (#5583)

2024-04-24 22:51:50 +08:00

vit.py

[shardformer] support bias_gelu_jit_fused for models (#5647)

2024-04-29 15:33:51 +08:00

whisper.py

[shardformer] update transformers (#5583)

2024-04-24 22:51:50 +08:00