Hongxin Liu 646b3c5a90 [shardformer] fix linear 1d row and support uneven splits for fused qkv linear (#6084) ...

* [tp] hotfix linear row

* [tp] support uneven split for fused linear

* [tp] support sp for fused linear

* [tp] fix gpt2 mlp policy

* [tp] fix gather fused and add fused linear row

2024-10-10 14:34:45 +08:00

..

__init__.py

[shardformer] fix linear 1d row and support uneven splits for fused qkv linear (#6084)

2024-10-10 14:34:45 +08:00

_operation.py

[shardformer] fix linear 1d row and support uneven splits for fused qkv linear (#6084)

2024-10-10 14:34:45 +08:00

attn.py

fix

2024-09-16 13:45:04 +08:00

dropout.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

embedding.py

[fp8] support hybrid parallel plugin (#5982)

2024-08-12 18:17:05 +08:00

linear.py

[shardformer] fix linear 1d row and support uneven splits for fused qkv linear (#6084)

2024-10-10 14:34:45 +08:00

loss.py

[Feature] Split cross-entropy computation in SP (#5959)

2024-09-10 12:06:50 +08:00

normalization.py

[fp8] Merge feature/fp8_comm to main branch of Colossalai (#6016)

2024-08-22 09:21:34 +08:00

parallel_module.py

[shardformer] refactor embedding resize (#5603)

2024-04-18 16:10:18 +08:00

qkv_fused_linear.py

[shardformer] fix linear 1d row and support uneven splits for fused qkv linear (#6084)

2024-10-10 14:34:45 +08:00

utils.py

[Feature] Split cross-entropy computation in SP (#5959)

2024-09-10 12:06:50 +08:00