flybird1111 7a3dfd0c64 [shardformer] update shardformer to use flash attention 2 (#4392) ...

* cherry-pick flash attention 2

cherry-pick flash attention 2

* [shardformer] update shardformer to use flash attention 2

[shardformer] update shardformer to use flash attention 2, fix

[shardformer] update shardformer to use flash attention 2, fix

[shardformer] update shardformer to use flash attention 2, fix

2023-08-15 23:25:14 +08:00

..

csrc

[bf16] add bf16 support (#3882)

2023-06-05 15:58:31 +08:00

mha

[coloattention] fix import error (#4380)

2023-08-04 16:28:41 +08:00

__init__.py

[shardformer] update shardformer to use flash attention 2 (#4392)

2023-08-15 23:25:14 +08:00

layer_norm.py

[kernel] fixed repeated loading of kernels (#2549)

2023-02-03 09:47:13 +08:00

multihead_attention.py

[nfc] fix typo colossalai/cli fx kernel (#3847)

2023-06-02 15:02:45 +08:00

scaled_softmax.py

[fix] coloattention support flash attention 2 (#4347)

2023-08-04 13:46:22 +08:00