github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-09 04:50:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

[Inference]Fused the gate and up proj in mlp，and optimized the autograd process. (#5365)

Browse Source 
* fused the gate and up proj in mlp

* fix code styles

* opt auto_grad

* rollback test_inference_engine.py

* modifications based on the review feedback.

* fix bugs in flash attn

* Change reshape to view

* fix test_rmsnorm_triton.py
This commit is contained in:
yuehuayingxueluo
2024-02-06 19:38:25 +08:00
committed by GitHub
parent 1dedb57747
commit 35382a7fbf
10 changed files with 484 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
colossalai/kernel/triton/no_pad_rotary_embedding.py
View File

@@ -274,7 +274,6 @@ def fused_rotary_embedding_kernel(
)
@torch.no_grad()
def rotary_embedding(
q: torch.Tensor,
k: torch.Tensor,