Yuanheng Zhao d63c469f45 [Infer] Revise and Adapt Triton Kernels for Spec-Dec (#5401) ...

* [Infer/Fix] Fix Dependency in test - RMSNorm kernel (#5399)

fix dependency in pytest

* resolve conflicts for revising flash-attn

* adapt kv cache copy kernel for spec-dec

* fix seqlen-n kvcache copy kernel/tests

* test kvcache copy - use torch.equal

* add assertions

* (trivial) comment out

2024-04-10 11:07:51 +08:00

..

__init__.py

[Infer] Revise and Adapt Triton Kernels for Spec-Dec (#5401)

2024-04-10 11:07:51 +08:00

context_attn_unpad.py

Optimized the execution interval time between cuda kernels caused by view and memcopy (#5390)

2024-02-21 13:23:57 +08:00

flash_decoding.py

[Infer] Revise and Adapt Triton Kernels for Spec-Dec (#5401)

2024-04-10 11:07:51 +08:00

fused_rotary_embedding.py

[Inference]Fused the gate and up proj in mlp，and optimized the autograd process. (#5365)

2024-02-06 19:38:25 +08:00

kvcache_copy.py

[Infer] Revise and Adapt Triton Kernels for Spec-Dec (#5401)

2024-04-10 11:07:51 +08:00

llama_act_combine_kernel.py

[devops] remove post commit ci (#5566)

2024-04-08 15:09:40 +08:00

no_pad_rotary_embedding.py

[Inference]Fused kv copy into rotary calculation (#5383)

2024-02-21 11:31:48 +08:00

qkv_matmul_kernel.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

rms_layernorm.py

[fix] multi graphs capture error

2024-03-11 10:49:31 +08:00

rotary_cache_copy.py

[Inference]Fused the gate and up proj in mlp，and optimized the autograd process. (#5365)

2024-02-06 19:38:25 +08:00

softmax.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00