yuehuayingxueluo 2a718c8be8 Optimized the execution interval time between cuda kernels caused by view and memcopy (#5390) ...

* opt_view_and_memcopy

* fix bugs in ci

* fix ci bugs

* update benchmark scripts

* fix ci bugs

2024-02-21 13:23:57 +08:00

..

__init__.py

[Inference]Fused kv copy into rotary calculation (#5383)

2024-02-21 11:31:48 +08:00

context_attn_unpad.py

Optimized the execution interval time between cuda kernels caused by view and memcopy (#5390)

2024-02-21 13:23:57 +08:00

custom_autotune.py

add autotune (#4822)

2023-09-28 13:47:35 +08:00

flash_decoding.py

Optimized the execution interval time between cuda kernels caused by view and memcopy (#5390)

2024-02-21 13:23:57 +08:00

fused_rotary_embedding.py

[Inference]Fused the gate and up proj in mlp，and optimized the autograd process. (#5365)

2024-02-06 19:38:25 +08:00

gptq_triton.py

[inference] add reference and fix some bugs (#4937)

2023-10-20 13:39:34 +08:00

kvcache_copy.py

[Inference]Fused kv copy into rotary calculation (#5383)

2024-02-21 11:31:48 +08:00

llama_act_combine_kernel.py

[moe] merge moe into main (#4978)

2023-11-02 02:21:24 +00:00

no_pad_rotary_embedding.py

[Inference]Fused kv copy into rotary calculation (#5383)

2024-02-21 11:31:48 +08:00

qkv_matmul_kernel.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

rms_layernorm.py

Optimized the execution interval time between cuda kernels caused by view and memcopy (#5390)

2024-02-21 13:23:57 +08:00

rotary_cache_copy.py

[Inference]Fused the gate and up proj in mlp，and optimized the autograd process. (#5365)

2024-02-06 19:38:25 +08:00

softmax.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00