github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-05-03 09:23:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
5be590b99eb6c58c3aa809d453680139fdd2b9f7
ColossalAI/examples/inference
History
Yuanheng Zhao 5be590b99e [kernel] Support new KCache Layout - Context Attention Triton Kernel (#5658) 
* add context attn triton kernel - new kcache layout

* add benchmark triton

* tiny revise

* trivial - code style, comment
2024-04-26 17:51:49 +08:00
..
benchmark_ops
[kernel] Support new KCache Layout - Context Attention Triton Kernel (#5658)
2024-04-26 17:51:49 +08:00
benchmark_llama3.py
[Fix/Inference]Fix vllm benchmark (#5630)
2024-04-24 14:51:36 +08:00
benchmark_llama.py
[Fix/Inference]Fix vllm benchmark (#5630)
2024-04-24 14:51:36 +08:00
llama_generation.py
[example] Update Llama Inference example (#5629)
2024-04-23 22:23:07 +08:00
run_benchmark.sh
[Fix/Inference]Fix vllm benchmark (#5630)
2024-04-24 14:51:36 +08:00