github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-07-12 02:38:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
8ccb6714e79137c8e6e50d9a585eadbf70ae6fc0
ColossalAI/examples/inference
History
Yuanheng Zhao 5be590b99e [kernel] Support new KCache Layout - Context Attention Triton Kernel (#5658) 
* add context attn triton kernel - new kcache layout

* add benchmark triton

* tiny revise

* trivial - code style, comment
2024-04-26 17:51:49 +08:00
..
benchmark_ops
[kernel] Support new KCache Layout - Context Attention Triton Kernel (#5658)
2024-04-26 17:51:49 +08:00
benchmark_llama3.py
[Fix/Inference]Fix vllm benchmark (#5630)
2024-04-24 14:51:36 +08:00
benchmark_llama.py
[Fix/Inference]Fix vllm benchmark (#5630)
2024-04-24 14:51:36 +08:00
llama_generation.py
[example] Update Llama Inference example (#5629)
2024-04-23 22:23:07 +08:00
run_benchmark.sh
[Fix/Inference]Fix vllm benchmark (#5630)
2024-04-24 14:51:36 +08:00