Yuanheng Zhao 6e487e7d3c [kernel/fix] Performance Optimization for Decoding Kernel and Benchmarking (#5274) ...

* prevent re-creating intermediate tensors

* add singleton class holding intermediate values

* fix triton kernel api

* add benchmark in pytest

* fix kernel api and add benchmark

* revise flash decoding triton kernel in/out shapes

* fix calling of triton kernel in modeling

* fix pytest: extract to util functions

2024-01-19 15:47:16 +08:00

..

__init__.py

fix bugs in request_handler

2024-01-11 13:39:56 +00:00

llama.py

[kernel/fix] Performance Optimization for Decoding Kernel and Benchmarking (#5274)

2024-01-19 15:47:16 +08:00