Yuanheng Zhao fa85e02b3b [kernel] Add KV cache copy kernel during decoding (#5261) ...

* add kv copy triton kernel during decoding stage

* add pytest and fix kernel

* fix test utilities

* revise kernel config

* add benchmark for kvcache copy

2024-01-15 17:37:20 +08:00

..

layers

[kernel] Add KV cache copy kernel during decoding (#5261)

2024-01-15 17:37:20 +08:00

models

fix bugs related to processing padding mask

2024-01-11 13:46:14 +00:00

policy

Fixed a bug in the inference frame

2024-01-11 13:39:56 +00:00