Steve Luo 5cd75ce4c7 [Inference/Kernel] refactor kvcache manager and rotary_embedding and kvcache_memcpy oper… (#5663) ...

* refactor kvcache manager and rotary_embedding and kvcache_memcpy operator

* refactor decode_kv_cache_memcpy

* enable alibi in pagedattention

2024-04-30 15:52:23 +08:00

..

common

[Inference/Feat] Add kvcache quantization support for FlashDecoding (#5656)

2024-04-26 19:40:37 +08:00

funcs

[Inference/Feat] Feat quant kvcache step2 (#5674)

2024-04-30 11:26:36 +08:00

kernel

[Inference/Kernel] refactor kvcache manager and rotary_embedding and kvcache_memcpy oper… (#5663)

2024-04-30 15:52:23 +08:00

__init__.py

[Inference/Refactor] Delete Duplicated code and refactor vec_copy utils and reduce utils (#5593)

2024-04-15 10:57:51 +08:00