yuehuayingxueluo 600881a8ea

[Inference]Add CUDA KVCache Kernel (#5406) ...

* add cuda KVCache kernel

* annotation benchmark_kvcache_copy

* add use cuda

* fix import path

* move benchmark scripts to example/

* rm benchmark codes in test_kv_cache_memcpy.py

* rm redundancy codes

* rm redundancy codes

* pr was modified according to the review

2024-02-28 14:36:50 +08:00

0 lines

Python

Raw Permalink Blame History