github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-28 21:17:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,090 Commits 59 Branches 51 Tags
2b28b54ac6d19d33079d9117b9717fd2779f2b08
Commit Graph

2 Commits

Author SHA1 Message Date
xs_courtesy
95c21498d4 add silu_and_mul for infer 2024-03-07 16:57:49 +08:00
yuehuayingxueluo
600881a8ea [Inference]Add CUDA KVCache Kernel (#5406) 
* add cuda KVCache kernel

* annotation benchmark_kvcache_copy

* add use cuda

* fix import path

* move benchmark scripts to example/

* rm benchmark codes in test_kv_cache_memcpy.py

* rm redundancy codes

* rm redundancy codes

* pr was modified according to the review
 2024-02-28 14:36:50 +08:00