github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-07-13 12:05:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,101 Commits 62 Branches 51 Tags
ed431de4e4f73584e6b9c11ab041ef54a8e83de6
Commit Graph

3 Commits

Author SHA1 Message Date
Steve Luo
f7aecc0c6b feat rmsnorm cuda kernel and add unittest, benchmark script (#5417) 2024-03-08 16:21:12 +08:00
yuehuayingxueluo
0aa27f1961 [Inference]Move benchmark-related code to the example directory. (#5408) 
* move benchmark-related code to the example directory.

* fix bugs in test_fused_rotary_embedding.py
 2024-02-28 16:46:03 +08:00
yuehuayingxueluo
600881a8ea [Inference]Add CUDA KVCache Kernel (#5406) 
* add cuda KVCache kernel

* annotation benchmark_kvcache_copy

* add use cuda

* fix import path

* move benchmark scripts to example/

* rm benchmark codes in test_kv_cache_memcpy.py

* rm redundancy codes

* rm redundancy codes

* pr was modified according to the review
 2024-02-28 14:36:50 +08:00