傅剑寒
|
7ff42cc06d
|
add vec_type_trait implementation (#5473)
|
2024-03-19 18:36:40 +08:00 |
|
xs_courtesy
|
5724b9e31e
|
add some comments
|
2024-03-15 11:18:57 +08:00 |
|
xs_courtesy
|
388e043930
|
add implementatino for GetGPULaunchConfig1D
|
2024-03-14 11:13:40 +08:00 |
|
yuehuayingxueluo
|
f366a5ea1f
|
[Inference/kernel]Add Fused Rotary Embedding and KVCache Memcopy CUDA Kernel (#5418)
* add rotary embedding kernel
* add rotary_embedding_kernel
* add fused rotary_emb and kvcache memcopy
* add fused_rotary_emb_and_cache_kernel.cu
* add fused_rotary_emb_and_memcopy
* fix bugs in fused_rotary_emb_and_cache_kernel.cu
* fix ci bugs
* use vec memcopy and opt the gloabl memory access
* fix code style
* fix test_rotary_embdding_unpad.py
* codes revised based on the review comments
* fix bugs about include path
* rm inline
|
2024-03-13 17:20:03 +08:00 |
|
xs_courtesy
|
5eb5ff1464
|
refactor code
|
2024-03-08 15:41:14 +08:00 |
|
xs_courtesy
|
95c21498d4
|
add silu_and_mul for infer
|
2024-03-07 16:57:49 +08:00 |
|