ColossalAI/colossalai/inference/kv_cache at fded91d049997ed87dee965fc42c35a239e3ec03 - ColossalAI - Gitea: Git with a cup of tea at home

github/ColossalAI

mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-04-12 15:14:55 +00:00

Files

History

yuehuayingxueluo d40eb26029 fix bugs in request_handler.py and engine.py

2024-01-11 13:46:14 +00:00

..

__init__.py

[Inference] Add CacheBlock and KV-Cache Manager (#5156 )

2024-01-11 13:39:29 +00:00

block_cache.py

[Inference] Add CacheBlock and KV-Cache Manager (#5156 )

2024-01-11 13:39:29 +00:00

kvcache_manager.py

fix bugs in request_handler.py and engine.py

2024-01-11 13:46:14 +00:00