ColossalAI/colossalai/inference/kv_cache at 1513f20f4d80f782fab381996368ff2c2f3c95c3 - ColossalAI - Gitea: Git with a cup of tea at home

github/ColossalAI

mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-04-25 01:03:35 +00:00

Files

History

yuehuayingxueluo d40eb26029 fix bugs in request_handler.py and engine.py

2024-01-11 13:46:14 +00:00

..

__init__.py

[Inference] Add CacheBlock and KV-Cache Manager (#5156 )

2024-01-11 13:39:29 +00:00

block_cache.py

[Inference] Add CacheBlock and KV-Cache Manager (#5156 )

2024-01-11 13:39:29 +00:00

kvcache_manager.py

fix bugs in request_handler.py and engine.py

2024-01-11 13:46:14 +00:00