[Inference] Add CacheBlock and KV-Cache Manager (#5156)

* [Inference] Add KVCache Manager

* function refactored

* add test for KVCache Manager

* add attr beam width

* Revise alloc func in CacheManager

* Fix docs and pytests

* add tp slicing for head number

* optimize shapes of tensors used as physical cache

* Apply using InferenceConfig on KVCacheManager

* rm duplicate config file

* Optimize cache allocation: use contiguous cache

* Fix config in pytest (and config)

colossalai/inference/kv_cache/__init__.py

@@ -0,0 +1,4 @@
+from .block_cache import CacheBlock
+from .kvcache_manager import KVCacheManager
+
+__all__ = ["CacheBlock", "KVCacheManager"]