[Inference] Add the logic of the inference engine (#5173)

* add infer_struct and infer_config

* update codes

* change InferConfig

* Add hf_model_config to the engine

* rm _get_hf_model_config

* update codes

* made adjustments according to the feedback from the reviewer.

* update codes

* add ci test for config and struct

* Add the logic of the inference engine

* update engine and test

* Recover cache_manager.py

* add logger

* fix conflict

* update codes

* update codes

* update model and tokenizer

* fix add the logic about shardformer

* change kvcache_manager docstring

* add policy

* fix ci bug in test_kvcache_manager.py

* remove codes related o tokenizer and move model_policy

* fix  code style

* add ordered_set to requirements-infer.txt

* Delete extra empty lines

* add ordered_set to requirements-test.txt

colossalai/inference/modeling/policy/__init__.py

@@ -0,0 +1,7 @@
+from .llama import LlamaModelInferPolicy
+
+model_policy_map = {
+    "llama": LlamaModelInferPolicy,
+}
+
+__all__ = ["LlamaModelInferPolicy", "model_polic_map"]

colossalai/inference/modeling/policy/llama.py

@@ -0,0 +1,7 @@
+from colossalai.shardformer.policies.llama import LlamaForCausalLMPolicy
+
+
+class LlamaModelInferPolicy(LlamaForCausalLMPolicy):
+    # The code here just for test and will be modified later.
+    def __init__(self) -> None:
+        super().__init__()