[inference] Refactor inference architecture (#5057)

* [inference] support only TP (#4998)

* support only tp

* enable tp

* add support for bloom (#5008)

* [refactor] refactor gptq and smoothquant llama (#5012)

* refactor gptq and smoothquant llama

* fix import error

* fix linear import torch-int

* fix smoothquant llama import error

* fix import accelerate error

* fix bug

* fix import smooth cuda

* fix smoothcuda

* [Inference Refactor] Merge chatglm2 with pp and tp (#5023)

merge chatglm with pp and tp

* [Refactor] remove useless inference code (#5022)

* remove useless code

* fix quant model

* fix test import bug

* mv original inference legacy

* fix chatglm2

* [Refactor] refactor policy search and quant type controlling in inference (#5035)

* [Refactor] refactor policy search and quant type controling in inference

* [inference] update readme (#5051)

* update readme

* update readme

* fix architecture

* fix table

* fix table

* [inference] udpate example (#5053)

* udpate example

* fix run.sh

* fix rebase bug

* fix some errors

* update readme

* add some features

* update interface

* update readme

* update benchmark

* add requirements-infer

---------

Co-authored-by: Bin Jia <45593998+FoolPlayer@users.noreply.github.com>
Co-authored-by: Zhongkai Zhao <kanezz620@gmail.com>

colossalai/legacy/inference/serving/ray_serve/send_requests.py

@@ -0,0 +1,27 @@
+import ray
+import requests
+
+
+@ray.remote
+def send_query(text):
+    resp = requests.get("http://localhost:8000/?text={}".format(text))
+    return resp.text
+
+
+test_sentences = [
+    "Introduce some landmarks in Beijing",
+    "What is the weather today",
+    "Coding requires practice and patience",
+    "Rainy days inspire cozy reading",
+    "Laughter is contagious and heartwarming",
+    "Hiking mountains builds strength and resilience",
+    "Family bonds grow stronger with time",
+    "Science unlocks mysteries of the universe",
+    "Music soothes the soul and ignites passion",
+    "Artistic expression knows no boundaries",
+]
+
+results = ray.get([send_query.remote(text) for text in test_sentences])
+print("Result returned:")
+for res in results:
+    print(res)