Zhongkai Zhao 361cf63cb0 [Refactor] refactor policy search and quant type controlling in inference (#5035) ... * [Refactor] refactor policy search and quant type controling in inference		2023-11-14 17:26:59 +08:00
..
serving	[Infer] Serving example w/ ray-serve (multiple GPU case) (#4841)	2023-10-02 17:48:38 +08:00
_utils.py	[Inference]ADD Bench Chatglm2 script (#4963)	2023-10-24 13:11:15 +08:00
bench_bloom.py	[Refactor] refactor policy search and quant type controlling in inference (#5035)	2023-11-14 17:26:59 +08:00
bench_chatglm2.py	[Refactor] refactor policy search and quant type controlling in inference (#5035)	2023-11-14 17:26:59 +08:00
bench_llama.py	[Refactor] refactor policy search and quant type controlling in inference (#5035)	2023-11-14 17:26:59 +08:00
benchmark.py	[Refactor] remove useless inference code (#5022)	2023-11-10 14:47:06 +08:00
gptq_bloom.py	[Refactor] refactor policy search and quant type controlling in inference (#5035)	2023-11-14 17:26:59 +08:00
gptq_llama.py	[Refactor] refactor policy search and quant type controlling in inference (#5035)	2023-11-14 17:26:59 +08:00
hybrid_gptq_llama.py	[refactor] refactor gptq and smoothquant llama (#5012)	2023-11-09 10:12:11 +08:00
hybrid_smoothquant_llama.py	[refactor] refactor gptq and smoothquant llama (#5012)	2023-11-09 10:12:11 +08:00
run.sh	[Refactor] remove useless inference code (#5022)	2023-11-10 14:47:06 +08:00
smoothquant_llama.py	[inference] Add smmoothquant for llama (#4904)	2023-10-16 11:28:44 +08:00