Zhongkai Zhao 70885d707d [hotfix] Suport extra_kwargs in ShardConfig (#5031) ...

* [refactor]: replace inference args with extra_kwargs in ShardConfig

* modify shardconfig

* polish code

* fix policy bug in llama

* fix bug in auto policy

* remove setattr in ShardConfig

2023-11-10 10:49:50 +08:00

..

serving

[hotfix] Suport extra_kwargs in ShardConfig (#5031)

2023-11-10 10:49:50 +08:00

_utils.py

[Inference]ADD Bench Chatglm2 script (#4963)

2023-10-24 13:11:15 +08:00

bench_bloom.py

[hotfix] Suport extra_kwargs in ShardConfig (#5031)

2023-11-10 10:49:50 +08:00

bench_chatglm2.py

[hotfix] Suport extra_kwargs in ShardConfig (#5031)

2023-11-10 10:49:50 +08:00

bench_llama.py

[hotfix] Suport extra_kwargs in ShardConfig (#5031)

2023-11-10 10:49:50 +08:00

gptq_bloom.py

[hotfix] Suport extra_kwargs in ShardConfig (#5031)

2023-11-10 10:49:50 +08:00

gptq_llama.py

[hotfix] Suport extra_kwargs in ShardConfig (#5031)

2023-11-10 10:49:50 +08:00

smoothquant_llama.py

[inference] Add smmoothquant for llama (#4904)

2023-10-16 11:28:44 +08:00