Jianghai 9e2342bde2 [Hotfix] Fix bugs in testing continuous batching (#5270) ...

* fix bug

* fix bugs

* fix bugs

* fix bugs and add padding

* add funcs and fix bugs

* fix typos

* fix bugs

* add func

2024-01-18 16:31:14 +08:00

..

benchmark_llama.py

[Hotfix] Fix bugs in testing continuous batching (#5270)

2024-01-18 16:31:14 +08:00

build_smoothquant_weight.py

[inference] refactor examples and fix schedule (#5077)

2023-11-21 10:46:03 +08:00

run_benchmark.sh

[Inference]Adapted to the triton attn kernels (#5264)

2024-01-17 16:03:10 +08:00

run_llama_inference.py

[inference] refactor examples and fix schedule (#5077)

2023-11-21 10:46:03 +08:00