Jianghai f47f2fbb24 [Inference] Fix API server, test and example (#5712) ...

* fix api server

* fix generation config

* fix api server

* fix comments

* fix infer hanging bug

* resolve comments, change backend to free port

2024-05-15 15:47:31 +08:00

..

locustfile.py

…

run_locust.sh

…

test_ci.sh

…