Jianghai c064032865 [Online Server] Chat Api for streaming and not streaming response (#5470) ...

* fix bugs

* fix bugs

* fix api server

* fix api server

* add chat api and test

* del request.n

2024-05-08 15:20:53 +00:00

..

__init__.py

[Inference] ADD async and sync Api server using FastAPI (#5396)

2024-05-08 15:18:28 +00:00

api_server.py

[Online Server] Chat Api for streaming and not streaming response (#5470)

2024-05-08 15:20:53 +00:00

chat_service.py

[Online Server] Chat Api for streaming and not streaming response (#5470)

2024-05-08 15:20:53 +00:00

completion_service.py

[Inference] Finish Online Serving Test, add streaming output api, continuous batching test and example (#5432)

2024-05-08 15:20:52 +00:00

utils.py

[Online Server] Chat Api for streaming and not streaming response (#5470)

2024-05-08 15:20:53 +00:00