Jianghai 69cd7e069d [Inference] ADD async and sync Api server using FastAPI (#5396) ...

* add api server

* fix

* add

* add completion service and fix bug

* add generation config

* revise shardformer

* fix bugs

* add docstrings and fix some bugs

* fix bugs and add choices for prompt template

2024-05-08 15:18:28 +00:00

0 lines

Python

Raw Permalink Blame History