github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-07-08 21:03:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,842 Commits 57 Branches 51 Tags 236 MiB
pre-commit-ci-update-config
Commit Graph

2 Commits

Author SHA1 Message Date
Jianghai
c064032865 [Online Server] Chat Api for streaming and not streaming response (#5470) 
* fix bugs

* fix bugs

* fix api server

* fix api server

* add chat api and test

* del request.n
 2024-05-08 15:20:53 +00:00
Jianghai
69cd7e069d [Inference] ADD async and sync Api server using FastAPI (#5396) 
* add api server

* fix

* add

* add completion service and fix bug

* add generation config

* revise shardformer

* fix bugs

* add docstrings and fix some bugs

* fix bugs and add choices for prompt template
 2024-05-08 15:18:28 +00:00