github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-04-26 01:35:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
7bbb28e48bdb5849d9dfb118d7bf2959d79bbe02
ColossalAI/tests/test_infer/test_async_engine
History
Jianghai 69cd7e069d [Inference] ADD async and sync Api server using FastAPI (#5396) 
* add api server

* fix

* add

* add completion service and fix bug

* add generation config

* revise shardformer

* fix bugs

* add docstrings and fix some bugs

* fix bugs and add choices for prompt template
2024-05-08 15:18:28 +00:00
..
test_async_engine.py
[Inference] ADD async and sync Api server using FastAPI (#5396)
2024-05-08 15:18:28 +00:00
test_request_tracker.py
[Inference] ADD async and sync Api server using FastAPI (#5396)
2024-05-08 15:18:28 +00:00