github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-06-17 11:19:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,850 Commits 54 Branches 51 Tags 235 MiB
hotfix/example_test
Commit Graph

2 Commits

Author SHA1 Message Date
Yuanheng Zhao
573f270537
[Infer] Serving example w/ ray-serve (multiple GPU case) (#4841) 
* fix imports

* add ray-serve with Colossal-Infer tp

* trivial: send requests script

* add README

* fix worker port

* fix readme

* use app builder and autoscaling

* trivial: input args

* clean code; revise readme

* testci (skip example test)

* use auto model/tokenizer

* revert imports fix (fixed in other PRs)
 2023-10-02 17:48:38 +08:00
Yuanheng Zhao
3a74eb4b3a
[Infer] Colossal-Inference serving example w/ TorchServe (single GPU case) (#4771) 
* add Colossal-Inference serving example w/ TorchServe

* add dockerfile

* fix dockerfile

* fix dockerfile: fix commit hash, install curl

* refactor file structure

* revise readme

* trivial

* trivial: dockerfile format

* clean dir; revise readme

* fix comments: fix imports and configs

* fix formats

* remove unused requirements
 2023-10-02 17:42:37 +08:00