3 Commits

Author	SHA1	Message	Date
CjhHa1	bc9063adf1	resolve rebase conflicts on Branch feat/online-serving	2024-05-08 15:20:53 +00:00
Jianghai	61a1b2e798	[Inference] Fix bugs and docs for feat/online-server (#5598) ... * fix test bugs * add do sample test * del useless lines * fix comments * fix tests * delete version tag * delete version tag * add * del test sever * fix test * fix * Revert "add" This reverts commit b9305fb024.	2024-05-08 15:20:53 +00:00
Jianghai	de378cd2ab	[Inference] Finish Online Serving Test, add streaming output api, continuous batching test and example (#5432) ... * finish online test and add examples * fix test_contionus_batching * fix some bugs * fix bash * fix * fix inference * finish revision * fix typos * revision	2024-05-08 15:20:52 +00:00