Jianghai 0e616462a7 [Inference] add logit processor and request handler (#5166) ...

* add logit processor and request handler

* add

* add

* add

* fix

* add search tokens and update func

* finish request handler

* add running list test

* fix test

* fix some bug

* add

* add

* fix bugs

* fix some bugs

* fix bug

* fix

* fix

* add copy fun

* del useless attn

* fix request status

---------

Co-authored-by: CjhHa1 <cjh18671720497outlook.com>

2024-01-11 13:39:56 +00:00

3.9 KiB

Raw Blame History

View Raw