yuehuayingxueluo d482922035 [Inference] Support the logic related to ignoring EOS token (#5693) ... * Adapt temperature processing logic * add ValueError for top_p and top_k * add GQA Test * fix except_msg * support ignore EOS token * change variable's name * fix annotation		2024-05-08 19:59:10 +08:00
..
__init__.py	[doc] updated inference readme (#5343)	2024-02-02 14:31:10 +08:00
engine.py	[Inference] Support the logic related to ignoring EOS token (#5693)	2024-05-08 19:59:10 +08:00
plugin.py	[Feat]Tensor Model Parallel Support For Inference (#5563)	2024-04-18 16:56:46 +08:00
request_handler.py	[Inference]Adapt temperature processing logic (#5689)	2024-05-08 17:58:29 +08:00