ColossalAI/tests/test_infer/test_batch_bucket.py at eb6b5dd62ebfa907acb044ecdce208bcf088a310

mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-07-16 17:16:14 +00:00

Files

yuehuayingxueluo bc1da87366 [Fix/Inference] Fix format of input prompts and input model in inference engine (#5395 )

* Fix bugs in inference_engine

* fix bugs in engine.py

* rm  CUDA_VISIBLE_DEVICES

* add request_ids in generate

* fix bug in engine.py

* add logger.debug for BatchBucket

2024-02-23 10:51:35 +08:00

5.1 KiB

Raw Blame History

View Raw

5.1 KiB Raw Blame History

5.1 KiB

Raw Blame History