Bin Jia 1db6727678 [Pipeline inference] Combine kvcache with pipeline inference (#4938) ...

* merge kvcache with pipeline inference and refactor the code structure

* support ppsize > 2

* refactor pipeline code

* do pre-commit

* modify benchmark

* fix bench mark

* polish code

* add docstring and update readme

* refactor the code

* fix some logic bug of ppinfer

* polish readme

* fix typo

* skip infer test

2023-10-27 16:19:54 +08:00

..

_utils.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_bloom_infer.py

[infer] fix test bug (#4838)

2023-10-04 10:01:03 +08:00

test_chatglm2_infer.py

[infer] fix test bug (#4838)

2023-10-04 10:01:03 +08:00

test_infer_engine.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_kvcache_manager.py

[misc] update pre-commit and run all files (#4752)

2023-09-19 14:20:26 +08:00

test_llama2_infer.py

[inference] add llama2 support (#4898)

2023-10-13 13:09:23 +08:00

test_llama_infer.py

fix test llama (#4884)

2023-10-11 17:43:01 +08:00

test_pipeline_infer.py

[Pipeline inference] Combine kvcache with pipeline inference (#4938)

2023-10-27 16:19:54 +08:00