github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-05-08 16:38:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
be396ad6cc
ColossalAI/tests/test_infer
History
Steve Luo be396ad6cc
[Inference/Kernel] Add Paged Decoding kernel, sequence split within the same thread block () 
* feat flash decoding for paged attention

* refactor flashdecodingattention

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-04-18 16:45:07 +08:00
..
test_models [inference/model]Adapted to the baichuan2-7B model () 2024-04-15 16:53:02 +08:00
test_ops [Inference/Kernel] Add Paged Decoding kernel, sequence split within the same thread block () 2024-04-18 16:45:07 +08:00
_utils.py [Inference] Add the logic of the inference engine () 2024-01-11 13:39:56 +00:00
test_batch_bucket.py [Fix/Inference] Fix format of input prompts and input model in inference engine () 2024-02-23 10:51:35 +08:00
test_config_and_struct.py [Inference] Optimize and Refactor Inference Batching/Scheduling () 2024-02-19 17:18:20 +08:00
test_cuda_graph.py [fix] 2024-03-25 11:37:58 +08:00
test_drafter.py [Inference/SpecDec] Support GLIDE Drafter Model () 2024-04-10 11:07:52 +08:00
test_inference_engine.py [Inference/SpecDec] Support GLIDE Drafter Model () 2024-04-10 11:07:52 +08:00
test_kvcache_manager.py [Inference] Optimize and Refactor Inference Batching/Scheduling () 2024-02-19 17:18:20 +08:00
test_request_handler.py [Inference] Optimize and Refactor Inference Batching/Scheduling () 2024-02-19 17:18:20 +08:00