ColossalAI/colossalai/inference/modeling
flybird11111 d0e13b85fd
[upgrade]Upgrade mixtral (#6317)
* upgrade mixtral

* fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* upgrade infer

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

* upgrade drafter

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

* upgrade lazy

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* upgrade mixtral

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-05-21 16:14:05 +08:00
..
backends [Inference] Fix flash-attn import and add model test (#5794) 2024-06-12 14:13:50 +08:00
layers [Feat] Distrifusion Acceleration Support for Diffusion Inference (#5895) 2024-07-30 10:43:26 +08:00
models [upgrade]Upgrade mixtral (#6317) 2025-05-21 16:14:05 +08:00
policy [shardformer] fix linear 1d row and support uneven splits for fused qkv linear (#6084) 2024-10-10 14:34:45 +08:00
__init__.py [doc] updated inference readme (#5343) 2024-02-02 14:31:10 +08:00