github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-02 01:28:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

[moe] init mixtral impl

Browse Source
This commit is contained in:
Xuanlei Zhao
2023-12-14 17:52:05 +08:00
committed by ver217
parent c53ddda88f
commit 7d8e0338a4
28 changed files with 2025 additions and 223 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
applications/ColossalMoE/infer.sh Normal file
View File

@@ -0,0 +1,7 @@
NUM_GPU=2
MODEL="mistralai/Mixtral-8x7B-v0.1"
# ep
torchrun --standalone --nproc_per_node $NUM_GPU infer.py \
--model_name $MODEL \
--plugin "ep" \