4 Commits

Author	SHA1	Message	Date
digger yu	5e1c93d732	[hotfix] fix typo change MoECheckpintIO to MoECheckpointIO (#5335) ... Co-authored-by: binmakeswell <binmakeswell@gmail.com>	2024-03-05 21:52:30 +08:00
Xuanlei Zhao	7d8e0338a4	[moe] init mixtral impl	2024-02-07 19:21:02 +08:00
Xuanlei Zhao	f71e63b0f3	[moe] support optimizer checkpoint (#5015) ... * Refactor MoE Manager setup method * unshard optim ckpt * optim io * update transformer version * update requirements * update ckpt * update ckpt * update ckpt * fix engine * fix engine	2023-11-08 15:07:03 +00:00
Xuanlei Zhao	dc003c304c	[moe] merge moe into main (#4978) ... * update moe module * support openmoe	2023-11-02 02:21:24 +00:00