10 Commits

Author	SHA1	Message	Date
LuGY	1ff7d5bfa5	[NFC] polish colossalai/engine/gradient_handler/_moe_gradient_handler.py (#3260)	2023-03-29 15:22:21 +08:00
Frank Lee	11f54c7b6b	[doc] improved docstring and assertion messages for the engine module (#871)	2022-04-26 10:00:18 +08:00
Jiarui Fang	e956d93ac2	[refactor] memory utils (#577)	2022-04-01 09:22:33 +08:00
HELSON	e6d50ec107	[zero] adapt zero for unsharded parameters (#561) ... * support existing sharded and unsharded parameters in zero * add unitest for moe-zero model init * polish moe gradient handler	2022-03-31 18:34:11 +08:00
Jiarui Fang	a445e118cf	[polish] polish singleton and global context (#500)	2022-03-23 18:03:39 +08:00
Jiarui Fang	65c0f380c2	[format] polish name format for MOE (#481)	2022-03-21 23:19:47 +08:00
HELSON	aff9d354f7	[MOE] polish moe_env (#467)	2022-03-19 15:36:25 +08:00
HELSON	84fd7c1d4d	add moe context, moe utilities and refactor gradient handler (#455)	2022-03-18 16:38:32 +08:00
HELSON	0f8c7f9804	Fixed docstring in colossalai (#171)	2022-01-21 10:44:30 +08:00
HELSON	dceae85195	Added MoE parallel (#127)	2022-01-07 15:08:36 +08:00