6 Commits

Author	SHA1	Message	Date
HELSON	d7ecaf362b	[zero] fix init bugs in zero context (#686) ... * adapt model weight initialization for methods in Pytorch nn.init	2022-04-07 17:38:45 +08:00
Jiarui Fang	0aab52301e	[hotfix] fix a bug in model data stats tracing (#655)	2022-04-03 21:48:06 +08:00
HELSON	055fbf5be6	[zero] adapt zero for unsharded paramters (Optimizer part) (#601)	2022-04-01 20:10:47 +08:00
HELSON	e6d50ec107	[zero] adapt zero for unsharded parameters (#561) ... * support existing sharded and unsharded parameters in zero * add unitest for moe-zero model init * polish moe gradient handler	2022-03-31 18:34:11 +08:00
Jiarui Fang	7675366fce	[polish] rename col_attr -> colo_attr (#558)	2022-03-31 12:25:45 +08:00
HELSON	8c90d4df54	[zero] add zero context manager to change config during initialization (#546)	2022-03-29 17:57:59 +08:00