3821 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Maruyama_Aya	d2dc6049b5	fix format (#580)	2022-04-06 11:40:59 +08:00
wky	174b9c1d85	fix format (#574)	2022-04-06 11:40:59 +08:00
BoxiangW	dfe423ae42	fix format (#572)	2022-04-06 11:40:59 +08:00
yuxuan-lou	cfb41297ff	'fix/format' (#573)	2022-04-06 11:40:59 +08:00
Kai Wang (Victor Kai)	b0f708dfc1	fix format (#570)	2022-04-06 11:40:59 +08:00
Xu Kai	2a915a8b62	fix format (#568)	2022-04-06 11:40:59 +08:00
YuliangLiu0306	9420d3ae31	fix format (#567)	2022-04-06 11:40:59 +08:00
Jie Zhu	0f1da44e5e	[format]colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp (#566)	2022-04-06 11:40:59 +08:00
coder-chin	5835631218	fix format (#564)	2022-04-06 11:40:59 +08:00
Luxios22	e014144c44	fix format (#565)	2022-04-06 11:40:59 +08:00
Ziyue Jiang	1762ba14ab	fix format (#563)	2022-04-06 11:40:59 +08:00
Sze-qq	ce8a3eae5b	update GPT-2 experiment result (#666)	2022-04-04 13:47:43 +08:00
HELSON	17e73e62cc	[hotfix] fix bugs for unsharded parameters when restore data (#664)	2022-04-03 22:02:11 +08:00
Jiarui Fang	0aab52301e	[hotfix] fix a bug in model data stats tracing (#655)	2022-04-03 21:48:06 +08:00
YuliangLiu0306	ade05a5d83	[refactor] pipeline, put runtime schedule into engine. (#627)	2022-04-03 20:46:45 +08:00
HELSON	e5d615aeee	[hotfix] fix bugs in testing (#659) ... * remove hybrid adam in test_moe_zero_optim * fix activation checkpointing and its unitest	2022-04-02 21:58:47 +08:00
Jiarui Fang	036404ca8a	Revert "[zero] polish init context (#645)" (#657)	2022-04-02 18:30:06 +08:00
HELSON	b31daed4cf	fix bugs in CPU adam (#633) ... * add cpu adam counter for all cpu adam * fixed updating error in adam kernel	2022-04-02 17:04:05 +08:00
LuGY	1e2557e801	[zero] fixed the activation offload (#647) ... * fixed the activation offload * polish	2022-04-02 16:21:32 +08:00
Liang Bowen	828e465622	[hotfix] Raise messages for indivisible batch sizes with tensor parallelism (#622)	2022-04-02 16:12:04 +08:00
binmakeswell	e0f875a8e2	[GitHub] Add prefix and label in issue template (#652)	2022-04-02 16:09:25 +08:00
Jiarui Fang	67b4928244	[zero] polish init context (#645)	2022-04-02 15:52:04 +08:00
ver217	f5d3a9c2b0	polish checkpoint docstring (#637)	2022-04-02 13:34:33 +08:00
HELSON	055fbf5be6	[zero] adapt zero for unsharded paramters (Optimizer part) (#601)	2022-04-01 20:10:47 +08:00
KAIYUAN GAN	229382c844	[NFC] polish colossalai/kernel/cuda_native/csrc/kernels/cuda_util.cu code stype (#625)	2022-04-01 17:45:53 +08:00
アマデウス	354b7954d1	[model checkpoint] added unit tests for checkpoint save/load (#599)	2022-04-01 16:53:32 +08:00
アマデウス	28b515d610	[model checkpoint] updated checkpoint hook (#598)	2022-04-01 16:53:03 +08:00
アマデウス	77ad24bf94	[model checkpoint] updated saving/loading for 3d layers (#597)	2022-04-01 16:52:47 +08:00
アマデウス	93089ed708	[model checkpoint] updated saving/loading for 2.5d layers (#596)	2022-04-01 16:52:33 +08:00
アマデウス	6302069c0e	[model checkpoint] updated communication ops for cpu tensors (#590)	2022-04-01 16:52:20 +08:00
アマデウス	c50bfb807b	[model checkpoint] updated saving/loading for 1d layers (#594)	2022-04-01 16:51:52 +08:00
アマデウス	7636d518e1	[model checkpoint] updated saving/loading for 2d layers (#595)	2022-04-01 16:50:34 +08:00
アマデウス	cd13b63832	[model checkpoint] reworked unified layers for ease of save/load states (#593)	2022-04-01 16:49:56 +08:00
アマデウス	acae68eb04	[model checkpoint] updated checkpoint save/load utils (#592)	2022-04-01 16:49:21 +08:00
Ziyue Jiang	1c40ee8749	[TP] add assert for tp1d (#621)	2022-04-01 16:44:23 +08:00
ver217	369a288bf3	polish utils docstring (#620)	2022-04-01 16:36:47 +08:00
ver217	e619a651fb	polish optimizer docstring (#619)	2022-04-01 16:27:03 +08:00
ver217	8432dc7080	polish moe docsrting (#618)	2022-04-01 16:15:36 +08:00
ver217	c5b488edf8	polish amp docstring (#616)	2022-04-01 16:09:39 +08:00
ver217	f69507dd22	update rst (#615)	2022-04-01 15:46:38 +08:00
FredHuang99	93f14d2a33	[zero] test zero tensor utils (#609)	2022-04-01 15:16:59 +08:00
ver217	0ef8819c67	polish docstring of zero (#612)	2022-04-01 14:50:56 +08:00
LuGY	02b187c14f	[zero] add sampling time for memstats collector (#610)	2022-04-01 14:03:00 +08:00
ver217	9bee119104	[hotfix] fix sharded optim zero grad (#604) ... * fix sharded optim zero grad * polish comments	2022-04-01 12:41:20 +08:00
アマデウス	297b8baae2	[model checkpoint] add gloo groups for cpu tensor communication (#589)	2022-04-01 10:15:52 +08:00
アマデウス	54e688b623	moved ensure_path_exists to utils.common (#591)	2022-04-01 09:46:33 +08:00
Jiarui Fang	e956d93ac2	[refactor] memory utils (#577)	2022-04-01 09:22:33 +08:00
ver217	104cbbb313	[hotfix] add hybrid adam to __init__ (#584)	2022-03-31 19:08:34 +08:00
HELSON	e6d50ec107	[zero] adapt zero for unsharded parameters (#561) ... * support existing sharded and unsharded parameters in zero * add unitest for moe-zero model init * polish moe gradient handler	2022-03-31 18:34:11 +08:00
LuGY	13ed4b6441	[model zoo] add activation offload for gpt model (#582)	2022-03-31 17:42:20 +08:00