19 Commits

Author	SHA1	Message	Date
YeAnbang	f8bd2db33f	add uuid to rollout log	2025-05-20 09:45:56 +08:00
YeAnbang	107470a360	fix logging rollouts	2025-05-17 21:12:58 +08:00
YeAnbang	203dfb1536	address conversation	2025-05-16 14:15:35 +08:00
YeAnbang	4ec73298b2	use consumer global step	2025-05-15 14:15:40 +08:00
YeAnbang	50070c1e84	move logging to producer	2025-05-14 18:10:57 +08:00
YeAnbang	47a7dc7142	Support evaluation during training	2025-05-14 11:03:11 +08:00
YeAnbang	eb6b5dd62e	[fix] revert reward update and evaluation (#6295) ... * Revert "rewrite reward fn" This reverts commit d06042b434. * Revert "upgrade reward math verification" This reverts commit a6085ff676. * Revert "fix bug" This reverts commit 01640ebd65. * Revert "reuse comm-group" This reverts commit bd61918dcf. * Revert "Support evaluation during training" This reverts commit 57a88395fe.	2025-05-07 10:56:47 +08:00
YeAnbang	01640ebd65	fix bug	2025-04-30 22:53:12 +08:00
YeAnbang	57a88395fe	Support evaluation during training	2025-04-30 18:31:49 +08:00
Tong Li	5fd4bcb9d8	[feat] Sync shard model (#6289) ... * [feat] support hybrid parallel model sync * update consumer and producer * update files * update producer * remove print * update --------- Co-authored-by: duanjunwen <935724073@qq.com> Co-authored-by: YeAnbang <44796419+YeAnbang@users.noreply.github.com> Co-authored-by: Tong Li <tong.li35271158@gmail.com>	2025-04-30 14:47:01 +08:00
YeAnbang	2ca1e3c630	fix pp+tp, fix dataloader (#6280)	2025-04-28 17:10:00 +08:00
YeAnbang	26d859f68e	[feat] Support DAPO (#6263) ... * update help information * update style * fix * minor fix * support PP training * add pp support * remove unused code * address conversation * fix memory leakage support tp+pp * move empty cache * move empty cache * add DAPO support * remove format reward * fix filtering, still buggy * small fix * add DAPO support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tested multi-node training; fix bind_batch bug * fix conversation; support sleep mode * support reusing excessive samples * add dynamic batching control flag * add dynamic batching control flag * refactored * fix logging --------- Co-authored-by: Tong Li <tong.li35271158@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2025-04-25 17:39:17 +08:00
YeAnbang	ed43a4be04	[Distributed RLHF] Integration of PP (#6257) ... * update help information * update style * fix * minor fix * support PP training * add pp support * remove unused code * address conversation --------- Co-authored-by: Tong Li <tong.li35271158@gmail.com>	2025-04-09 13:23:24 +08:00
YeAnbang	50153005b4	[feat] add microbatch forwarding (#6251) ... * add microbatch forwarding * fix forward microbatch * fix producer OOM * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change project name * fix temperature annealing * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * address conversation --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2025-03-28 10:24:58 +08:00
YeAnbang	0472f44163	fix logprob, add filtering, temperature annealing, lr descent	2025-03-21 10:24:24 +08:00
YeAnbang	7ee4452f8c	fix vllm	2025-03-19 17:11:10 +08:00
Tong Li	0f566cc2d4	add algo selection	2025-03-06 14:29:22 +08:00
Tong Li	ffd3878a1e	add simple grpo	2025-02-23 22:54:26 +08:00
Hongxin Liu	43c9b5fb44	[chat] add distributed impl (#6210)	2025-02-21 15:24:23 +08:00