13 Commits

Author	SHA1	Message	Date
YeAnbang	38008858e4	fix checkpoint naming; add num_epoch parameter (#6277)	2025-04-26 14:00:28 +08:00
YeAnbang	26d859f68e	[feat] Support DAPO (#6263) ... * update help information * update style * fix * minor fix * support PP training * add pp support * remove unused code * address conversation * fix memory leakage support tp+pp * move empty cache * move empty cache * add DAPO support * remove format reward * fix filtering, still buggy * small fix * add DAPO support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tested multi-node training; fix bind_batch bug * fix conversation; support sleep mode * support reusing excessive samples * add dynamic batching control flag * add dynamic batching control flag * refactored * fix logging --------- Co-authored-by: Tong Li <tong.li35271158@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2025-04-25 17:39:17 +08:00
Tong Li	b823c6eec7	[feat] Add final save at the end (#6274) ... * add final save * default 1 episode	2025-04-23 10:03:46 +08:00
YeAnbang	9467c10690	[hot-fix] Fix memory leakage bug, support TP+PP (#6258) ... * update help information * update style * fix * minor fix * support PP training * add pp support * remove unused code * address conversation * fix memory leakage support tp+pp * move empty cache * move empty cache --------- Co-authored-by: Tong Li <tong.li35271158@gmail.com>	2025-04-10 10:52:18 +08:00
YeAnbang	ed43a4be04	[Distributed RLHF] Integration of PP (#6257) ... * update help information * update style * fix * minor fix * support PP training * add pp support * remove unused code * address conversation --------- Co-authored-by: Tong Li <tong.li35271158@gmail.com>	2025-04-09 13:23:24 +08:00
YeAnbang	50153005b4	[feat] add microbatch forwarding (#6251) ... * add microbatch forwarding * fix forward microbatch * fix producer OOM * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change project name * fix temperature annealing * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * address conversation --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2025-03-28 10:24:58 +08:00
YeAnbang	0472f44163	fix logprob, add filtering, temperature annealing, lr descent	2025-03-21 10:24:24 +08:00
duanjunwen	7795d4c50d	[Feature] Support Distributed LogProb for GRPO Training (#6247) ... * [fix] fix qwen VocabParallelLMHead1D and gather output * fix tp bug * fix consumer * [feat] Support Distributed LogProb for GRPO Training * [fix] fix loss func * [fix] fix log prob plugin * [fix] fix qwen modeling param * [fix] rm comments * [fix] rm hard-code;fix non-dist version * [fix] fix test file param name and benchmark tp gather output=True/False * [fix] rm non-dist version in dist log prob * [fix] fix comments * [fix] fix dis log prob plugin * [fix] fix test case * [fix] fix qwen VocabParallelLMHead1D and gather output * [fix] fix DistLogProb comments * [fix] restore tp size * [fix] fix comments * [fix] fix comment; fix LogSoftmax usage --------- Co-authored-by: Tong Li <tong.li35271158@gmail.com>	2025-03-18 17:47:55 +08:00
Tong Li	afddfde2dd	fix consumer	2025-03-13 14:55:26 +08:00
pre-commit-ci[bot]	eb6337f07f	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	2025-03-06 08:29:59 +00:00
Tong Li	0cc0c843ed	add save	2025-03-06 16:26:14 +08:00
Hongxin Liu	de282dd694	[feature] fit RL style generation (#6213) ... * [feature] fit rl style generation * [doc] add docstr * [doc] add docstr	2025-02-21 17:28:19 +08:00
Hongxin Liu	43c9b5fb44	[chat] add distributed impl (#6210)	2025-02-21 15:24:23 +08:00