9 Commits

Author	SHA1	Message	Date
YeAnbang	53834b74b9	fix num_train_step update	2025-02-20 18:24:04 +08:00
YeAnbang	d20c8ffd97	Add GRPO and Support RLVR for PPO (#6186) ... * add grpo, support rlvr * add grpo, support rlvr * tested deepseek r1 pipeline * add ci * verify grpo r1 * verify grpo r1 * update readme, remove unused code * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove path * clean code * fix circular import * fix ci OOM * fix ci OOM * skip kto tp, fix qwen generation --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2025-02-18 09:43:36 +08:00
Tong Li	39e2597426	[ColossalChat] Add PP support (#6001) ... * support pp training * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update rm * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refactor * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update test case * fix * change to 4 * fix eval * test * add pp * hotfix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * support pp training * update rm * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refactor * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update test case * fix * change to 4 * fix eval * test * add pp * hotfix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update * skip pp eval * update all reduce * update sft * update ignore * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update no cache * add eval * remove fi * remove debug * remove parentheses to avoid warning * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert "add eval" This reverts commit 3ab2f6fa32. * add all reduce --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2024-08-21 10:47:39 +08:00
YeAnbang	0b2d55c4ab	Support overall loss, update KTO logging	2024-08-02 06:51:38 +00:00
YeAnbang	8a3ff4f315	fix style	2024-07-26 09:55:15 +00:00
YeAnbang	b3594d4d68	fix orpo cross entropy loss	2024-07-15 02:12:05 +00:00
YeAnbang	e7a8634636	fix eval	2024-07-11 03:35:03 +00:00
YeAnbang	d888c3787c	add benchmark for sft, dpo, simpo, orpo. Add benchmarking result. Support lora with gradient checkpoint	2024-07-10 10:17:08 +00:00
YeAnbang	c8d1b4a968	add orpo	2024-06-27 07:20:28 +00:00