github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-08-21 01:13:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,867 Commits 58 Branches 51 Tags 237 MiB
grpo-reward-dev
Commit Graph

10 Commits

Author SHA1 Message Date
YeAnbang
50070c1e84 move logging to producer 2025-05-14 18:10:57 +08:00
YeAnbang
47a7dc7142 Support evaluation during training 2025-05-14 11:03:11 +08:00
Tong Li
b920af427b
update pad seq (#6303) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-05-13 16:51:27 +08:00
YeAnbang
eb6b5dd62e
[fix] revert reward update and evaluation (#6295) 
* Revert "rewrite reward fn"

This reverts commit d06042b434.

* Revert "upgrade reward math verification"

This reverts commit a6085ff676.

* Revert "fix bug"

This reverts commit 01640ebd65.

* Revert "reuse comm-group"

This reverts commit bd61918dcf.

* Revert "Support evaluation during training"

This reverts commit 57a88395fe.
 2025-05-07 10:56:47 +08:00
YeAnbang
57a88395fe Support evaluation during training 2025-04-30 18:31:49 +08:00
YeAnbang
26d859f68e
[feat] Support DAPO (#6263) 
* update help information

* update style

* fix

* minor fix

* support PP training

* add pp support

* remove unused code

* address conversation

* fix memory leakage support tp+pp

* move empty cache

* move empty cache

* add DAPO support

* remove format reward

* fix filtering, still buggy

* small fix

* add DAPO support

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tested multi-node training; fix bind_batch bug

* fix conversation; support sleep mode

* support reusing excessive samples

* add dynamic batching control flag

* add dynamic batching control flag

* refactored

* fix logging

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2025-04-25 17:39:17 +08:00
duanjunwen
7795d4c50d
[Feature] Support Distributed LogProb for GRPO Training (#6247) 
* [fix] fix qwen VocabParallelLMHead1D and gather output

* fix tp bug

* fix consumer

* [feat] Support Distributed LogProb for GRPO Training

* [fix] fix loss func

* [fix] fix log prob plugin

* [fix] fix qwen modeling param

* [fix] rm comments

* [fix] rm hard-code;fix non-dist version

* [fix] fix test file param name and benchmark tp gather output=True/False

* [fix] rm non-dist version in dist log prob

* [fix] fix comments

* [fix] fix dis log prob plugin

* [fix] fix test case

* [fix] fix qwen VocabParallelLMHead1D and gather output

* [fix] fix DistLogProb comments

* [fix] restore tp size

* [fix] fix comments

* [fix] fix comment; fix LogSoftmax usage

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-03-18 17:47:55 +08:00
Tong Li
ffd3878a1e add simple grpo 2025-02-23 22:54:26 +08:00
Hongxin Liu
de282dd694
[feature] fit RL style generation (#6213) 
* [feature] fit rl style generation

* [doc] add docstr

* [doc] add docstr
 2025-02-21 17:28:19 +08:00
Hongxin Liu
43c9b5fb44
[chat] add distributed impl (#6210) 2025-02-21 15:24:23 +08:00