github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-08-18 16:07:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,913 Commits 58 Branches 51 Tags 237 MiB
grpo_optimization
Commit Graph

31 Commits

Author SHA1 Message Date
Tong Li
8abf186ce2 fix behind 2025-06-26 10:27:00 +08:00
YeAnbang
ff6696a9bb support n_behind, add profiling 2025-06-20 03:14:00 +00:00
Tong Li
bb6f5d98fc
move out evaluation func (#6343) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-06-10 13:53:19 +08:00
YeAnbang
d0e12c508a remove debug code 2025-06-09 09:42:58 +08:00
YeAnbang
3bed6ae9ee fix bug, tested 2025-06-09 09:37:28 +08:00
YeAnbang
dc3033e68a support code generation tasks 2025-06-05 18:05:22 +08:00
YeAnbang
ee939d9aa5 address conversation 2025-05-29 10:25:59 +08:00
YeAnbang
58f8c9bb43 Merge branch 'grpo-latest' of https://github.com/hpcaitech/ColossalAI into grpo-latest-dev 2025-05-28 17:34:52 +08:00
Tong Li
de2ad3b206
fix default eval setting (#6321) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-05-22 11:52:41 +08:00
YeAnbang
78a06f5ce3 fix missing tags parameter 2025-05-21 10:51:32 +08:00
YeAnbang
107470a360 fix logging rollouts 2025-05-17 21:12:58 +08:00
YeAnbang
021914c565 support logging rollouts to wandb 2025-05-16 15:56:03 +08:00
YeAnbang
203dfb1536 address conversation 2025-05-16 14:15:35 +08:00
YeAnbang
6abffb9100 fix evaluation 2025-05-16 09:42:35 +08:00
YeAnbang
957e3a521a disable wandb tb syncing 2025-05-15 16:52:31 +08:00
YeAnbang
4ec73298b2 use consumer global step 2025-05-15 14:15:40 +08:00
YeAnbang
50070c1e84 move logging to producer 2025-05-14 18:10:57 +08:00
YeAnbang
47a7dc7142 Support evaluation during training 2025-05-14 11:03:11 +08:00
Tong Li
b920af427b
update pad seq (#6303) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-05-13 16:51:27 +08:00
YeAnbang
eb6b5dd62e
[fix] revert reward update and evaluation (#6295) 
* Revert "rewrite reward fn"

This reverts commit d06042b434.

* Revert "upgrade reward math verification"

This reverts commit a6085ff676.

* Revert "fix bug"

This reverts commit 01640ebd65.

* Revert "reuse comm-group"

This reverts commit bd61918dcf.

* Revert "Support evaluation during training"

This reverts commit 57a88395fe.
 2025-05-07 10:56:47 +08:00
YeAnbang
01640ebd65 fix bug 2025-04-30 22:53:12 +08:00
YeAnbang
bd61918dcf reuse comm-group 2025-04-30 21:36:11 +08:00
YeAnbang
57a88395fe Support evaluation during training 2025-04-30 18:31:49 +08:00
Tong Li
5fd4bcb9d8
[feat] Sync shard model (#6289) 
* [feat] support hybrid parallel model sync

* update consumer and producer

* update files

* update producer

* remove print

* update

---------

Co-authored-by: duanjunwen <935724073@qq.com>
Co-authored-by: YeAnbang <44796419+YeAnbang@users.noreply.github.com>
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-04-30 14:47:01 +08:00
YeAnbang
2ca1e3c630
fix pp+tp, fix dataloader (#6280) 2025-04-28 17:10:00 +08:00
YeAnbang
26d859f68e
[feat] Support DAPO (#6263) 
* update help information

* update style

* fix

* minor fix

* support PP training

* add pp support

* remove unused code

* address conversation

* fix memory leakage support tp+pp

* move empty cache

* move empty cache

* add DAPO support

* remove format reward

* fix filtering, still buggy

* small fix

* add DAPO support

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tested multi-node training; fix bind_batch bug

* fix conversation; support sleep mode

* support reusing excessive samples

* add dynamic batching control flag

* add dynamic batching control flag

* refactored

* fix logging

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2025-04-25 17:39:17 +08:00
YeAnbang
50153005b4
[feat] add microbatch forwarding (#6251) 
* add microbatch forwarding

* fix forward microbatch

* fix producer OOM

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change project name

* fix temperature annealing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* address conversation

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2025-03-28 10:24:58 +08:00
YeAnbang
2aa7385c88 update logging 2025-03-21 16:12:07 +08:00
YeAnbang
0472f44163 fix logprob, add filtering, temperature annealing, lr descent 2025-03-21 10:24:24 +08:00
Tong Li
45ac6c6cb2 print results 2025-03-13 16:51:22 +08:00
Hongxin Liu
43c9b5fb44
[chat] add distributed impl (#6210) 2025-02-21 15:24:23 +08:00