github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-07-23 03:33:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,913 Commits 57 Branches 51 Tags 236 MiB
grpo-zero-bubble
Commit Graph

3913 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
YeAnbang
5c5cb1863b hotfix 2025-07-21 18:04:20 +08:00
YeAnbang
e774edeb80 fix racing condition 2025-07-21 17:24:11 +08:00
YeAnbang
f54ae56f12 add entropy 2025-07-16 18:08:28 +08:00
YeAnbang
c5e97f4e25 fix code evaluation 2025-07-14 16:25:03 +08:00
YeAnbang
509274c47e add code for zero-bubble implementation 2025-07-09 11:21:43 +08:00
YeAnbang
b1f646c7e7
[feat[ Support one-behind to reduce bubble time. Add profiling code (#6353) 
* support n_behind, add profiling

* fix bugs

* fix visualization

* fix behind

* fix loop issue

* add profiling

* fix update

* update assert

* remove assert

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-06-30 13:21:08 +08:00
Tong Li
8880b83791
add dp rank for multi-dp (#6351) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-06-19 14:02:08 +08:00
YeAnbang
dd49444dcb
Merge pull request #6348 from hpcaitech/grpo_optimization 
[Feat] optimize pp log_softmax memory usage
 2025-06-19 13:21:51 +08:00
YeAnbang
6b06430ca4 fix small bug 2025-06-19 01:37:52 +00:00
YeAnbang
e3d56cbd86 implement memory efficient logprob 2025-06-18 10:24:48 +00:00
YeAnbang
30a6859f77 optimize pp log_softmax OOM 2025-06-13 18:21:54 +08:00
YeAnbang
0e69b98c28
Merge pull request #6347 from hpcaitech/hotfix/fix_num_update_per_episode 
[fix] fix num update per episode
 2025-06-12 15:56:49 +08:00
YeAnbang
51b7abe9dd fix num_update_per_episode 2025-06-12 15:06:01 +08:00
YeAnbang
ac069357a9
Merge pull request #6316 from hpcaitech/grpo-support-multi-machine 
[feat] Manually schedule resources and support auto master address assigning
 2025-06-12 11:23:06 +08:00
YeAnbang
2f02a28777
Update README.md 2025-06-12 11:21:31 +08:00
Tong Li
8992def757
fix pp memory issue (#6344) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-06-11 17:54:18 +08:00
YeAnbang
1330b5753b add ray timeout handling instruction 2025-06-10 18:21:42 +08:00
YeAnbang
dc29c74632 update readme 2025-06-10 17:17:41 +08:00
YeAnbang
25599246c5 modify readme 2025-06-10 17:00:35 +08:00
YeAnbang
21d517d0fa Manually schedule resources and support auto master address assigning 2025-06-10 15:00:48 +08:00
Tong Li
bb6f5d98fc
move out evaluation func (#6343) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-06-10 13:53:19 +08:00
YeAnbang
c308b42f38
Merge pull request #6341 from hpcaitech/grpo-code 
[feat] Support Code Generation RFT, Move Reward Calculation to Producer
 2025-06-09 17:02:13 +08:00
pre-commit-ci[bot]
9ca920c1af [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2025-06-09 01:48:20 +00:00
YeAnbang
d0e12c508a remove debug code 2025-06-09 09:42:58 +08:00
YeAnbang
3bed6ae9ee fix bug, tested 2025-06-09 09:37:28 +08:00
YeAnbang
dc3033e68a support code generation tasks 2025-06-05 18:05:22 +08:00
YeAnbang
ceb7065d6d
Merge pull request #6312 from hpcaitech/grpo-latest-dev 
[feat] Move prompt-level-filtering to buffer side
 2025-06-05 15:51:38 +08:00
YeAnbang
96faf54542 fix typ and parameter description 2025-06-05 15:41:14 +08:00
pre-commit-ci[bot]
0d008110e7 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2025-05-29 10:16:56 +00:00
YeAnbang
7b921acc8a merge grpo-latest 2025-05-29 18:14:43 +08:00
YeAnbang
ee939d9aa5 address conversation 2025-05-29 10:25:59 +08:00
Tong Li
c8b368c294
add overlength sample count (#6332) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-05-28 19:18:09 +08:00
YeAnbang
58f8c9bb43 Merge branch 'grpo-latest' of https://github.com/hpcaitech/ColossalAI into grpo-latest-dev 2025-05-28 17:34:52 +08:00
YeAnbang
4c3656870a address conversation 2025-05-28 17:34:11 +08:00
Tong Li
de2ad3b206
fix default eval setting (#6321) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-05-22 11:52:41 +08:00
YeAnbang
78a06f5ce3 fix missing tags parameter 2025-05-21 10:51:32 +08:00
YeAnbang
88e3b09c79 merge grpo-latest 2025-05-20 18:16:43 +08:00
YeAnbang
37663386bc fix metric calculation 2025-05-20 18:14:05 +08:00
Tong Li
32afa7bf29
fix empty tensor (#6319) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-05-20 17:41:44 +08:00
YeAnbang
bcf2459db5
Merge pull request #6314 from hpcaitech/grpo-reward-dev 
[feat] upgrade reward functions
 2025-05-20 10:06:00 +08:00
YeAnbang
f8bd2db33f add uuid to rollout log 2025-05-20 09:45:56 +08:00
YeAnbang
116621d004 merge reward and eval 2025-05-19 11:53:47 +08:00
YeAnbang
107470a360 fix logging rollouts 2025-05-17 21:12:58 +08:00
YeAnbang
03b41d6fb5 upgrade reward functions 2025-05-16 18:04:38 +08:00
YeAnbang
3c42c0ce82
Merge pull request #6309 from hpcaitech/grpo-eval-dev 
[feat] Support evaluation during training
 2025-05-16 16:11:23 +08:00
YeAnbang
021914c565 support logging rollouts to wandb 2025-05-16 15:56:03 +08:00
YeAnbang
203dfb1536 address conversation 2025-05-16 14:15:35 +08:00
YeAnbang
11a5854b50 remove redundant code and fix bugs 2025-05-16 14:08:23 +08:00
Tong Li
ab95624915
handle empty index (#6311) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-05-16 10:00:10 +08:00
YeAnbang
6abffb9100 fix evaluation 2025-05-16 09:42:35 +08:00