YeAnbang
cd32236e53
[Fix] Add L2 Regularization ( #6372 )
...
* fix no L2 regularization error
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-07-29 16:56:52 +08:00
YeAnbang
57e92104a2
hotfix entropy calculation ( #6364 )
2025-07-22 10:02:02 +08:00
YeAnbang
4cf5ce20bf
add entropy ( #6363 )
2025-07-17 15:05:10 +08:00
YeAnbang
f5c155ab48
Merge pull request #6361 from hpcaitech/grpo-latest-fix-code-reward
...
[fix] Grpo latest fix code reward
2025-07-14 18:25:23 +08:00
YeAnbang
d850475208
fix style
2025-07-14 18:23:39 +08:00
YeAnbang
a992da9f0f
fix code evaluation
2025-07-14 16:26:59 +08:00
YeAnbang
b1f646c7e7
[feat[ Support one-behind to reduce bubble time. Add profiling code ( #6353 )
...
* support n_behind, add profiling
* fix bugs
* fix visualization
* fix behind
* fix loop issue
* add profiling
* fix update
* update assert
* remove assert
---------
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-06-30 13:21:08 +08:00
Tong Li
8880b83791
add dp rank for multi-dp ( #6351 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-06-19 14:02:08 +08:00
YeAnbang
dd49444dcb
Merge pull request #6348 from hpcaitech/grpo_optimization
...
[Feat] optimize pp log_softmax memory usage
2025-06-19 13:21:51 +08:00
YeAnbang
6b06430ca4
fix small bug
2025-06-19 01:37:52 +00:00
YeAnbang
e3d56cbd86
implement memory efficient logprob
2025-06-18 10:24:48 +00:00
YeAnbang
30a6859f77
optimize pp log_softmax OOM
2025-06-13 18:21:54 +08:00
YeAnbang
0e69b98c28
Merge pull request #6347 from hpcaitech/hotfix/fix_num_update_per_episode
...
[fix] fix num update per episode
2025-06-12 15:56:49 +08:00
YeAnbang
51b7abe9dd
fix num_update_per_episode
2025-06-12 15:06:01 +08:00
YeAnbang
ac069357a9
Merge pull request #6316 from hpcaitech/grpo-support-multi-machine
...
[feat] Manually schedule resources and support auto master address assigning
2025-06-12 11:23:06 +08:00
YeAnbang
2f02a28777
Update README.md
2025-06-12 11:21:31 +08:00
Tong Li
8992def757
fix pp memory issue ( #6344 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-06-11 17:54:18 +08:00
YeAnbang
1330b5753b
add ray timeout handling instruction
2025-06-10 18:21:42 +08:00
YeAnbang
dc29c74632
update readme
2025-06-10 17:17:41 +08:00
YeAnbang
25599246c5
modify readme
2025-06-10 17:00:35 +08:00
YeAnbang
21d517d0fa
Manually schedule resources and support auto master address assigning
2025-06-10 15:00:48 +08:00
Tong Li
bb6f5d98fc
move out evaluation func ( #6343 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-06-10 13:53:19 +08:00
YeAnbang
c308b42f38
Merge pull request #6341 from hpcaitech/grpo-code
...
[feat] Support Code Generation RFT, Move Reward Calculation to Producer
2025-06-09 17:02:13 +08:00
pre-commit-ci[bot]
9ca920c1af
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-06-09 01:48:20 +00:00
YeAnbang
d0e12c508a
remove debug code
2025-06-09 09:42:58 +08:00
YeAnbang
3bed6ae9ee
fix bug, tested
2025-06-09 09:37:28 +08:00
YeAnbang
dc3033e68a
support code generation tasks
2025-06-05 18:05:22 +08:00
YeAnbang
ceb7065d6d
Merge pull request #6312 from hpcaitech/grpo-latest-dev
...
[feat] Move prompt-level-filtering to buffer side
2025-06-05 15:51:38 +08:00
YeAnbang
96faf54542
fix typ and parameter description
2025-06-05 15:41:14 +08:00
pre-commit-ci[bot]
0d008110e7
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-05-29 10:16:56 +00:00
YeAnbang
7b921acc8a
merge grpo-latest
2025-05-29 18:14:43 +08:00
YeAnbang
ee939d9aa5
address conversation
2025-05-29 10:25:59 +08:00
Tong Li
c8b368c294
add overlength sample count ( #6332 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-05-28 19:18:09 +08:00
YeAnbang
58f8c9bb43
Merge branch 'grpo-latest' of https://github.com/hpcaitech/ColossalAI into grpo-latest-dev
2025-05-28 17:34:52 +08:00
YeAnbang
4c3656870a
address conversation
2025-05-28 17:34:11 +08:00
Tong Li
de2ad3b206
fix default eval setting ( #6321 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-05-22 11:52:41 +08:00
YeAnbang
78a06f5ce3
fix missing tags parameter
2025-05-21 10:51:32 +08:00
YeAnbang
88e3b09c79
merge grpo-latest
2025-05-20 18:16:43 +08:00
YeAnbang
37663386bc
fix metric calculation
2025-05-20 18:14:05 +08:00
Tong Li
32afa7bf29
fix empty tensor ( #6319 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-05-20 17:41:44 +08:00
YeAnbang
bcf2459db5
Merge pull request #6314 from hpcaitech/grpo-reward-dev
...
[feat] upgrade reward functions
2025-05-20 10:06:00 +08:00
YeAnbang
f8bd2db33f
add uuid to rollout log
2025-05-20 09:45:56 +08:00
YeAnbang
116621d004
merge reward and eval
2025-05-19 11:53:47 +08:00
YeAnbang
107470a360
fix logging rollouts
2025-05-17 21:12:58 +08:00
YeAnbang
03b41d6fb5
upgrade reward functions
2025-05-16 18:04:38 +08:00
YeAnbang
3c42c0ce82
Merge pull request #6309 from hpcaitech/grpo-eval-dev
...
[feat] Support evaluation during training
2025-05-16 16:11:23 +08:00
YeAnbang
021914c565
support logging rollouts to wandb
2025-05-16 15:56:03 +08:00
YeAnbang
203dfb1536
address conversation
2025-05-16 14:15:35 +08:00
YeAnbang
11a5854b50
remove redundant code and fix bugs
2025-05-16 14:08:23 +08:00
Tong Li
ab95624915
handle empty index ( #6311 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-05-16 10:00:10 +08:00
