duanjunwen
d4ef7f57be
[ColossalRL] Support ColossalRL on Ascend ( #6324 )
...
* [fix] support npu
* [feat] multinode 14B
* [feat] enlarge seqlen
* [fix]
* [fix] ready to updated
* [fix] ready to merge grpo-latest
* [fix] rm comments
* [feat] support msprof-analyze, add analsys result
* [feat] support ColossalaiRL on Ascend
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [feat] rm comments in qwen modeling
* [Doc] Drafted README.md
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [feat] fix ascend readme format
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [fix] fix readme
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [fix] fix readme
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [fix] fix Readme, rm irrelevant testcase
* [fix] fix some adapt modification
* [fix] rm comments in modeling qwen
* [fix] rm comm, test and debug print
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: YeAnbang <44796419+YeAnbang@users.noreply.github.com>
2025-06-23 13:56:18 +08:00
Tong Li
8880b83791
add dp rank for multi-dp ( #6351 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-06-19 14:02:08 +08:00
YeAnbang
dd49444dcb
Merge pull request #6348 from hpcaitech/grpo_optimization
...
[Feat] optimize pp log_softmax memory usage
2025-06-19 13:21:51 +08:00
YeAnbang
6b06430ca4
fix small bug
2025-06-19 01:37:52 +00:00
YeAnbang
e3d56cbd86
implement memory efficient logprob
2025-06-18 10:24:48 +00:00
YeAnbang
30a6859f77
optimize pp log_softmax OOM
2025-06-13 18:21:54 +08:00
YeAnbang
0e69b98c28
Merge pull request #6347 from hpcaitech/hotfix/fix_num_update_per_episode
...
[fix] fix num update per episode
2025-06-12 15:56:49 +08:00
YeAnbang
51b7abe9dd
fix num_update_per_episode
2025-06-12 15:06:01 +08:00
YeAnbang
ac069357a9
Merge pull request #6316 from hpcaitech/grpo-support-multi-machine
...
[feat] Manually schedule resources and support auto master address assigning
2025-06-12 11:23:06 +08:00
YeAnbang
2f02a28777
Update README.md
2025-06-12 11:21:31 +08:00
Tong Li
8992def757
fix pp memory issue ( #6344 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-06-11 17:54:18 +08:00
YeAnbang
1330b5753b
add ray timeout handling instruction
2025-06-10 18:21:42 +08:00
YeAnbang
dc29c74632
update readme
2025-06-10 17:17:41 +08:00
YeAnbang
25599246c5
modify readme
2025-06-10 17:00:35 +08:00
YeAnbang
21d517d0fa
Manually schedule resources and support auto master address assigning
2025-06-10 15:00:48 +08:00
Tong Li
bb6f5d98fc
move out evaluation func ( #6343 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-06-10 13:53:19 +08:00
YeAnbang
c308b42f38
Merge pull request #6341 from hpcaitech/grpo-code
...
[feat] Support Code Generation RFT, Move Reward Calculation to Producer
2025-06-09 17:02:13 +08:00
pre-commit-ci[bot]
9ca920c1af
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-06-09 01:48:20 +00:00
YeAnbang
d0e12c508a
remove debug code
2025-06-09 09:42:58 +08:00
YeAnbang
3bed6ae9ee
fix bug, tested
2025-06-09 09:37:28 +08:00
YeAnbang
dc3033e68a
support code generation tasks
2025-06-05 18:05:22 +08:00
YeAnbang
ceb7065d6d
Merge pull request #6312 from hpcaitech/grpo-latest-dev
...
[feat] Move prompt-level-filtering to buffer side
2025-06-05 15:51:38 +08:00
YeAnbang
96faf54542
fix typ and parameter description
2025-06-05 15:41:14 +08:00
pre-commit-ci[bot]
0d008110e7
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-05-29 10:16:56 +00:00
YeAnbang
7b921acc8a
merge grpo-latest
2025-05-29 18:14:43 +08:00
YeAnbang
ee939d9aa5
address conversation
2025-05-29 10:25:59 +08:00
Tong Li
c8b368c294
add overlength sample count ( #6332 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-05-28 19:18:09 +08:00
YeAnbang
58f8c9bb43
Merge branch 'grpo-latest' of https://github.com/hpcaitech/ColossalAI into grpo-latest-dev
2025-05-28 17:34:52 +08:00
YeAnbang
4c3656870a
address conversation
2025-05-28 17:34:11 +08:00
Tong Li
de2ad3b206
fix default eval setting ( #6321 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-05-22 11:52:41 +08:00
YeAnbang
78a06f5ce3
fix missing tags parameter
2025-05-21 10:51:32 +08:00
YeAnbang
88e3b09c79
merge grpo-latest
2025-05-20 18:16:43 +08:00
YeAnbang
37663386bc
fix metric calculation
2025-05-20 18:14:05 +08:00
Tong Li
32afa7bf29
fix empty tensor ( #6319 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-05-20 17:41:44 +08:00
YeAnbang
bcf2459db5
Merge pull request #6314 from hpcaitech/grpo-reward-dev
...
[feat] upgrade reward functions
2025-05-20 10:06:00 +08:00
YeAnbang
f8bd2db33f
add uuid to rollout log
2025-05-20 09:45:56 +08:00
YeAnbang
116621d004
merge reward and eval
2025-05-19 11:53:47 +08:00
YeAnbang
107470a360
fix logging rollouts
2025-05-17 21:12:58 +08:00
YeAnbang
03b41d6fb5
upgrade reward functions
2025-05-16 18:04:38 +08:00
YeAnbang
3c42c0ce82
Merge pull request #6309 from hpcaitech/grpo-eval-dev
...
[feat] Support evaluation during training
2025-05-16 16:11:23 +08:00
YeAnbang
021914c565
support logging rollouts to wandb
2025-05-16 15:56:03 +08:00
YeAnbang
203dfb1536
address conversation
2025-05-16 14:15:35 +08:00
YeAnbang
11a5854b50
remove redundant code and fix bugs
2025-05-16 14:08:23 +08:00
Tong Li
ab95624915
handle empty index ( #6311 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-05-16 10:00:10 +08:00
YeAnbang
6abffb9100
fix evaluation
2025-05-16 09:42:35 +08:00
Tong Li
1644adf684
handle empty index
2025-05-15 20:32:05 +08:00
YeAnbang
a528921944
move prompt-level-filtering to buffer side
2025-05-15 18:30:32 +08:00
YeAnbang
55eee129d2
move prompt-level-filtering to buffer side
2025-05-15 18:16:50 +08:00
YeAnbang
957e3a521a
disable wandb tb syncing
2025-05-15 16:52:31 +08:00
YeAnbang
4ec73298b2
use consumer global step
2025-05-15 14:15:40 +08:00
