github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-07-30 15:00:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,879 Commits 57 Branches 51 Tags 236 MiB
ad1ceb0424
Commit Graph

3879 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
xysheng-colossal
ad1ceb0424 [feat] Update sync model by tensor, fix tMbs problem, add qwen train benchmark. 2025-07-17 16:29:26 +08:00
xysheng-colossal
d9b5f10d82 [feat][merge] Support one-behind to reduce bubble time. Add profiling code. 2025-06-27 17:35:51 +08:00
xysheng-colossal
9379a89677
[feat][npu] Merge form grpo-latest (#6346) 
* move prompt-level-filtering to buffer side

* move prompt-level-filtering to buffer side

* remove redundant code and fix bugs

* fix metric calculation

* fix missing tags parameter

* address conversation

* add overlength sample count (#6332)

Co-authored-by: Tong Li <tong.li35271158@gmail.com>

* address conversation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix typ and parameter description

* [feat] Update requriments and set return logits False

---------

Co-authored-by: YeAnbang <anbangy2@outlook.com>
Co-authored-by: Tong Li <tong.li352711588@gmail.com>
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2025-06-23 11:49:13 +08:00
duanjunwen
84f523a080
[Hotfix] fix requirsments (#6338) 
* [fix] fix colossalai ascend requirments

* [fix] fix colossalai chat requirements

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [fix] fix requirments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2025-06-04 09:51:31 +08:00
Tong Li
180cea709b update to conform to json format 2025-05-28 13:13:21 +08:00
Tong Li
f4e3063dc3
[Ascend] Update README (#6331) 
* update readme

* [fix] add vllm & vllm-ascend installation

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
Co-authored-by: duanjunwen <935724073@qq.com>
 2025-05-28 11:35:35 +08:00
duanjunwen
e1ca2d22ae
[ColossalRL] Support ColossalRL on Ascend (#6324) 
* [fix] support npu

* [feat] multinode 14B

* [feat] enlarge seqlen

* [fix]

* [fix] ready to updated

* [fix] ready to merge grpo-latest

* [fix] rm comments

* [feat] support msprof-analyze, add analsys result

* [feat] support ColossalaiRL on Ascend

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [feat] rm comments in qwen modeling

* [Doc] Drafted README.md

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [feat] fix ascend readme format

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [fix] fix readme

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [fix] fix readme

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [fix] fix Readme, rm irrelevant testcase

* [fix] fix some adapt modification

* [fix] rm comments in modeling qwen

* [fix] rm comm, test and debug print

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: YeAnbang <44796419+YeAnbang@users.noreply.github.com>
 2025-05-28 10:43:13 +08:00
Tong Li
de2ad3b206
fix default eval setting (#6321) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-05-22 11:52:41 +08:00
Tong Li
32afa7bf29
fix empty tensor (#6319) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-05-20 17:41:44 +08:00
YeAnbang
bcf2459db5
Merge pull request #6314 from hpcaitech/grpo-reward-dev 
[feat] upgrade reward functions
 2025-05-20 10:06:00 +08:00
YeAnbang
f8bd2db33f add uuid to rollout log 2025-05-20 09:45:56 +08:00
YeAnbang
107470a360 fix logging rollouts 2025-05-17 21:12:58 +08:00
YeAnbang
03b41d6fb5 upgrade reward functions 2025-05-16 18:04:38 +08:00
YeAnbang
3c42c0ce82
Merge pull request #6309 from hpcaitech/grpo-eval-dev 
[feat] Support evaluation during training
 2025-05-16 16:11:23 +08:00
YeAnbang
021914c565 support logging rollouts to wandb 2025-05-16 15:56:03 +08:00
YeAnbang
203dfb1536 address conversation 2025-05-16 14:15:35 +08:00
Tong Li
ab95624915
handle empty index (#6311) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-05-16 10:00:10 +08:00
YeAnbang
6abffb9100 fix evaluation 2025-05-16 09:42:35 +08:00
Tong Li
1644adf684 handle empty index 2025-05-15 20:32:05 +08:00
YeAnbang
957e3a521a disable wandb tb syncing 2025-05-15 16:52:31 +08:00
YeAnbang
4ec73298b2 use consumer global step 2025-05-15 14:15:40 +08:00
YeAnbang
094f119b3a merge 2025-05-14 18:13:47 +08:00
YeAnbang
50070c1e84 move logging to producer 2025-05-14 18:10:57 +08:00
Tong Li
aca547623f
[feat] Support prompt level dynamic (#6300) 
* adjust to dynamic prompt bs

* remove debug

* update pad seq (#6303)

Co-authored-by: Tong Li <tong.li35271158@gmail.com>

* adjust to dynamic prompt bs

* remove debug

* fix dp issue

* fix

* fix default settings

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-05-14 16:40:35 +08:00
YeAnbang
47a7dc7142 Support evaluation during training 2025-05-14 11:03:11 +08:00
Tong Li
b920af427b
update pad seq (#6303) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-05-13 16:51:27 +08:00
YeAnbang
eb6b5dd62e
[fix] revert reward update and evaluation (#6295) 
* Revert "rewrite reward fn"

This reverts commit d06042b434.

* Revert "upgrade reward math verification"

This reverts commit a6085ff676.

* Revert "fix bug"

This reverts commit 01640ebd65.

* Revert "reuse comm-group"

This reverts commit bd61918dcf.

* Revert "Support evaluation during training"

This reverts commit 57a88395fe.
 2025-05-07 10:56:47 +08:00
YeAnbang
17928ad84f
Merge pull request #6292 from hpcaitech/grpo-latest-dev-reward-update 
[feat] Update reward verification
 2025-05-03 10:00:32 +08:00
YeAnbang
d06042b434 rewrite reward fn 2025-05-01 11:28:05 +08:00
YeAnbang
a6085ff676 upgrade reward math verification 2025-04-30 22:59:54 +08:00
YeAnbang
01640ebd65 fix bug 2025-04-30 22:53:12 +08:00
YeAnbang
bd61918dcf reuse comm-group 2025-04-30 21:36:11 +08:00
YeAnbang
57a88395fe Support evaluation during training 2025-04-30 18:31:49 +08:00
Tong Li
5fd4bcb9d8
[feat] Sync shard model (#6289) 
* [feat] support hybrid parallel model sync

* update consumer and producer

* update files

* update producer

* remove print

* update

---------

Co-authored-by: duanjunwen <935724073@qq.com>
Co-authored-by: YeAnbang <44796419+YeAnbang@users.noreply.github.com>
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-04-30 14:47:01 +08:00
YeAnbang
14f237ce7e
[feat] Support boxed math reward (#6284) 
* fix pp+tp, fix dataloader

* fixed plugin micro-batch size

* support boxed reward

* add boxed reward

* fix pp state dict incomplete issue

* Revert "fix pp state dict incomplete issue"

This reverts commit 6c1b3b694f.
 2025-04-29 16:46:47 +08:00
YeAnbang
2ca1e3c630
fix pp+tp, fix dataloader (#6280) 2025-04-28 17:10:00 +08:00
Tong Li
28795f560c
fix save issue (#6279) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-04-27 17:54:06 +08:00
YeAnbang
38008858e4
fix checkpoint naming; add num_epoch parameter (#6277) 2025-04-26 14:00:28 +08:00
YeAnbang
26d859f68e
[feat] Support DAPO (#6263) 
* update help information

* update style

* fix

* minor fix

* support PP training

* add pp support

* remove unused code

* address conversation

* fix memory leakage support tp+pp

* move empty cache

* move empty cache

* add DAPO support

* remove format reward

* fix filtering, still buggy

* small fix

* add DAPO support

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tested multi-node training; fix bind_batch bug

* fix conversation; support sleep mode

* support reusing excessive samples

* add dynamic batching control flag

* add dynamic batching control flag

* refactored

* fix logging

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2025-04-25 17:39:17 +08:00
Tong Li
b823c6eec7
[feat] Add final save at the end (#6274) 
* add final save

* default 1 episode
 2025-04-23 10:03:46 +08:00
Tong Li
03f4b1dde3
add prompt template (#6273) 
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-04-22 10:39:47 +08:00
YeAnbang
9467c10690
[hot-fix] Fix memory leakage bug, support TP+PP (#6258) 
* update help information

* update style

* fix

* minor fix

* support PP training

* add pp support

* remove unused code

* address conversation

* fix memory leakage support tp+pp

* move empty cache

* move empty cache

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-04-10 10:52:18 +08:00
YeAnbang
ed43a4be04
[Distributed RLHF] Integration of PP (#6257) 
* update help information

* update style

* fix

* minor fix

* support PP training

* add pp support

* remove unused code

* address conversation

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-04-09 13:23:24 +08:00
YeAnbang
50153005b4
[feat] add microbatch forwarding (#6251) 
* add microbatch forwarding

* fix forward microbatch

* fix producer OOM

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change project name

* fix temperature annealing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* address conversation

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2025-03-28 10:24:58 +08:00
YeAnbang
489f215ad9
Merge pull request #6250 from hpcaitech/grpo-latest-dev 
[feat] Fix Vllm, logprob, add filtering, temperature annealing, lr descent
 2025-03-21 16:25:35 +08:00
YeAnbang
2aa7385c88 update logging 2025-03-21 16:12:07 +08:00
YeAnbang
d8eaf0d473 simplify vllm preprocessing input ids 2025-03-21 15:03:10 +08:00
YeAnbang
0472f44163 fix logprob, add filtering, temperature annealing, lr descent 2025-03-21 10:24:24 +08:00
YeAnbang
7ee4452f8c fix vllm 2025-03-19 17:11:10 +08:00
duanjunwen
7795d4c50d
[Feature] Support Distributed LogProb for GRPO Training (#6247) 
* [fix] fix qwen VocabParallelLMHead1D and gather output

* fix tp bug

* fix consumer

* [feat] Support Distributed LogProb for GRPO Training

* [fix] fix loss func

* [fix] fix log prob plugin

* [fix] fix qwen modeling param

* [fix] rm comments

* [fix] rm hard-code;fix non-dist version

* [fix] fix test file param name and benchmark tp gather output=True/False

* [fix] rm non-dist version in dist log prob

* [fix] fix comments

* [fix] fix dis log prob plugin

* [fix] fix test case

* [fix] fix qwen VocabParallelLMHead1D and gather output

* [fix] fix DistLogProb comments

* [fix] restore tp size

* [fix] fix comments

* [fix] fix comment; fix LogSoftmax usage

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-03-18 17:47:55 +08:00