Tong Li
1644adf684
handle empty index
2025-05-15 20:32:05 +08:00
YeAnbang
094f119b3a
merge
2025-05-14 18:13:47 +08:00
YeAnbang
50070c1e84
move logging to producer
2025-05-14 18:10:57 +08:00
Tong Li
aca547623f
[feat] Support prompt level dynamic ( #6300 )
...
* adjust to dynamic prompt bs
* remove debug
* update pad seq (#6303 )
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
* adjust to dynamic prompt bs
* remove debug
* fix dp issue
* fix
* fix default settings
---------
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-05-14 16:40:35 +08:00
YeAnbang
47a7dc7142
Support evaluation during training
2025-05-14 11:03:11 +08:00
Tong Li
b920af427b
update pad seq ( #6303 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-05-13 16:51:27 +08:00
YeAnbang
eb6b5dd62e
[fix] revert reward update and evaluation ( #6295 )
...
* Revert "rewrite reward fn"
This reverts commit d06042b434a6085ff67601640ebd65bd61918dcf57a88395fe
2025-05-07 10:56:47 +08:00
YeAnbang
d06042b434
rewrite reward fn
2025-05-01 11:28:05 +08:00
YeAnbang
bd61918dcf
reuse comm-group
2025-04-30 21:36:11 +08:00
YeAnbang
57a88395fe
Support evaluation during training
2025-04-30 18:31:49 +08:00
Tong Li
5fd4bcb9d8
[feat] Sync shard model ( #6289 )
...
* [feat] support hybrid parallel model sync
* update consumer and producer
* update files
* update producer
* remove print
* update
---------
Co-authored-by: duanjunwen <935724073@qq.com>
Co-authored-by: YeAnbang <44796419+YeAnbang@users.noreply.github.com>
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-04-30 14:47:01 +08:00
YeAnbang
14f237ce7e
[feat] Support boxed math reward ( #6284 )
...
* fix pp+tp, fix dataloader
* fixed plugin micro-batch size
* support boxed reward
* add boxed reward
* fix pp state dict incomplete issue
* Revert "fix pp state dict incomplete issue"
This reverts commit 6c1b3b694f
2025-04-29 16:46:47 +08:00
YeAnbang
2ca1e3c630
fix pp+tp, fix dataloader ( #6280 )
2025-04-28 17:10:00 +08:00
YeAnbang
38008858e4
fix checkpoint naming; add num_epoch parameter ( #6277 )
2025-04-26 14:00:28 +08:00
YeAnbang
26d859f68e
[feat] Support DAPO ( #6263 )
...
* update help information
* update style
* fix
* minor fix
* support PP training
* add pp support
* remove unused code
* address conversation
* fix memory leakage support tp+pp
* move empty cache
* move empty cache
* add DAPO support
* remove format reward
* fix filtering, still buggy
* small fix
* add DAPO support
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tested multi-node training; fix bind_batch bug
* fix conversation; support sleep mode
* support reusing excessive samples
* add dynamic batching control flag
* add dynamic batching control flag
* refactored
* fix logging
---------
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-04-25 17:39:17 +08:00
Tong Li
b823c6eec7
[feat] Add final save at the end ( #6274 )
...
* add final save
* default 1 episode
2025-04-23 10:03:46 +08:00
YeAnbang
9467c10690
[hot-fix] Fix memory leakage bug, support TP+PP ( #6258 )
...
* update help information
* update style
* fix
* minor fix
* support PP training
* add pp support
* remove unused code
* address conversation
* fix memory leakage support tp+pp
* move empty cache
* move empty cache
---------
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-04-10 10:52:18 +08:00
YeAnbang
ed43a4be04
[Distributed RLHF] Integration of PP ( #6257 )
...
* update help information
* update style
* fix
* minor fix
* support PP training
* add pp support
* remove unused code
* address conversation
---------
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-04-09 13:23:24 +08:00
YeAnbang
50153005b4
[feat] add microbatch forwarding ( #6251 )
...
* add microbatch forwarding
* fix forward microbatch
* fix producer OOM
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* change project name
* fix temperature annealing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address conversation
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-03-28 10:24:58 +08:00
YeAnbang
0472f44163
fix logprob, add filtering, temperature annealing, lr descent
2025-03-21 10:24:24 +08:00
duanjunwen
7795d4c50d
[Feature] Support Distributed LogProb for GRPO Training ( #6247 )
...
* [fix] fix qwen VocabParallelLMHead1D and gather output
* fix tp bug
* fix consumer
* [feat] Support Distributed LogProb for GRPO Training
* [fix] fix loss func
* [fix] fix log prob plugin
* [fix] fix qwen modeling param
* [fix] rm comments
* [fix] rm hard-code;fix non-dist version
* [fix] fix test file param name and benchmark tp gather output=True/False
* [fix] rm non-dist version in dist log prob
* [fix] fix comments
* [fix] fix dis log prob plugin
* [fix] fix test case
* [fix] fix qwen VocabParallelLMHead1D and gather output
* [fix] fix DistLogProb comments
* [fix] restore tp size
* [fix] fix comments
* [fix] fix comment; fix LogSoftmax usage
---------
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-03-18 17:47:55 +08:00
Tong Li
afddfde2dd
fix consumer
2025-03-13 14:55:26 +08:00
pre-commit-ci[bot]
eb6337f07f
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-03-06 08:29:59 +00:00
Tong Li
0cc0c843ed
add save
2025-03-06 16:26:14 +08:00
Hongxin Liu
de282dd694
[feature] fit RL style generation ( #6213 )
...
* [feature] fit rl style generation
* [doc] add docstr
* [doc] add docstr
2025-02-21 17:28:19 +08:00
Hongxin Liu
43c9b5fb44
[chat] add distributed impl ( #6210 )
2025-02-21 15:24:23 +08:00
