YeAnbang
25e0062de6
add dynamic batching control flag
2025-04-24 17:45:20 +08:00
YeAnbang
807a5a43b2
support reusing excessive samples
2025-04-23 16:56:12 +08:00
YeAnbang
ca6093a582
merge
2025-04-23 14:46:33 +08:00
YeAnbang
ab55b9f033
fix conversation; support sleep mode
2025-04-23 14:43:54 +08:00
Tong Li
b823c6eec7
[feat] Add final save at the end ( #6274 )
...
* add final save
* default 1 episode
2025-04-23 10:03:46 +08:00
Tong Li
03f4b1dde3
add prompt template ( #6273 )
...
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-04-22 10:39:47 +08:00
YeAnbang
39a1faac87
tested multi-node training; fix bind_batch bug
2025-04-21 17:33:50 +08:00
pre-commit-ci[bot]
e397327de0
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-04-16 10:00:11 +00:00
YeAnbang
c0c0da2f26
add DAPO support
2025-04-16 17:54:14 +08:00
YeAnbang
9474316132
small fix
2025-04-16 15:59:25 +08:00
YeAnbang
cc4faa7300
fix filtering, still buggy
2025-04-16 14:10:43 +08:00
YeAnbang
24c01864df
merge grpo-latest
2025-04-16 10:03:19 +08:00
YeAnbang
447ab74fb4
remove format reward
2025-04-16 09:42:45 +08:00
YeAnbang
6e71e2a3ce
add DAPO support
2025-04-15 18:28:35 +08:00
YeAnbang
9467c10690
[hot-fix] Fix memory leakage bug, support TP+PP ( #6258 )
...
* update help information
* update style
* fix
* minor fix
* support PP training
* add pp support
* remove unused code
* address conversation
* fix memory leakage support tp+pp
* move empty cache
* move empty cache
---------
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-04-10 10:52:18 +08:00
YeAnbang
1723a02860
move empty cache
2025-04-10 10:22:43 +08:00
YeAnbang
f7e532511c
move empty cache
2025-04-10 10:22:28 +08:00
YeAnbang
ff3ca30a35
Merge branch 'grpo-latest' into pp-integration
2025-04-09 17:27:01 +08:00
YeAnbang
1ea3b72c22
fix memory leakage support tp+pp
2025-04-09 17:11:55 +08:00
YeAnbang
ed43a4be04
[Distributed RLHF] Integration of PP ( #6257 )
...
* update help information
* update style
* fix
* minor fix
* support PP training
* add pp support
* remove unused code
* address conversation
---------
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-04-09 13:23:24 +08:00
YeAnbang
a40d82f629
address conversation
2025-04-09 12:53:40 +08:00
YeAnbang
061d8cb3b6
remove unused code
2025-04-04 10:11:11 +08:00
YeAnbang
09a3173a49
add pp support
2025-04-04 10:05:16 +08:00
Tong Li
d961a5f725
support PP training
2025-04-01 11:24:09 +08:00
Tong Li
6604654b2f
minor fix
2025-03-31 13:15:18 +08:00
Tong Li
9d6ede98dc
fix
2025-03-31 13:14:41 +08:00
Tong Li
27d0108c9f
update style
2025-03-31 11:44:10 +08:00
Tong Li
729456957a
update help information
2025-03-31 11:35:23 +08:00
YeAnbang
50153005b4
[feat] add microbatch forwarding ( #6251 )
...
* add microbatch forwarding
* fix forward microbatch
* fix producer OOM
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* change project name
* fix temperature annealing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address conversation
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-03-28 10:24:58 +08:00
YeAnbang
489f215ad9
Merge pull request #6250 from hpcaitech/grpo-latest-dev
...
[feat] Fix Vllm, logprob, add filtering, temperature annealing, lr descent
2025-03-21 16:25:35 +08:00
YeAnbang
2aa7385c88
update logging
2025-03-21 16:12:07 +08:00
YeAnbang
d8eaf0d473
simplify vllm preprocessing input ids
2025-03-21 15:03:10 +08:00
YeAnbang
0472f44163
fix logprob, add filtering, temperature annealing, lr descent
2025-03-21 10:24:24 +08:00
YeAnbang
7ee4452f8c
fix vllm
2025-03-19 17:11:10 +08:00
duanjunwen
7795d4c50d
[Feature] Support Distributed LogProb for GRPO Training ( #6247 )
...
* [fix] fix qwen VocabParallelLMHead1D and gather output
* fix tp bug
* fix consumer
* [feat] Support Distributed LogProb for GRPO Training
* [fix] fix loss func
* [fix] fix log prob plugin
* [fix] fix qwen modeling param
* [fix] rm comments
* [fix] rm hard-code;fix non-dist version
* [fix] fix test file param name and benchmark tp gather output=True/False
* [fix] rm non-dist version in dist log prob
* [fix] fix comments
* [fix] fix dis log prob plugin
* [fix] fix test case
* [fix] fix qwen VocabParallelLMHead1D and gather output
* [fix] fix DistLogProb comments
* [fix] restore tp size
* [fix] fix comments
* [fix] fix comment; fix LogSoftmax usage
---------
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-03-18 17:47:55 +08:00
YeAnbang
bc0171d392
fix transformers backend
2025-03-14 18:12:35 +08:00
Tong Li
57b49da5e4
setup update
2025-03-13 16:52:15 +08:00
Tong Li
45ac6c6cb2
print results
2025-03-13 16:51:22 +08:00
Tong Li
4702d57841
convert to 8 generation
2025-03-13 16:49:02 +08:00
Tong Li
afddfde2dd
fix consumer
2025-03-13 14:55:26 +08:00
Tong Li
131eeceb5d
fix tp bug
2025-03-13 14:52:09 +08:00
Tong Li
704866a240
detach
2025-03-11 16:17:02 +08:00
Tong Li
47d6493778
add response length
2025-03-11 13:06:09 +08:00
Tong Li
abca66e69f
fix reward score
2025-03-11 10:17:32 +08:00
Tong Li
71a0181fce
update reward
2025-03-10 14:19:10 +08:00
Tong Li
754b16dfbf
update reward fn
2025-03-10 14:18:22 +08:00
Tong Li
9d9d51614e
update grpo
2025-03-10 14:12:04 +08:00
pre-commit-ci[bot]
eb6337f07f
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-03-06 08:29:59 +00:00
Tong Li
22cc1558a8
Merge branch 'grpo-latest' of github.com:hpcaitech/ColossalAI into grpo-latest
2025-03-06 16:28:47 +08:00
Tong Li
0590f10fb7
update select algo
2025-03-06 16:27:13 +08:00
