github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-11-26 00:45:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,964 Commits 60 Branches 51 Tags
edcef9edaff712882e8f5dc152cda5121bce62cd
Commit Graph

30 Commits

Author SHA1 Message Date
YeAnbang
edcef9edaf add custom agentic producer 2025-09-19 10:00:07 +08:00
YeAnbang
62f82a75ae add langgraph agent, still buggy 2025-09-19 10:00:06 +08:00
YeAnbang
f3155409b5 support agentic with asyncllm 2025-09-19 09:59:27 +08:00
YeAnbang
ae51e5b244 support asyncllm 2025-09-19 09:58:36 +08:00
sglucas
083766d54c Add new implementations of RL algorithms (#6383) 
* add new algorithm

* move common calculations

* delete data

* move common calculations of rewards

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2025-09-03 13:48:06 +08:00
YeAnbang
594c2c6522 [feat[ Support one-behind to reduce bubble time. Add profiling code (#6353) 
* support n_behind, add profiling

* fix bugs

* fix visualization

* fix behind

* fix loop issue

* add profiling

* fix update

* update assert

* remove assert

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-08-05 14:04:10 +08:00
YeAnbang
0f71c79760 fix num_update_per_episode 2025-08-05 14:04:09 +08:00
YeAnbang
3b3c48d9a8 Manually schedule resources and support auto master address assigning 2025-08-05 14:04:09 +08:00
YeAnbang
177144794b support code generation tasks 2025-08-05 14:04:09 +08:00
YeAnbang
4b1c515f52 fix missing tags parameter 2025-08-05 14:01:45 +08:00
YeAnbang
70c3daa4ee add uuid to rollout log 2025-08-05 14:01:43 +08:00
YeAnbang
06cfbe313b fix metric calculation 2025-08-05 14:01:20 +08:00
YeAnbang
c7c73df60a fix logging rollouts 2025-08-05 14:01:20 +08:00
YeAnbang
88f49ddc5e remove redundant code and fix bugs 2025-08-05 13:59:56 +08:00
YeAnbang
280aa0b830 use consumer global step 2025-08-05 13:59:56 +08:00
YeAnbang
3416a4fc9c move logging to producer 2025-08-05 13:59:03 +08:00
YeAnbang
af4366f0cb Support evaluation during training 2025-08-05 13:59:03 +08:00
YeAnbang
9544c51a74 [fix] revert reward update and evaluation (#6295) 
* Revert "rewrite reward fn"

This reverts commit d06042b434.

* Revert "upgrade reward math verification"

This reverts commit a6085ff676.

* Revert "fix bug"

This reverts commit 01640ebd65.

* Revert "reuse comm-group"

This reverts commit bd61918dcf.

* Revert "Support evaluation during training"

This reverts commit 57a88395fe.
 2025-08-05 13:59:02 +08:00
YeAnbang
1be993de3e fix bug 2025-08-05 13:59:02 +08:00
YeAnbang
16600f3509 Support evaluation during training 2025-08-05 13:59:02 +08:00
Tong Li
6a1bd833e0 [feat] Sync shard model (#6289) 
* [feat] support hybrid parallel model sync

* update consumer and producer

* update files

* update producer

* remove print

* update

---------

Co-authored-by: duanjunwen <935724073@qq.com>
Co-authored-by: YeAnbang <44796419+YeAnbang@users.noreply.github.com>
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-08-05 13:59:02 +08:00
YeAnbang
fb4e507d00 fix pp+tp, fix dataloader (#6280) 2025-08-05 13:59:02 +08:00
YeAnbang
5f913e8b77 [feat] Support DAPO (#6263) 
* update help information

* update style

* fix

* minor fix

* support PP training

* add pp support

* remove unused code

* address conversation

* fix memory leakage support tp+pp

* move empty cache

* move empty cache

* add DAPO support

* remove format reward

* fix filtering, still buggy

* small fix

* add DAPO support

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tested multi-node training; fix bind_batch bug

* fix conversation; support sleep mode

* support reusing excessive samples

* add dynamic batching control flag

* add dynamic batching control flag

* refactored

* fix logging

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2025-08-05 13:59:02 +08:00
YeAnbang
5d79b9e692 [Distributed RLHF] Integration of PP (#6257) 
* update help information

* update style

* fix

* minor fix

* support PP training

* add pp support

* remove unused code

* address conversation

---------

Co-authored-by: Tong Li <tong.li35271158@gmail.com>
 2025-08-05 13:59:02 +08:00
YeAnbang
12da4d14aa [feat] add microbatch forwarding (#6251) 
* add microbatch forwarding

* fix forward microbatch

* fix producer OOM

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change project name

* fix temperature annealing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* address conversation

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2025-08-05 13:59:02 +08:00
YeAnbang
16e68a071d fix logprob, add filtering, temperature annealing, lr descent 2025-08-05 13:59:02 +08:00
YeAnbang
f983071b10 fix vllm 2025-08-05 13:59:02 +08:00
Tong Li
1f15dc70df add algo selection 2025-08-05 13:59:01 +08:00
Tong Li
40d601802d add simple grpo 2025-08-05 13:59:01 +08:00
Hongxin Liu
162bb42321 [chat] add distributed impl (#6210) 2025-08-05 13:59:01 +08:00