Files
ColossalAI/applications/ColossalChat/coati/distributed
Tong Li 6a1bd833e0 [feat] Sync shard model (#6289)
* [feat] support hybrid parallel model sync

* update consumer and producer

* update files

* update producer

* remove print

* update

---------

Co-authored-by: duanjunwen <935724073@qq.com>
Co-authored-by: YeAnbang <44796419+YeAnbang@users.noreply.github.com>
Co-authored-by: Tong Li <tong.li35271158@gmail.com>
2025-08-05 13:59:02 +08:00
..
2025-08-05 13:59:01 +08:00
2025-08-05 13:59:02 +08:00
2025-08-05 13:59:02 +08:00
2025-08-05 13:59:02 +08:00
2025-08-05 13:59:02 +08:00
2025-08-05 13:59:01 +08:00
2025-08-05 13:59:02 +08:00

Requirements

pip install cupy-cuda12x
python -m cupyx.tools.install_library --cuda 12.x --library nccl