github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-06 11:32:10 +00:00
Code Issues Packages Projects Releases Wiki Activity

[ColossalChat] Hotfix for ColossalChat (#5910)

Browse Source 
* add ignore and tiny llama

* fix path issue

* run style

* fix issue

* update bash

* add ignore and tiny llama

* fix path issue

* run style

* fix issue

* update bash

* fix ddp issue

* add Qwen 1.5 32B
This commit is contained in:
Tong Li
2024-07-19 13:40:07 +08:00
committed by GitHub
parent 8cc8f645cd
commit f585d4e38e
11 changed files with 42 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
applications/ColossalChat/examples/training_scripts/train_sft.py
View File

@@ -61,7 +61,7 @@ def train(args):
Default torch ddp plugin without any acceleration, for
debugging purpose acceleration, for debugging purpose
"""
plugin = TorchDDPPlugin(find_unused_parameters=True)
plugin = TorchDDPPlugin(find_unused_parameters=True if args.grad_checkpoint is False else False)
elif args.plugin == "gemini":
plugin = GeminiPlugin(
precision=args.mixed_precision,