github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-06 19:40:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

[Chat] fix sft loss nan (#5345)

Browse Source 
* fix script

* fix script

* fix chat nan

* fix chat nan
This commit is contained in:
YeAnbang
2024-02-01 14:25:16 +08:00
committed by GitHub
parent abd8e77ad8
commit c5239840e6
2 changed files with 7 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
applications/Chat/examples/train_sft.sh
View File

@@ -25,4 +25,4 @@ torchrun --standalone --nproc_per_node=4 train_sft.py \
--accumulation_steps 8 \
--lr 2e-5 \
--max_datasets_size 512 \
--max_epochs 1
--max_epochs 1