github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-07 20:10:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

[example] titans for gpt (#2484)

Browse Source
This commit is contained in:
Jiarui Fang
2023-01-16 15:55:41 +08:00
committed by GitHub
parent 7c31706227
commit 3a21485ead
14 changed files with 1754 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/language/gpt/titans/run.sh Normal file
View File

@@ -0,0 +1,2 @@
export DATA=/data/scratch/gpt_data/small-gpt-dataset.json
colossalai run --nproc_per_node=4 train_gpt.py --config ./configs/gpt2_small_zero3_pp1d.py --from_torch