github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-14 13:42:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

[example] titans for gpt (#2484)

Browse Source
This commit is contained in:
Jiarui Fang
2023-01-16 15:55:41 +08:00
committed by GitHub
parent 7c31706227
commit 3a21485ead
14 changed files with 1754 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
examples/language/gpt/titans/test_ci.sh Normal file
View File

@@ -0,0 +1 @@
colossalai run --nproc_per_node=4 train_gpt.py --config ./configs/gpt2_small_zero3_pp1d.py --from_torch --use_dummy_dataset