mirror of
https://github.com/hpcaitech/ColossalAI.git
synced 2026-07-12 02:38:52 +00:00
[tutorial] edited hands-on practices (#1899)
* Add handson to ColossalAI. * Change names of handsons and edit sequence parallel example. * Edit wrong folder name * resolve conflict * delete readme
This commit is contained in:
16
examples/tutorial/opt/zero/README.md
Normal file
16
examples/tutorial/opt/zero/README.md
Normal file
@@ -0,0 +1,16 @@
|
||||
## Overview
|
||||
This example shows how to use ColossalAI to run huggingface GPT training with Gemini and ZeRO DDP.
|
||||
|
||||
## GPT
|
||||
We use the huggingface transformers GPT2 model. The input data is randonly generated.
|
||||
|
||||
## Our Modifications
|
||||
We adapt the OPT training code to ColossalAI by leveraging Gemini and ZeRO DDP.
|
||||
|
||||
## Quick Start
|
||||
You can launch training by using the following bash script
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
bash run.sh
|
||||
```
|
||||
Reference in New Issue
Block a user