mirror of
https://github.com/hpcaitech/ColossalAI.git
synced 2026-04-25 09:12:10 +00:00
ca6e75bc28
* Add handson to ColossalAI. * Change names of handsons and edit sequence parallel example. * Edit wrong folder name * resolve conflict * delete readme
17 lines
448 B
Markdown
17 lines
448 B
Markdown
# Handson 4: Comparison of Large Batch Training Optimization
|
|
|
|
## Prepare Dataset
|
|
|
|
We use CIFAR10 dataset in this example. The dataset will be downloaded to `../data` by default.
|
|
If you wish to use customized directory for the dataset. You can set the environment variable `DATA` via the following command.
|
|
|
|
```bash
|
|
export DATA=/path/to/data
|
|
```
|
|
|
|
|
|
## Run on 2*2 device mesh
|
|
|
|
```bash
|
|
colossalai run --nproc_per_node 4 train.py --config config.py
|
|
``` |