add Chinese README

This commit is contained in:
binmakeswell
2022-02-18 16:28:37 +08:00
committed by Frank Lee
parent 82023779bb
commit 753035edd3
5 changed files with 253 additions and 18 deletions

View File

@@ -13,9 +13,52 @@
[![Build](https://github.com/hpcaitech/ColossalAI/actions/workflows/PR_CI.yml/badge.svg)](https://github.com/hpcaitech/ColossalAI/actions/workflows/PR_CI.yml)
[![Documentation](https://readthedocs.org/projects/colossalai/badge/?version=latest)](https://colossalai.readthedocs.io/en/latest/?badge=latest)
[![codebeat badge](https://codebeat.co/badges/bfe8f98b-5d61-4256-8ad2-ccd34d9cc156)](https://codebeat.co/projects/github-com-hpcaitech-colossalai-main)
| [English](README.md) | [中文](README-zh-Hans.md) |
</div>
An integrated large-scale model training system with efficient parallelization techniques.
## Features
Colossal-AI provides a collection of parallel training components for you. We aim to support you to write your
distributed deep learning models just like how you write your single-GPU model. We provide friendly tools to kickstart
distributed training in a few lines.
- Data Parallelism
- Pipeline Parallelism
- 1D, 2D, 2.5D, 3D tensor parallelism
- Sequence parallelism
- Friendly trainer and engine
- Extensible for new parallelism
- Mixed Precision Training
- Zero Redundancy Optimizer (ZeRO)
## Examples
### ViT
<img src="./docs/images/ViT_TP.png" width="400" />
- 14x larger batch size
- 5x faster training
### GPT-3 & GPT-2
![GPT_2_3](./docs/images/GPT_2_3.png)
- Free 50% GPU resources, or 10.7% acceleration for GPT-3
- 11x lower GPU RAM, or superlinear scaling for GPT-2
### BERT
![BERT_seq](./docs/images/BERT_seq.png)
- 2x faster training
- 50% longer sequence length
Please visit our [documentation and tutorials](https://www.colossalai.org/) for more details.
## Installation
### PyPI
@@ -37,7 +80,7 @@ pip install colossalai[zero]
### Install From Source
> The documentation will be in line with the main branch of the repository. Feel free to raise an issue if you encounter any problem. :)
> The version of Colossal-AI will be in line with the main branch of the repository. Feel free to raise an issue if you encounter any problem. :)
```shell
git clone https://github.com/hpcaitech/ColossalAI.git
@@ -107,13 +150,13 @@ train_dataloader = get_dataloader(dataset=dataset,
)
# build your
# build your optimizer
optimizer = ...
# build your loss function
criterion = ...
# build your lr_scheduler
# initialize colossalai
engine, train_dataloader, _, _ = colossalai.initialize(
model=model,
optimizer=optimizer,
@@ -157,21 +200,6 @@ class MLP_2D(nn.Module):
```
## Features
Colossal-AI provides a collection of parallel training components for you. We aim to support you to write your
distributed deep learning models just like how you write your single-GPU model. We provide friendly tools to kickstart
distributed training in a few lines.
- Data Parallelism
- Pipeline Parallelism
- 1D, 2D, 2.5D, 3D and sequence parallelism
- Friendly trainer and engine
- Extensible for new parallelism
- Mixed Precision Training
- Zero Redundancy Optimizer (ZeRO)
Please visit our [documentation and tutorials](https://www.colossalai.org/) for more details.
## Cite Us