diff --git a/README-zh-Hans.md b/README-zh-Hans.md
index c68e48513..842f0c378 100644
--- a/README-zh-Hans.md
+++ b/README-zh-Hans.md
@@ -1,24 +1,57 @@
# Colossal-AI
+
-[](https://www.colossalai.org/)
+ [](https://www.colossalai.org/)
+
+ 一个整合高效并行技术的AI大模型训练系统。
-
-
+
博客
[](https://github.com/hpcaitech/ColossalAI/actions/workflows/PR_CI.yml)
[](https://colossalai.readthedocs.io/en/latest/?badge=latest)
[](https://codebeat.co/projects/github-com-hpcaitech-colossalai-main)
[](https://join.slack.com/t/colossalaiworkspace/shared_invite/zt-z7b26eeb-CBp7jouvu~r0~lcFzX832w)
- [](./docs/images/WeChat.png)
+ [](https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/WeChat.png)
| [English](README.md) | [中文](README-zh-Hans.md) |
+
-一个整合高效并行技术的AI大模型训练系统。
+
+
+## 目录
+
## 特点
@@ -33,30 +66,33 @@ Colossal-AI为您提供了一系列并行训练组件。我们的目标是让您
- 混合精度
- 零冗余优化器 (ZeRO)
-## 样例
+
(返回顶端)
+
+## 展示样例
### ViT
-

+

- 14倍批大小和5倍训练速度(张量并行=64)
### GPT-3
-

+

- 释放 50% GPU 资源占用, 或 10.7% 加速
### GPT-2
-

+

- 降低11倍GPU显存占用,或超线性扩展
### BERT
-

+

- 2倍训练速度,或1.5倍序列长度
请访问我们的[文档和教程](https://www.colossalai.org/)以了解详情。
+
(返回顶端)
## 安装
@@ -97,6 +133,8 @@ pip install .
pip install --global-option="--no_cuda_ext" .
```
+
(返回顶端)
+
## 使用 Docker
运行以下命令从我们提供的docker文件中建立docker镜像。
@@ -112,11 +150,12 @@ docker build -t colossalai ./docker
docker run -ti --gpus all --rm --ipc=host colossalai bash
```
+
(返回顶端)
## 社区
欢迎通过[论坛](https://github.com/hpcaitech/ColossalAI/discussions),
[Slack](https://join.slack.com/t/colossalaiworkspace/shared_invite/zt-z7b26eeb-CBp7jouvu~r0~lcFzX832w),
-或[微信](https://github.com/hpcaitech/public_assets/blob/main/colossalai/img/WeChat.png "qrcode")加入Colossal-AI社区,与我们分享你的建议和问题。
+或[微信](https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/WeChat.png "qrcode")加入Colossal-AI社区,与我们分享你的建议和问题。
## 做出贡献
@@ -129,10 +168,11 @@ docker run -ti --gpus all --rm --ipc=host colossalai bash
*贡献者头像的展示顺序是随机的。*
+
(返回顶端)
## 快速预览
-### Start Distributed Training in Lines
+### 几行代码开启分布式训练
```python
import colossalai
@@ -209,10 +249,9 @@ class MLP_2D(nn.Module):
```
+
(返回顶端)
-
-
-## 引用
+## 引用我们
```
@article{bian2021colossal,
@@ -222,3 +261,5 @@ class MLP_2D(nn.Module):
year={2021}
}
```
+
+
(返回顶端)
diff --git a/README.md b/README.md
index dd181341e..840c6db50 100644
--- a/README.md
+++ b/README.md
@@ -1,25 +1,56 @@
# Colossal-AI
+
-[](https://www.colossalai.org/)
+ [](https://www.colossalai.org/)
+
+ An integrated large-scale model training system with efficient parallelization techniques.
-
-
+
Blog
[](https://github.com/hpcaitech/ColossalAI/actions/workflows/PR_CI.yml)
[](https://colossalai.readthedocs.io/en/latest/?badge=latest)
[](https://codebeat.co/projects/github-com-hpcaitech-colossalai-main)
[](https://join.slack.com/t/colossalaiworkspace/shared_invite/zt-z7b26eeb-CBp7jouvu~r0~lcFzX832w)
- [](./docs/images/WeChat.png)
+ [](https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/WeChat.png)
| [English](README.md) | [中文](README-zh-Hans.md) |
-
-An integrated large-scale model training system with efficient parallelization techniques.
+
+
+## Table of Contents
+
## Features
@@ -36,7 +67,9 @@ distributed training in a few lines.
- Mixed Precision Training
- Zero Redundancy Optimizer (ZeRO)
-## Examples
+
(back to top)
+
+## Demo
### ViT

@@ -59,6 +92,7 @@ distributed training in a few lines.
Please visit our [documentation and tutorials](https://www.colossalai.org/) for more details.
+
(back to top)
## Installation
@@ -99,6 +133,7 @@ If you don't want to install and enable CUDA kernel fusion (compulsory installat
pip install --global-option="--no_cuda_ext" .
```
+
(back to top)
## Use Docker
@@ -115,14 +150,13 @@ Run the following command to start the docker container in interactive mode.
docker run -ti --gpus all --rm --ipc=host colossalai bash
```
+
(back to top)
## Community
Join the Colossal-AI community on [Forum](https://github.com/hpcaitech/ColossalAI/discussions),
[Slack](https://join.slack.com/t/colossalaiworkspace/shared_invite/zt-z7b26eeb-CBp7jouvu~r0~lcFzX832w),
-and [WeChat](./docs/images/WeChat.png "qrcode") to share your suggestions, advice, and questions with our engineering team.
-
-
+and [WeChat](https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/WeChat.png "qrcode") to share your suggestions, advice, and questions with our engineering team.
## Contributing
@@ -134,6 +168,8 @@ Thanks so much to all of our amazing contributors!
*The order of contributor avatars is randomly shuffled.*
+
(back to top)
+
## Quick View
### Start Distributed Training in Lines
@@ -216,7 +252,7 @@ class MLP_2D(nn.Module):
```
-
+
(back to top)
## Cite Us
@@ -228,3 +264,5 @@ class MLP_2D(nn.Module):
year={2021}
}
```
+
+
(back to top)
diff --git a/docs/images/BERT.png b/docs/images/BERT.png
deleted file mode 100644
index c1e283ade..000000000
Binary files a/docs/images/BERT.png and /dev/null differ
diff --git a/docs/images/Colossal-AI_logo.png b/docs/images/Colossal-AI_logo.png
deleted file mode 100644
index 886f35beb..000000000
Binary files a/docs/images/Colossal-AI_logo.png and /dev/null differ
diff --git a/docs/images/GPT2.png b/docs/images/GPT2.png
deleted file mode 100644
index e05e444b3..000000000
Binary files a/docs/images/GPT2.png and /dev/null differ
diff --git a/docs/images/GPT3.png b/docs/images/GPT3.png
deleted file mode 100644
index 2a96dd8c7..000000000
Binary files a/docs/images/GPT3.png and /dev/null differ
diff --git a/docs/images/ViT.png b/docs/images/ViT.png
deleted file mode 100644
index 37df168a4..000000000
Binary files a/docs/images/ViT.png and /dev/null differ
diff --git a/docs/images/WeChat.png b/docs/images/WeChat.png
deleted file mode 100644
index 2a6af7faa..000000000
Binary files a/docs/images/WeChat.png and /dev/null differ