[doc] Add user document for Shardformer (#4702)

* create shardformer doc files

* add docstring for seq-parallel

* update ShardConfig docstring

* add links to llama example

* add outdated massage

* finish introduction & supporting information

* finish 'how shardformer works'

* finish shardformer.md English doc

* fix doctest fail

* add Chinese document
This commit is contained in:
Baizhou Zhang
2023-09-15 10:56:39 +08:00
committed by GitHub
parent ce97790ed7
commit f911d5b09d
11 changed files with 315 additions and 33 deletions

View File

@@ -2,6 +2,8 @@
作者: Zhengda Bian, Yongbin Li
> ⚠️ 此页面上的信息已经过时并将被废弃。请在[Shardformer](./shardformer.md)页面查阅更新。
**前置教程**
- [定义配置文件](../basics/define_your_config.md)
- [并行配置](../basics/configure_parallelization.md)
@@ -118,3 +120,5 @@ Output of the first linear layer: torch.Size([16, 512])
Output of the second linear layer: torch.Size([16, 256])
```
第一个线性层的输出被划分成2块 (每个形状为 `[16, 512]`), 而第二层在整个 GPU 上的输出是相同的。
<!-- doc-test-command: echo -->