github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-01 17:17:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

[doc] update hybrid parallelism doc (#3770)

Browse Source
This commit is contained in:
jiangmingyan
2023-05-18 14:16:13 +08:00
committed by GitHub
parent 15024e40d9
commit 48bd056761
8 changed files with 41 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/en/features/3D_tensor_parallel.md
View File

@@ -9,7 +9,7 @@ Author: Zhengda Bian, Yongbin Li
- [2D Tensor Parallelism](./2D_tensor_parallel.md)
**Example Code**
- [ColossalAI-Examples - 3D Tensor Parallelism](https://github.com/hpcaitech/ColossalAI-Examples/tree/main/features/tensor_parallel/tensor_parallel_3d.py)
- [ColossalAI-Examples - 3D Tensor Parallelism](https://github.com/hpcaitech/ColossalAI-Examples/blob/main/features/tensor_parallel/README.md)
**Related Paper**
- [Maximizing Parallelism in Distributed Training for Huge Neural Networks](https://arxiv.org/pdf/2105.14450.pdf)