github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-07-23 03:33:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
tongyu 2025-04-27 04:24:35 +08:00 committed by GitHub
parent 46ed5d856b
commit e44e65d61c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 6 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
colossalai/shardformer/README.md
View File

@ -83,6 +83,12 @@ Following are the description `ShardConfig`'s arguments:
- `extra_kwargs`: A dict to store extra kwargs for ShardFormer.
- - `tensor_parallel_mode`: A choice of parallel modes for tensor operations. Supported modes are:
- `'1d'`: 1D tensor parallelism
- `'2d'`: 2D tensor parallelism
- `'2.5d'`: 2.5D tensor parallelism
- `'3d'`: 3D tensor parallelism
### Write your own policy
If you have a custom model, you can also use Shardformer to parallelize it by writing your own sharding policy. More information about the sharding policy can be found in [API Design](#-api-design).