github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-08-06 02:24:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

[NFC] polish <colossalai/context/process_group_initializer/initializer_data.py> code stype (#626)

Browse Source 
Co-authored-by: 姜卓 <jiangzhuo@jiangzhuodeMacBook-Pro.local>
This commit is contained in:
Jiang Zhuo 2022-04-01 17:55:06 +08:00 committed by binmakeswell
parent 701bad439b
commit 0a96338b13
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
colossalai/context/process_group_initializer/initializer_data.py
View File

@ -20,6 +20,7 @@ class Initializer_Data(ProcessGroupInitializer):
pipeline_parallel_size (int): Size of pipeline parallel. pipeline_parallel_size (int): Size of pipeline parallel.
tensor_parallel_size (int): Size of tensor parallel. tensor_parallel_size (int): Size of tensor parallel.
""" """
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs) super().__init__(*args, **kwargs)
self.num_data_parallel_group = self.world_size // self.data_parallel_size self.num_data_parallel_group = self.world_size // self.data_parallel_size