github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-30 05:55:56 +00:00
Code Issues Packages Projects Releases Wiki Activity

[pipeline/fix-bug] num_microbatches support any integrate | stable chimera | launch tool for rpc pp framework (#1684)

Browse Source 
* [pipeline/tuning] improve dispatch performance both time and space cost

* [pipeline/converge] add interface for testing convergence

* [NFC] polish colossalai/utils/multi_tensor_apply/multi_tensor_apply.py code style

* Update PipelineBase.py

* [pipeline/chimera] reconstruct PipelineBase and Worker to support more feasible custom schedule | finish Chimera

* [pipeline/chimera] test chimera | fix bug of initializing

* [pipeline/pytree] add pytree to process args and kwargs | provide  to process args and kwargs after forward

* [pipeline/fix-bug] num_microbatches support any integrate | stable chimera | launch tool for rpc pp framework
This commit is contained in:
Kirigaya Kazuto
2022-10-10 16:01:02 +08:00
committed by GitHub
parent e5ab6be72e
commit 0df5034a36
4 changed files with 98 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
colossalai/pipeline/pipeline_process_group.py
View File

@@ -50,6 +50,7 @@ class PipelineProcessGroup:
self.is_initialize = True
# lock
self.initialise_lock = threading.Lock()
self.chimera_lock = threading.Lock()
def _initialize_process_group(self):