github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-05-01 21:26:42 +00:00
Code Issues Packages Projects Releases Wiki Activity
5ce6c9d86f
ColossalAI/docs/source/en/features/cluster_utils.md
Hongxin Liu 5ce6c9d86f
[doc] add tutorial for cluster utils (#3763) 
* [doc] add en cluster utils doc

* [doc] add zh cluster utils doc

* [doc] add cluster utils doc in sidebar
2023-05-19 12:12:20 +08:00

1.0 KiB
Raw Blame History

Cluster Utilities

Author: Hongxin Liu

Prerequisite:

Introduction

We provide a utility class colossalai.cluster.DistCoordinator to coordinate distributed training. It's useful to get various information about the cluster, such as the number of nodes, the number of processes per node, etc.

API Reference

{{ autodoc:colossalai.cluster.DistCoordinator }}

{{ autodoc:colossalai.cluster.DistCoordinator.is_master }}

{{ autodoc:colossalai.cluster.DistCoordinator.is_node_master }}

{{ autodoc:colossalai.cluster.DistCoordinator.is_last_process }}

{{ autodoc:colossalai.cluster.DistCoordinator.print_on_master }}

{{ autodoc:colossalai.cluster.DistCoordinator.print_on_node_master }}

{{ autodoc:colossalai.cluster.DistCoordinator.priority_execution }}

{{ autodoc:colossalai.cluster.DistCoordinator.destroy }}

{{ autodoc:colossalai.cluster.DistCoordinator.block_all }}

{{ autodoc:colossalai.cluster.DistCoordinator.on_master_only }}