github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-04 18:40:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

try import deepspeed when using zero (#130)

Browse Source
This commit is contained in:
ver217
2022-01-07 17:24:57 +08:00
committed by GitHub
parent b7975d2bcd
commit 9ef05ed1fc
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
colossalai/zero/__init__.py
View File

@@ -29,6 +29,7 @@ def convert_to_zero(model: nn.Module,
:return: (model, optimizer)
:rtype: Tuple
"""
import deepspeed
assert level == 2 or level == 3, 'Only ZERO Optimizer Level 2 and 3 are provided'
model = NaiveAMPModel(model, output_to_fp32=False)