github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-10-27 11:31:25 +00:00
Code Issues Packages Projects Releases Wiki Activity

[zero] add zero optimizer for ColoTensor (#1046)

Browse Source 
* add zero optimizer

* torch ok

* unit test ok

* polish code

* fix bugs

* polish unit test

* polish zero optim

* polish colo ddp v2

* refactor folder structure

* add comment

* polish unit test

* polish zero optim

* polish unit test
This commit is contained in:
ver217
2022-06-02 12:13:15 +08:00
committed by GitHub
parent e32470b6de
commit 51b9a49655
7 changed files with 245 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
colossalai/zero/utils/zero_hook_v2.py
View File

@@ -1,5 +1,6 @@
import torch
from colossalai.tensor import ParamOpHook, ChunkManager, TensorState
from colossalai.tensor.param_op_hook import ParamOpHook
from colossalai.tensor.chunk import ChunkManager, TensorState
from enum import Enum
from typing import List
from contextlib import contextmanager