github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-08 12:30:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

[Tensor] initialize the ColoOptimizer (#898)

Browse Source 
* [Tensor] activation is an attr of ColoTensor

* [Tensor] add optimizer

* only detach parameters in context

* polish code
This commit is contained in:
Jiarui Fang
2022-04-28 15:23:40 +08:00
committed by GitHub
parent 676f191532
commit d16671da75
5 changed files with 130 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
colossalai/tensor/__init__.py
View File

@@ -4,8 +4,9 @@ from .op_wrapper import (
from .colo_tensor import ColoTensor
from .utils import convert_parameter, named_params_with_colotensor
from ._ops import *
from .optim.colo_optimizer import ColoOptimizer
__all__ = [
'ColoTensor', 'convert_parameter', 'colo_op_impl', 'ComputePattern', 'TensorSpec', 'ParallelAction',
'named_params_with_colotensor', 'ShardPattern'
'named_params_with_colotensor', 'ShardPattern', 'ColoOptimizer'
]