github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-11 05:49:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

[test] add no master test for low level zero plugin (#4934)

Browse Source
This commit is contained in:
Zhongkai Zhao
2023-10-18 11:41:23 +08:00
committed by GitHub
parent 1f5d2e8062
commit c7aa319ba0
2 changed files with 9 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
colossalai/nn/optimizer/cpu_adam.py
View File

@@ -9,7 +9,8 @@ from .nvme_optimizer import NVMeOptimizer
class CPUAdam(NVMeOptimizer):
"""Implements Adam algorithm.
"""
Implements Adam algorithm.
Supports parameters updating on both GPU and CPU, depending on the device of parameters.
But the parameters and gradients should on the same device: