github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-07-05 11:37:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

[hotfix]fixed bugs of assigning grad states to non leaf nodes (#711)

Browse Source 
* fixed bugs of assigning grad states to non leaf nodes

* use detach()
This commit is contained in:
LuGY 2022-04-11 14:04:58 +08:00 committed by GitHub
parent eda30a058e
commit 140263a394
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
colossalai/utils/activation_checkpoint.py
View File

@ -12,7 +12,7 @@ def copy_to_device(obj, device):
if torch.is_tensor(obj):
# Notice:
# When in no_grad context, requires_gard is False after movement
ret = obj.to(device)
ret = obj.to(device).detach()
ret.requires_grad = obj.requires_grad
return ret
elif isinstance(obj, list):