github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-22 18:09:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

[fx] support module with bias addition (#1780)

Browse Source 
* [autoparallel] refactor tracer to fix bias addition issue

* [fx] support module with bias addition

* create bias_addition_module

* refactor file structure

* polish code

* fix unit test
This commit is contained in:
YuliangLiu0306
2022-11-01 22:53:51 +08:00
committed by GitHub
parent f3f19a5c47
commit e859380bf7
41 changed files with 624 additions and 259 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
colossalai/fx/tracer/meta_patch/patched_function/embedding.py
View File

@@ -1,5 +1,6 @@
import torch
from ..registry import meta_patched_function
from ...registry import meta_patched_function
@meta_patched_function.register(torch.nn.functional.embedding)
@@ -10,4 +11,4 @@ def torch_nn_functional_embedding(input,
norm_type=2.0,
scale_grad_by_freq=False,
sparse=False):
return torch.empty(*input.shape, weight.shape[-1], device="meta")
return torch.empty(*input.shape, weight.shape[-1], device="meta")