github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-26 04:03:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

[upgrade]Upgrade vit (#6308)

Browse Source 
* fix

* fix

* fix rotate embedding test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
flybird11111
2025-05-21 16:14:20 +08:00
committed by GitHub
parent d0e13b85fd
commit 04516bb756
4 changed files with 6 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
colossalai/shardformer/policies/vit.py
View File

@@ -93,10 +93,6 @@ class ViTPolicy(Policy):
"use_zbv": use_zbv,
},
),
SubModuleReplacementDescription(
suffix="attention.attention.dropout",
target_module=col_nn.DropoutForParallelInput,
),
SubModuleReplacementDescription(
suffix="attention.output.dense",
target_module=col_nn.Linear1D_Row,