[shardformer] support pipeline base vit model (#4284)

* Feature/vit support (#4182)

* [shardformer] added tests

* [shardformer] vit test finish and support

* fix attention dropout

* support base vit pipeline

* support vit downstream model

* fix vit shard test

* modify hidden states return type

---------

Co-authored-by: Kun Lin <81014421+klhhhhh@users.noreply.github.com>
This commit is contained in:
FoolPlayer
2023-07-25 15:02:29 +08:00
committed by Hongxin Liu
parent 083d7da33d
commit b3f5d7a3ba
7 changed files with 728 additions and 104 deletions

View File

@@ -5,3 +5,4 @@ from .gpt import *
from .llama import *
from .opt import *
from .t5 import *
from .vit import *