github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-05 19:13:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

[shardformer] support Blip2 (#4243)

Browse Source 
* support base blip2

* add support for downstream blip2 model

* update readme

* add forward injection

* skip not compatible models test

* fix test for gemini and low_level_zero_pugin
This commit is contained in:
FoolPlayer
2023-07-25 14:29:10 +08:00
committed by Hongxin Liu
parent 8120eca0c0
commit 879301d0da
8 changed files with 541 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
colossalai/shardformer/modeling/sam.py
View File

@@ -1,6 +1,4 @@
import torch
import torch.distributed as dist
from torch.distributed import ProcessGroup
def forward_fn():