github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-29 05:26:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,767 Commits 59 Branches 51 Tags
cloud/coati
Commit Graph

3 Commits

Author SHA1 Message Date
FoolPlayer
dd2bf02679 [shardformer] support SAM (#4231) 
* 1.support sam 2.add fused qkv for nn.Linear

* update utils support set element in list

* overtwrite SamVisionAttention foward to use DropoutForParallelInput

* remove unused code
 2023-08-15 23:25:14 +08:00
Frank Lee
ae035d305d [shardformer] added embedding gradient check (#4124) 2023-07-04 16:05:01 +08:00
Frank Lee
f22ddacef0 [shardformer] refactored the shardformer layer structure (#4053) 2023-07-04 16:05:01 +08:00