[shardformer] init shardformer code structure (#3731)

Browse Source

* init shardformer code structure

* add implement of sharder (inject and replace)

* add implement of replace layer to colossal layer

* separate different layer policy, add some notion

* implement 1d and 2d slicer, can tell col or row

* fix bug when slicing and inject model

* fix some bug; add inference test example

...

This commit is contained in:

FoolPlayer

2023-05-22 15:02:17 +08:00

committed by FrankLeeeee

parent a98e16ed07

commit 6a69b44dfc

16 changed files with 1033 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

0

colossalai/shardformer/__init__.py Normal file