[moe] merge moe into main (#4978)

Browse Source

* update moe module
* support openmoe

...

This commit is contained in:

Xuanlei Zhao

2023-11-02 10:21:24 +08:00

committed by GitHub

parent 8993c8a817

commit dc003c304c

67 changed files with 7618 additions and 1657 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

0

colossalai/tensor/moe_tensor/__init__.py Normal file