Frank Lee 40d376c566 [setup] support pre-build and jit-build of cuda kernels (#2374) ...

* [setup] support pre-build and jit-build of cuda kernels

* polish code

* polish code

* polish code

* polish code

* polish code

* polish code

2023-01-06 20:50:26 +08:00

..

colossalai_layer

added skip_bias_add for non-tp linear

2022-11-09 15:41:08 +08:00

moe

[setup] support pre-build and jit-build of cuda kernels (#2374)

2023-01-06 20:50:26 +08:00

parallel_1d

[tensorparallel] fixed tp layers (#1938)

2022-11-14 17:34:03 +08:00

parallel_2d

[utils] refactor parallel layers checkpoint and bcast model on loading checkpoint (#1548)

2022-09-06 20:18:35 +08:00

parallel_2p5d

[utils] refactor parallel layers checkpoint and bcast model on loading checkpoint (#1548)

2022-09-06 20:18:35 +08:00

parallel_3d

improved allgather & reducescatter for 3d

2023-01-03 17:46:08 +08:00

parallel_sequence

[NFC] polish colossalai/nn/layer/parallel_sequence/layers.py code style (#1280)

2022-07-13 12:08:21 +08:00

utils

…

vanilla

added skip_bias_add for non-tp linear

2022-11-09 15:41:08 +08:00

wrapper

[NFC] polish colossalai/nn/layer/wrapper/pipeline_wrapper.py code style (#1303)

2022-07-13 19:01:07 +08:00

__init__.py

…

base_layer.py

[utils] refactor parallel layers checkpoint and bcast model on loading checkpoint (#1548)

2022-09-06 20:18:35 +08:00