Jiarui Fang c9e7d9582d [zero] polish shard strategy (#310) ...

* init shard param from shape tuple

* add more unitest for shard param

* add set_payload method for ShardedParam

* [zero] add shareded tensor class

* polish code

* add shard stratgy

* move shard and gather logic to shard strategy from shard tensor.

* polish code

2022-03-11 15:50:28 +08:00

..

test_comm

Hotfix/Colossalai layers (#92)

2021-12-29 23:32:10 +08:00

test_config

[profiler] primary memory tracer

2022-03-11 15:50:28 +08:00

test_context

Optimize pipeline schedule (#94)

2021-12-30 15:56:46 +08:00

test_data

added CI for unit testing (#69)

2021-12-16 10:32:08 +08:00

test_data_pipeline_tensor_parallel

Optimize pipeline schedule (#94)

2021-12-30 15:56:46 +08:00

test_engine/test_engine

add a common util for hooks registered on parameter. (#292)

2022-03-11 15:50:28 +08:00

test_layers

fixed padding index issue for vocab parallel embedding layers; updated 3D linear to be compatible with examples in the tutorial

2022-03-11 15:50:28 +08:00

test_moe

Added TPExpert for special situation

2022-03-11 15:50:28 +08:00

test_trainer

pipeline last stage supports multi output (#151)

2022-01-17 15:57:47 +08:00

test_utils

Feature/zero (#279)

2022-03-11 15:50:28 +08:00

test_zero_data_parallel

[zero] polish shard strategy (#310)

2022-03-11 15:50:28 +08:00

test_zero_tensor_parallel

Feature/zero (#279)

2022-03-11 15:50:28 +08:00