ver217 a241f61b34 [zero] Update initialize for ZeRO (#458) ...

* polish code

* shard strategy receive pg in shard() / gather()

* update zero engine

* polish code

2022-03-18 16:18:31 +08:00

..

components_to_test

[zero] new interface for ShardedOptimv2 (#406)

2022-03-14 20:48:41 +08:00

test_amp

optimized context test time consumption (#446)

2022-03-17 14:40:52 +08:00

test_comm

Hotfix/Colossalai layers (#92)

2021-12-29 23:32:10 +08:00

test_config

[profiler] primary memory tracer

2022-03-11 15:50:28 +08:00

test_context

optimized context test time consumption (#446)

2022-03-17 14:40:52 +08:00

test_data

added CI for unit testing (#69)

2021-12-16 10:32:08 +08:00

test_data_pipeline_tensor_parallel

optimize engine and trainer test (#448)

2022-03-17 15:44:17 +08:00

test_engine

optimize engine and trainer test (#448)

2022-03-17 15:44:17 +08:00

test_layers

fixed padding index issue for vocab parallel embedding layers; updated 3D linear to be compatible with examples in the tutorial

2022-03-11 15:50:28 +08:00

test_moe

Added TPExpert for special situation

2022-03-11 15:50:28 +08:00

test_optimizer

[hotfix] run cpu adam unittest in pytest (#424)

2022-03-16 10:39:55 +08:00

test_trainer

optimize engine and trainer test (#448)

2022-03-17 15:44:17 +08:00

test_utils

[test] optimized zero data parallel test (#452)

2022-03-18 11:35:54 +08:00

test_zero_data_parallel

[zero] Update initialize for ZeRO (#458)

2022-03-18 16:18:31 +08:00

test_zero_tensor_parallel

Feature/zero (#279)

2022-03-11 15:50:28 +08:00

__init__.py

[zero] Update sharded model v2 using sharded param v2 (#323)

2022-03-11 15:50:28 +08:00