Genghan Zhang d655eea515 [autoparallel] mix gather (#1977) ...

* Add mix-gather

* Add comments

* Add comments

* Polish comments

* Change the global rank assumption

* Add tests

* Add two-step tests

* Fix 10 and 01

* Skip test becasue the number of GPUs

2022-11-23 21:49:17 +08:00

..

common_utils

[colotensor] add Tensor.view op and its unit test (#1343)

2022-07-21 10:53:15 +08:00

core

[colotensor] add Tensor.view op and its unit test (#1343)

2022-07-21 10:53:15 +08:00

model

[utils] remove lazy_memory_allocate from ColoInitContext (#1844)

2022-11-09 11:50:33 +08:00

test_colo_checkpoint_tools.py

[colotensor] use cpu memory to store state_dict (#1367)

2022-07-26 14:13:38 +08:00

test_comm_spec_apply.py

[autoparallel] shard param and buffer as expected (#1753)

2022-10-21 15:45:13 +08:00

test_context.py

[ColoTensor] reconfig ColoInitContext, decouple default_pg and default_dist_spec. (#1953)

2022-11-15 16:24:16 +08:00

test_mix_gather.py

[autoparallel] mix gather (#1977)

2022-11-23 21:49:17 +08:00

test_parameter.py

[refactor] refactor ColoTensor's unit tests (#1340)

2022-07-19 15:46:24 +08:00

test_shape_consistency_apply.py

[autoparallel] shard param and buffer as expected (#1753)

2022-10-21 15:45:13 +08:00

test_shape_consistency.py

[autoparallel] update CommSpec (#1667)

2022-09-29 11:20:59 +08:00

test_sharded_linear.py

[ColoTensor] ColoInitContext initialize parameters in shard mode. (#1937)

2022-11-14 16:05:09 +08:00

test_sharding_spec.py

[autoparallel] handled illegal sharding strategy (#1728)

2022-10-19 12:53:06 +08:00

test_tp_with_zero.py

[Gemini] add GeminiAdamOptimizer (#1960)

2022-11-16 14:44:28 +08:00