ver217
|
9492a561c3
|
[tensor] ColoTensor supports ZeRo (#1015)
* impl chunk manager
* impl param op hook
* add reduce_chunk
* add zero hook v2
* add zero dp
* fix TensorInfo
* impl load balancing when using zero without chunk
* fix zero hook
* polish chunk
* fix bugs
* ddp ok
* zero ok
* polish code
* fix bugs about load balancing
* polish code
* polish code
* add ene-to-end test
* polish code
* polish code
* polish code
* fix typo
* add test_chunk
* fix bugs
* fix bugs
* polish code
|
2022-05-31 12:00:12 +08:00 |
|
Ziyue Jiang
|
7c530b9de2
|
[Tensor] add Parameter inheritance for ColoParameter (#1041)
* add Parameter inheritance for ColoParameter
* remove tricks
* remove tricks
* polish
* polish
|
2022-05-30 17:23:44 +08:00 |
|
Ziyue Jiang
|
6c5996a56e
|
[Tensor] add module check and bert test (#1031)
* add Embedding
* Add bert test
* polish
* add check module test
* polish
* polish
* polish
* polish
|
2022-05-26 18:15:42 +08:00 |
|
ver217
|
a3b66f6def
|
[tensor] refactor parallel action (#1007)
* refactor parallel action
* polish unit tests
|
2022-05-20 20:19:58 +08:00 |
|
ver217
|
ad536e308e
|
[tensor] refactor colo-tensor (#992)
* refactor colo-tensor and update linear op
* polish code
* polish code
* update ops and unit tests
* update unit tests
* polish code
* rename dist_spec module
* polish code
* polish code
* remove unneeded import
* fix pipelinable
|
2022-05-19 12:44:59 +08:00 |
|
Jiarui Fang
|
ab95ec9aea
|
[Tensor] init ColoParameter (#914)
|
2022-05-06 12:57:14 +08:00 |
|