Jiarui Fang
|
ae7d3f4927
|
[refactor] move process group from _DistSpec to ColoTensor. (#1203)
|
2022-07-06 16:15:16 +08:00 |
|
Jiarui Fang
|
060b917daf
|
[refactor] remove gpc dependency in colotensor's _ops (#1189)
|
2022-07-04 18:54:37 +08:00 |
|
Jiarui Fang
|
4b9bba8116
|
[ColoTensor] rename APIs and add output_replicate to ComputeSpec (#1168)
|
2022-06-24 13:08:54 +08:00 |
|
Jiarui Fang
|
f4ef224358
|
[Tensor] remove ParallelAction, use ComputeSpec instread (#1166)
|
2022-06-23 17:34:59 +08:00 |
|
Jiarui Fang
|
177c374401
|
remove gather out in parallel action (#1163)
|
2022-06-23 16:35:05 +08:00 |
|
ver217
|
a3b66f6def
|
[tensor] refactor parallel action (#1007)
* refactor parallel action
* polish unit tests
|
2022-05-20 20:19:58 +08:00 |
|
ver217
|
8e3d0ad8f1
|
[unit test] refactor test tensor (#1005)
* polish test_gpt
* update op unit tests
* update test model
|
2022-05-19 18:57:56 +08:00 |
|
ver217
|
ad536e308e
|
[tensor] refactor colo-tensor (#992)
* refactor colo-tensor and update linear op
* polish code
* polish code
* update ops and unit tests
* update unit tests
* polish code
* rename dist_spec module
* polish code
* polish code
* remove unneeded import
* fix pipelinable
|
2022-05-19 12:44:59 +08:00 |
|
ver217
|
c2fdc6a011
|
[tensor] derive compute pattern from dist spec (#971)
* derive compute pattern from dist spec
* polish code
|
2022-05-16 14:58:08 +08:00 |
|
ver217
|
67c33f57eb
|
[tensor] design DistSpec and DistSpecManager for ColoTensor (#934)
* add dist spec
* update linear op
* polish code
* polish code
* update embedding op
* polish unit tests
* polish unit tests
* polish comments
* polish code
* add test_dist_spec_mgr
* polish code
* refactor folder structure
* polish unit tests
* add get_process_group() for TensorSpec
* polish code
|
2022-05-13 15:13:52 +08:00 |
|
Ziyue Jiang
|
193d629311
|
update pytest.mark.parametrize in tensor tests (#913)
|
2022-05-06 11:16:40 +08:00 |
|
Ziyue Jiang
|
2c0d19d755
|
[Tensor] add ColoTensor TP1Dcol Embedding (#899)
|
2022-04-28 17:45:06 +08:00 |
|
Ziyue Jiang
|
cb182da7c5
|
[tensor] refine linear and add gather for laynorm (#893)
* refine linear and add function to ColoTensor
* add gather for layernorm
* polish
* polish
|
2022-04-28 10:55:40 +08:00 |
|
Ziyue Jiang
|
1d0aba4153
|
[tensor] add ColoTensor 1Dcol (#888)
|
2022-04-27 14:13:55 +08:00 |
|
Ziyue Jiang
|
26d4ab8b03
|
[Tensor] Add function to spec and update linear 1Drow and unit tests (#869)
|
2022-04-26 10:15:26 +08:00 |
|
Ziyue Jiang
|
bcc8655021
|
[Tensor ] Add 1Drow weight reshard by spec (#854)
|
2022-04-24 18:30:20 +08:00 |
|
Jiarui Fang
|
62f059251b
|
[Tensor] init a tp network training unittest (#849)
|
2022-04-24 16:43:44 +08:00 |
|
Ziyue Jiang
|
2a0a427e04
|
[tensor]add assert for colo_tensor 1Drow (#846)
|
2022-04-24 14:12:45 +08:00 |
|
Ziyue Jiang
|
05023ecfee
|
[Tensor] TP Linear 1D row (#843)
|
2022-04-24 13:43:12 +08:00 |
|
Jiarui Fang
|
cb5a4778e1
|
Revert "[WIP] Applying ColoTensor on TP-1D-row Linear. (#831)" (#835)
This reverts commit ac88de6dfc.
|
2022-04-22 14:45:57 +08:00 |
|
Jiarui Fang
|
ac88de6dfc
|
[WIP] Applying ColoTensor on TP-1D-row Linear. (#831)
* revert zero tensors back
* [tensor] init row 1d linear
|
2022-04-22 14:03:26 +08:00 |
|