github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-01-15 14:47:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,559 Commits 60 Branches 51 Tags
e94c79f15b2ee1428ccda2cdb2d00007697c578c
Commit Graph

41 Commits

Author SHA1 Message Date
HELSON
2458659919 [zero] fix error for BEiT models (#2169) 
* [zero] fix error for BEiT models

* [ColoParameter] add unpack operation for tuple arguments

* fix bugs

* fix chunkv2 unit testing

* add assertion for gradient state
 2022-12-26 15:03:54 +08:00
Tongping Liu
ab54fed292 [hotfix] add kwargs for colo_addmm (#2171) 2022-12-22 13:25:30 +08:00
Jiarui Fang
96134e7be3 [hotfix] add bert test for gemini fwd bwd (#2035) 2022-11-29 11:19:52 +08:00
Jiarui Fang
8daf1b4db1 [Gemini] patch for supporting orch.add_ function for ColoTensor (#2003) 2022-11-25 20:06:35 +08:00
Jiarui Fang
a2d3266648 [hotfix] make Gemini work for conv DNN (#1998) 2022-11-22 14:52:36 +08:00
jim
e5ab6be72e [hotfix[ fix colotensor.type() raise NotImplementedError (#1682) 2022-10-10 10:13:31 +08:00
BigOneLiXiaoMing
0c4c9aa6e0 [NFC] polish colossalai/nn/_ops/embedding.py code style (#1561) 2022-09-08 22:11:04 +08:00
yuxuan-lou
413f9c19f4 [NFC] polish colossalai/nn/_ops/layernorm.py code style (#1555) 2022-09-08 22:11:04 +08:00
Maruyama_Aya
bd2d789832 [NFC] polish colossalai/nn/_ops/embedding_bag.py code style (#1552) 2022-09-08 22:11:04 +08:00
Jiarui Fang
87134524fd [embedding] tablewise sharding polish (#1535) 2022-09-02 11:09:37 +08:00
Frank Lee
ae1b58cd16 [tensor] added linear implementation for the new sharding spec (#1416) 
* [tensor] added linear implementation for the new sharding spec

* polish code
 2022-08-12 11:33:09 +08:00
Jiarui Fang
30b4dd17c0 [FAW] export FAW in _ops (#1438) 2022-08-11 13:43:24 +08:00
Jiarui Fang
c9427a323f hotfix #1434 (#1437) 2022-08-11 13:14:25 +08:00
Jiarui Fang
10b3df65c8 [FAW] move coloparam setting in test code. (#1429) 2022-08-10 14:31:53 +08:00
Jiarui Fang
cb98cf5558 [FAW] parallel FreqAwareEmbedding (#1424) 2022-08-10 13:44:30 +08:00
Jiarui Fang
d209aff684 Add FreqAwareEmbeddingBag (#1421) 2022-08-09 16:26:12 +08:00
Jiarui Fang
504419d261 [FAW] add cache manager for the cached embedding (#1419) 2022-08-09 15:17:17 +08:00
HELSON
7a8702c06d [colotensor] add Tensor.view op and its unit test (#1343) 
[colotensor] add megatron initialization for gpt2
 2022-07-21 10:53:15 +08:00
HELSON
260a55804a [hotfix] fix shape error in backward when using ColoTensor (#1298) 2022-07-13 23:06:12 +08:00
HELSON
abba4d84e1 [hotfix] fix bert model test in unitests (#1272) 2022-07-12 23:26:45 +08:00
Jiarui Fang
1aad903c15 [tensor] redistribute among different process groups (#1247) 
* make it faster

* [tensor] rename convert_to_dist -> redistribute

* [tensor] ShardSpec and ReplicaSpec

* [tensor] redistribute among diff pgs

* polish code
 2022-07-12 10:24:05 +08:00
Jiarui Fang
9bcd2fd4af [tensor] a shorter shard and replicate spec (#1245) 2022-07-11 15:51:48 +08:00
Jiarui Fang
2699dfbbfd [rename] convert_to_dist -> redistribute (#1243) 2022-07-11 13:05:44 +08:00
Jiarui Fang
4a76084dc9 [tensor] add zero_like colo op, important for Optimizer (#1236) 2022-07-08 14:55:27 +08:00
Jiarui Fang
3b500984b1 [tensor] fix some unittests (#1234) 2022-07-08 14:18:30 +08:00
HELSON
0453776def [tensor] fix a assertion in colo_tensor cross_entropy (#1232) 2022-07-08 11:18:00 +08:00
HELSON
42ab36b762 [tensor] add unitest for colo_tensor 1DTP cross_entropy (#1230) 2022-07-07 19:17:23 +08:00
Jiarui Fang
a98319f023 [tensor] torch function return colotensor (#1229) 2022-07-07 18:09:18 +08:00
Jiarui Fang
ae7d3f4927 [refactor] move process group from _DistSpec to ColoTensor. (#1203) 2022-07-06 16:15:16 +08:00
Jiarui Fang
060b917daf [refactor] remove gpc dependency in colotensor's _ops (#1189) 2022-07-04 18:54:37 +08:00
Jiarui Fang
1b657f9ce1 [tensor] revert local view back (#1178) 2022-06-27 18:38:34 +08:00
Jiarui Fang
0dd4e2bbfb [Tensor] rename some APIs in TensorSpec and Polish view unittest (#1176) 2022-06-27 15:56:11 +08:00
Jiarui Fang
aa7bef73d4 [Tensor] distributed view supports inter-process hybrid parallel (#1169) 2022-06-27 09:45:26 +08:00
Jiarui Fang
4b9bba8116 [ColoTensor] rename APIs and add output_replicate to ComputeSpec (#1168) 2022-06-24 13:08:54 +08:00
Jiarui Fang
f4ef224358 [Tensor] remove ParallelAction, use ComputeSpec instread (#1166) 2022-06-23 17:34:59 +08:00
Jiarui Fang
177c374401 remove gather out in parallel action (#1163) 2022-06-23 16:35:05 +08:00
Jiarui Fang
07f9c781f9 [graph] improve the graph building. (#1157) 2022-06-22 16:47:20 +08:00
ver217
22717a856f [tensor] add embedding bag op (#1156) 2022-06-22 15:54:03 +08:00
ver217
ae86151968 [tensor] add more element-wise ops (#1155) 
* add more element-wise ops

* update test_op

* polish unit test
 2022-06-22 15:16:47 +08:00
ver217
ccf3c58c89 embedding op use gather_out (#1143) 2022-06-21 13:21:20 +08:00
Jiarui Fang
a00644079e reorgnize colotensor directory (#1062) 
* reorgnize colotensor directory

* polish code
 2022-06-03 18:04:22 +08:00