github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-06-01 20:05:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
684 Commits 51 Branches 50 Tags 235 MiB
1f894e033f
Commit Graph

3 Commits

Author SHA1 Message Date
ver217
1f894e033f
[gemini] zero supports gemini (#1093) 
* add placement policy

* add gemini mgr

* update mem stats collector

* update zero

* update zero optim

* fix bugs

* zero optim monitor os

* polish unit test

* polish unit test

* add assert
 2022-06-10 14:48:28 +08:00
Jiarui Fang
49832b2344
[refactory] add nn.parallel module (#1068) 2022-06-06 15:34:41 +08:00
ver217
9492a561c3
[tensor] ColoTensor supports ZeRo (#1015) 
* impl chunk manager

* impl param op hook

* add reduce_chunk

* add zero hook v2

* add zero dp

* fix TensorInfo

* impl load balancing when using zero without chunk

* fix zero hook

* polish chunk

* fix bugs

* ddp ok

* zero ok

* polish code

* fix bugs about load balancing

* polish code

* polish code

* add ene-to-end test

* polish code

* polish code

* polish code

* fix typo

* add test_chunk

* fix bugs

* fix bugs

* polish code
 2022-05-31 12:00:12 +08:00