github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-01-25 23:04:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
4b9bba81160469ab70c106a7f5eaee8f69edf069
ColossalAI/colossalai/zero
History
ver217 8106d7b8c7 [ddp] refactor ColoDDP and ZeroDDP (#1146) 
* ColoDDP supports overwriting default process group

* rename ColoDDPV2 to ZeroDDP

* add docstr for ZeroDDP

* polish docstr
2022-06-21 16:35:23 +08:00
..
init_ctx
[hotfix] fix zero init ctx numel (#1128)
2022-06-16 17:17:27 +08:00
shard_utils
[gemini] add GeminiMemoryManger (#832)
2022-04-24 13:08:48 +08:00
sharded_model
[hotfix] prevent nested ZeRO (#1140)
2022-06-21 11:33:53 +08:00
sharded_optim
[hotfix] prevent nested ZeRO (#1140)
2022-06-21 11:33:53 +08:00
sharded_param
[gemini] add GeminiMemoryManger (#832)
2022-04-24 13:08:48 +08:00
utils
[zero] avoid zero hook spam by changing log to debug level (#1137)
2022-06-21 10:44:01 +08:00
__init__.py
[zero] add zero optimizer for ColoTensor (#1046)
2022-06-02 12:13:15 +08:00
zero_optimizer.py
[ddp] refactor ColoDDP and ZeroDDP (#1146)
2022-06-21 16:35:23 +08:00