github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-01-19 09:04:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
a9b8300d54f842c4ae840552e99f2ffadffcc172
ColossalAI/tests/test_zero_data_parallel/common.py
HELSON a9b8300d54 [zero] improve adaptability for not-shard parameters (#708) 
* adapt post grad hooks for not-shard parameters
* adapt optimizer for not-shard parameters
* offload gradients for not-replicated parameters
2022-04-11 13:38:51 +08:00

5.7 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink