github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-11-01 06:19:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,777 Commits 59 Branches 51 Tags
8b0ed614909154388bdcb3884324a159c162f524
Commit Graph

6 Commits

Author SHA1 Message Date
Hongxin Liu
7f8b16635b [misc] refactor launch API and tensor constructor (#5666) 
* [misc] remove config arg from initialize

* [misc] remove old tensor contrusctor

* [plugin] add npu support for ddp

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [devops] fix doc test ci

* [test] fix test launch

* [doc] update launch doc

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-04-29 10:40:11 +08:00
Baizhou Zhang
21ba89cab6 [gemini] support gradient accumulation (#4869) 
* add test

* fix no_sync bug in low level zero plugin

* fix test

* add argument for grad accum

* add grad accum in backward hook for gemini

* finish implementation, rewrite tests

* fix test

* skip stuck model in low level zero test

* update doc

* optimize communication & fix gradient checkpoint

* modify doc

* cleaning codes

* update cpu adam fp16 case
 2023-10-17 14:07:21 +08:00
Hongxin Liu
66f3926019 [doc] clean up outdated docs (#4765) 
* [doc] clean up outdated docs

* [doc] fix linking

* [doc] fix linking
 2023-09-21 11:36:20 +08:00
flybird1111
f40b718959 [doc] Fix gradient accumulation doc. (#4349) 
* [doc] fix gradient accumulation doc

* [doc] fix gradient accumulation doc
 2023-08-04 17:24:35 +08:00
jiangmingyan
278fcbc444 [doc]fix 2023-05-23 17:53:11 +08:00
jiangmingyan
ef02d7ef6d [doc] update gradient accumulation (#3771) 
* [doc]update gradient accumulation

* [doc]update gradient accumulation

* [doc]update gradient accumulation

* [doc]update gradient accumulation

* [doc]update gradient accumulation, fix

* [doc]update gradient accumulation, fix

* [doc]update gradient accumulation, fix

* [doc]update gradient accumulation, add sidebars

* [doc]update gradient accumulation, fix

* [doc]update gradient accumulation, fix

* [doc]update gradient accumulation, fix

* [doc]update gradient accumulation, resolve comments

* [doc]update gradient accumulation, resolve comments

* fix
 2023-05-23 10:52:30 +08:00