Commit Graph

5 Commits

Author SHA1 Message Date
ver217
96780e6ee4 Optimize pipeline schedule (#94)
* add pipeline shared module wrapper and update load batch

* added model parallel process group for amp and clip grad (#86)

* added model parallel process group for amp and clip grad

* update amp and clip with model parallel process group

* remove pipeline_prev/next group (#88)

* micro batch offload

* optimize pipeline gpu memory usage

* pipeline can receive tensor shape (#93)

* optimize pipeline gpu memory usage

* fix grad accumulation step counter

* rename classes and functions

Co-authored-by: Frank Lee <somerlee.9@gmail.com>
2021-12-30 15:56:46 +08:00
Frank Lee
9a0466534c update markdown docs (english) (#60) 2021-12-10 14:37:33 +08:00
binmakeswell
05e7069a5b fixed some typos in the documents, added blog link and paper author information in README 2021-11-03 17:18:43 +08:00
Fan Cui
18ba66e012 added Chinese documents and fixed some typos in English documents 2021-11-02 23:28:44 +08:00
zbian
404ecbdcc6 Migrated project 2021-10-28 18:21:23 +02:00