HELSON a7d95b7024 [example] add zero1, zero2 example in GPT examples (#2146) ...

* [example] add zero1 and zero2 for GPT

* update readme in gpt example

* polish code

* change init value

* update readme

2022-12-20 14:30:27 +08:00

..

bookkeeping

[zero] migrate zero1&2 (#1878)

2022-11-11 09:26:40 +08:00

__init__.py

[zero] migrate zero1&2 (#1878)

2022-11-11 09:26:40 +08:00

_utils.py

[zero] test gradient accumulation (#1964)

2022-11-29 13:00:30 +08:00

low_level_optim.py

[example] add zero1, zero2 example in GPT examples (#2146)

2022-12-20 14:30:27 +08:00

sharded_optim_v2.py

fix move fp32 shards (#1604)

2022-09-16 17:33:16 +08:00