HELSON 1468e4bcfc [zero] add constant placement policy (#1705) ...

* fixes memory leak when paramter is in fp16 in ZeroDDP init.
* bans chunk releasement in CUDA. Only when a chunk is about to offload, it is allowed to release.
* adds a constant placement policy. With it, users can allocate a reserved caching memory space for parameters.

2022-10-14 17:53:16 +08:00

..

process_group_initializer

[usability] improved error messages in the context module (#856)

2022-04-25 13:42:31 +08:00

random

[doc] update rst and docstring (#1351)

2022-07-21 15:54:53 +08:00

__init__.py

[polish] polish singleton and global context (#500)

2022-03-23 18:03:39 +08:00

config.py

Refactored docstring to google style

2022-03-29 17:17:47 +08:00

moe_context.py

[zero] add constant placement policy (#1705)

2022-10-14 17:53:16 +08:00

parallel_context.py

[usability] improved error messages in the context module (#856)

2022-04-25 13:42:31 +08:00

parallel_mode.py

[MOE] remove old MoE legacy (#493)

2022-03-22 17:37:16 +08:00

singleton_meta.py

[autoparallel] refactored shape consistency to remove redundancy (#1591)

2022-09-13 18:30:18 +08:00