HELSON 1468e4bcfc [zero] add constant placement policy (#1705) ...

* fixes memory leak when paramter is in fp16 in ZeroDDP init.
* bans chunk releasement in CUDA. Only when a chunk is about to offload, it is allowed to release.
* adds a constant placement policy. With it, users can allocate a reserved caching memory space for parameters.

2022-10-14 17:53:16 +08:00

..

__init__.py

fix import error (#880)

2022-04-26 19:28:40 +08:00

colo_init_context.py

[zero] add constant placement policy (#1705)

2022-10-14 17:53:16 +08:00

lazy_init_context.py

[hotfix] fix init context (#1543)

2022-09-06 11:45:08 +08:00

utils.py

[hotfix] fix init context (#1543)

2022-09-06 11:45:08 +08:00