HELSON e5ea3fdeef [gemini] add GeminiMemoryManger (#832) ...

* refactor StatefulTensor, tensor utilities

* add unitest for GeminiMemoryManager

2022-04-24 13:08:48 +08:00

..

colossalai_layer

[TP] allow layernorm without bias (#750)

2022-04-14 11:43:56 +08:00

moe

[gemini] add GeminiMemoryManger (#832)

2022-04-24 13:08:48 +08:00

parallel_1d

[TP] allow layernorm without bias (#750)

2022-04-14 11:43:56 +08:00

parallel_2d

[TP] change the check assert in split batch 2d (#772)

2022-04-16 21:29:57 +08:00

parallel_2p5d

[TP] change the check assert in split batch 2d (#772)

2022-04-16 21:29:57 +08:00

parallel_3d

[TP] allow layernorm without bias (#750)

2022-04-14 11:43:56 +08:00

parallel_sequence

Refactored docstring to google style

2022-03-29 17:17:47 +08:00

utils

Refactored docstring to google style

2022-03-29 17:17:47 +08:00

vanilla

[TP] allow layernorm without bias (#750)

2022-04-14 11:43:56 +08:00

wrapper

Refactored docstring to google style

2022-03-29 17:17:47 +08:00

__init__.py

[MOE] changed parallelmode to dist process group (#460)

2022-03-19 13:46:29 +08:00

base_layer.py

[model checkpoint] reworked unified layers for ease of save/load states (#593)

2022-04-01 16:49:56 +08:00