HELSON e7d3afc9cc [optimizer] add div_scale for optimizers (#2117) ...

* [optimizer] add div_scale for optimizers

* [zero] use div_scale in zero optimizer

* fix testing error

2022-12-12 17:58:57 +08:00

..

kernels

Recover kernal files

2022-07-13 12:08:21 +08:00

colossal_C_frontend.cpp

[optimizer] add div_scale for optimizers (#2117)

2022-12-12 17:58:57 +08:00

compat.h

refactor kernel (#142)

2022-01-13 16:47:17 +08:00

cpu_adam.cpp

[hotfix] fix CPUAdam kernel nullptr (#1410)

2022-08-05 19:45:45 +08:00

cpu_adam.h

[hotfix] fix CPUAdam kernel nullptr (#1410)

2022-08-05 19:45:45 +08:00

layer_norm_cuda_kernel.cu

[NFC] polish colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu code style (#661)

2022-04-06 11:40:59 +08:00

layer_norm_cuda.cpp

[NFC] polish colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp code style (#973)

2022-05-17 10:25:06 +08:00

moe_cuda_kernel.cu

[NFC] polish moe_cuda_kernel.cu code style (#940)

2022-05-17 10:25:06 +08:00

moe_cuda.cpp

[NFC] polish colossalai/kernel/cuda_native/csrc/moe_cuda.cpp code style (#942)

2022-05-17 10:25:06 +08:00

multi_tensor_adam.cu

[optimizer] add div_scale for optimizers (#2117)

2022-12-12 17:58:57 +08:00

multi_tensor_apply.cuh

Recover kernal files

2022-07-13 12:08:21 +08:00

multi_tensor_l2norm_kernel.cu

[NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu code style (#958)

2022-05-17 10:25:06 +08:00

multi_tensor_lamb.cu

[NFC] Polish colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu code style. (#937)

2022-05-17 10:25:06 +08:00

multi_tensor_scale_kernel.cu

[NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu code style (#977)

2022-05-17 10:25:06 +08:00

multi_tensor_sgd_kernel.cu

[optim] refactor fused sgd (#1134)

2022-06-20 11:19:38 +08:00

multihead_attention_1d.cpp

[hotfix] fix build error when torch version >= 1.13 (#1803)

2022-11-08 09:40:24 +08:00

multihead_attention_1d.h

[hotfix] fix build error when torch version >= 1.13 (#1803)

2022-11-08 09:40:24 +08:00

scaled_masked_softmax_cuda.cu

[NFC] polish colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu code style (#949)

2022-05-17 10:25:06 +08:00

scaled_masked_softmax.cpp

Recover kernal files

2022-07-13 12:08:21 +08:00

scaled_masked_softmax.h

Recover kernal files

2022-07-13 12:08:21 +08:00

scaled_upper_triang_masked_softmax_cuda.cu

[NFC] polish pre-commit run --files colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu code style (#943)

2022-05-17 10:25:06 +08:00

scaled_upper_triang_masked_softmax.cpp

[NFC] polish colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp code style (#959)

2022-05-17 10:25:06 +08:00

scaled_upper_triang_masked_softmax.h

Recover kernal files

2022-07-13 12:08:21 +08:00

type_shim.h

Recover kernal files

2022-07-13 12:08:21 +08:00