1
0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-05-06 07:28:12 +00:00
ColossalAI/colossalai/kernel/cuda_native/csrc
Xu Kai 611a5a80ca
[inference] Add smmoothquant for llama ()
* [inference] add int8 rotary embedding kernel for smoothquant ()

* [inference] add smoothquant llama attention ()

* add smoothquant llama attention

* remove uselss code

* remove useless code

* fix import error

* rename file name

* [inference] add silu linear fusion for smoothquant llama mlp  ()

* add silu linear

* update skip condition

* catch smoothquant cuda lib exception

* prcocess exception for tests

* [inference] add llama mlp for smoothquant ()

* add llama mlp for smoothquant

* fix down out scale

* remove duplicate lines

* add llama mlp check

* delete useless code

* [inference] add smoothquant llama ()

* add smoothquant llama

* fix attention accuracy

* fix accuracy

* add kv cache and save pretrained

* refactor example

* delete smooth

* refactor code

* [inference] add smooth function and delete useless code for smoothquant ()

* add smooth function and delete useless code

* update datasets

* remove duplicate import

* delete useless file

* refactor codes ()

* rafactor code

* add license

* add torch-int and smoothquant license
2023-10-16 11:28:44 +08:00
..
gptq [NFC] polish code style () 2023-10-07 13:36:52 +08:00
kernels [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
smoothquant [inference] Add smmoothquant for llama () 2023-10-16 11:28:44 +08:00
colossal_C_frontend.cpp [optimizer] add div_scale for optimizers () 2022-12-12 17:58:57 +08:00
compat.h [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
cpu_adam.cpp [hotfix] fix CPUAdam kernel nullptr () 2022-08-05 19:45:45 +08:00
cpu_adam.h [hotfix] fix CPUAdam kernel nullptr () 2022-08-05 19:45:45 +08:00
layer_norm_cuda_kernel.cu [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
layer_norm_cuda.cpp [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
moe_cuda_kernel.cu [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
moe_cuda.cpp [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
multi_tensor_adam.cu [doc] add deepspeed citation and copyright () 2023-03-04 20:08:11 +08:00
multi_tensor_apply.cuh [doc] add deepspeed citation and copyright () 2023-03-04 20:08:11 +08:00
multi_tensor_l2norm_kernel.cu [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
multi_tensor_lamb.cu [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
multi_tensor_scale_kernel.cu [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
multi_tensor_sgd_kernel.cu [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
multihead_attention_1d.cpp [hotfix] fix error for torch 2.0 () 2022-12-30 23:11:55 +08:00
multihead_attention_1d.h [hotfix] fix error for torch 2.0 () 2022-12-30 23:11:55 +08:00
scaled_masked_softmax_cuda.cu [NFC] polish colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu code style () 2022-05-17 10:25:06 +08:00
scaled_masked_softmax.cpp [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
scaled_masked_softmax.h [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
scaled_upper_triang_masked_softmax_cuda.cu [NFC] polish pre-commit run --files colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu code style () 2022-05-17 10:25:06 +08:00
scaled_upper_triang_masked_softmax.cpp [NFC] polish colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp code style () 2022-05-17 10:25:06 +08:00
scaled_upper_triang_masked_softmax.h [misc] update pre-commit and run all files () 2023-09-19 14:20:26 +08:00
type_shim.h [bf16] add bf16 support () 2023-06-05 15:58:31 +08:00