github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2026-01-20 09:34:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

[NPU]support npu (#6089)

Browse Source 
* support npu

* support pretrain

support pretrain

fix

* support lora

fix

fix

* support chatglm

fix

fxi

fix

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

fix

fix

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

fix

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

fix

fix

fix

* Update train.py

* Update train.py
This commit is contained in:
flybird11111
2024-11-20 15:28:35 +08:00
committed by GitHub
parent 4c8e85ee0d
commit 64f74a157e
18 changed files with 296 additions and 153 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
extensions/pybind/flash_attention/flash_attention_npu.py
View File

@@ -27,6 +27,7 @@ class FlashAttentionNpuExtension(_Extension):
)
def load(self):
import math
from typing import Optional
import torch
@@ -47,6 +48,8 @@ class FlashAttentionNpuExtension(_Extension):
q_indices: Optional[torch.Tensor] = None,
kv_indices: Optional[torch.Tensor] = None,
):
if scale is None:
scale = 1.0 / math.sqrt(q.size(-1))
num_heads = q.size(1)
return torch_npu.npu_fusion_attention(
q,