github/DB-GPT
1
0
Fork 0
mirror of https://github.com/csunny/DB-GPT.git synced 2025-09-12 20:53:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(model): Fix device MPS load llama error (#1033)

Browse Source
This commit is contained in:
Fangyin Cheng
2024-01-05 14:19:13 +08:00
committed by GitHub
parent d8393a9b32
commit 186b6a5668
2 changed files with 16 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
dbgpt/model/llm/monkey_patch.py
View File

@@ -34,6 +34,7 @@ def forward(
past_key_value: Optional[Tuple[torch.Tensor]] = None,
output_attentions: bool = False,
use_cache: bool = False,
padding_mask: Optional[torch.LongTensor] = None,
) -> Tuple[torch.Tensor, Optional[torch.Tensor], Optional[Tuple[torch.Tensor]]]:
bsz, q_len, _ = hidden_states.size()