github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-06-24 14:33:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

[fix] fix test_lora in llama policy

Browse Source
This commit is contained in:
duanjunwen 2024-11-15 10:27:13 +00:00
parent 2980da559f
commit f48a85e91d
1 changed files with 0 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
colossalai/shardformer/policies/llama.py
View File

@ -506,7 +506,6 @@ class LlamaForSequenceClassificationPolicy(LlamaPolicy):
) )
} }
policy.update(new_item) policy.update(new_item)
# TODO: test lora bug here
# enable tp, replace layer to LinearWithGradAccum # enable tp, replace layer to LinearWithGradAccum
elif use_zbv: elif use_zbv:
# add a new item for sequence classification # add a new item for sequence classification