github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-23 18:39:56 +00:00
Code Issues Packages Projects Releases Wiki Activity

Optimized some syntax errors in the documentation and code under applications/ (#4127)

Browse Source 
Co-authored-by: flybird11111 <1829166702@qq.com>
This commit is contained in:
digger yu
2023-09-15 14:18:22 +08:00
committed by GitHub
parent 50e5602c2d
commit e4fc57c3de
8 changed files with 12 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
applications/Chat/coati/models/lora.py
View File

@@ -48,7 +48,7 @@ class LoraLinear(lora.LoRALayer, nn.Module):
def reset_parameters(self):
if hasattr(self, 'lora_A'):
# initialize A the same way as the default for nn.Linear and B to zero
# Initialize A with the default values for nn.Linear and set B to zero.
nn.init.kaiming_uniform_(self.lora_A, a=math.sqrt(5))
nn.init.zeros_(self.lora_B)