github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-08-30 23:56:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

[chat] fix compute_approx_kl (#4338)

Browse Source
This commit is contained in:
Wenhao Chen 2023-08-01 10:21:45 +08:00 committed by GitHub
parent 03654c0ce2
commit 75c5389037
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
applications/Chat/coati/models/utils.py
View File

@ -19,7 +19,7 @@ def compute_approx_kl(log_probs: torch.Tensor,
action_mask: Mask for actions.
"""
log_ratio = log_probs - log_probs_base
log_ratio = log_probs_base - log_probs
approx_kl = (log_ratio.exp() - 1) - log_ratio
if action_mask is not None:
approx_kl = masked_mean(approx_kl, action_mask, dim=1)