github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-05-30 19:49:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix call _get_keys for custom_evaluator (#9763)

Browse Source 
In the function _load_run_evaluators the function _get_keys was not
called if only custom_evaluators parameter is used


- Description: In the function _load_run_evaluators the function
_get_keys was not called if only custom_evaluators parameter is used,
  - Issue: no issue created for this yet,
  - Dependencies: None,
  - Tag maintainer: @vowelparrot,
  - Twitter handle: Buckler89

---------

Co-authored-by: ddroghini <d.droghini@mflgroup.com>
This commit is contained in:
Buckler89 2023-08-30 15:35:23 +02:00 committed by GitHub
parent 2d2b097fab
commit a28e888b36
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 7 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
libs/langchain/langchain/smith/evaluation/runner_utils.py
View File

@ -566,8 +566,13 @@ def _load_run_evaluators(
eval_llm = config.eval_llm or ChatOpenAI(model="gpt-4", temperature=0.0)
run_evaluators = []
input_key, prediction_key, reference_key = None, None, None
if config.evaluators or any(
[isinstance(e, EvaluatorType) for e in config.evaluators]
if (
config.evaluators
or any([isinstance(e, EvaluatorType) for e in config.evaluators])
or (
config.custom_evaluators
and any([isinstance(e, StringEvaluator) for e in config.custom_evaluators])
)
):
input_key, prediction_key, reference_key = _get_keys(
config, run_inputs, run_outputs, example_outputs