William FH 36204c2baf Evaluation Callback Multi Response (#12505) ...

1. Allow run evaluators to return {"results": [list of evaluation
results]} in the evaluator callback.
2. Allows run evaluators to pick the target run ID to provide feedback
to

(1) means you could do something like a function call that populates a
full rubric in one go (not sure how reliable that is in general though)
rather than splitting off into separate LLM calls - cheaper and less
code to write
(2) means you can provide feedback to runs on subsequent calls.
Immediate use case is if you wanted to add an evaluator to a chat bot
and assign to assign to previous conversation turns


have a corresponding one in the SDK

2023-10-28 23:18:29 -07:00

..

cli

various templates improvements (#12500)

2023-10-28 22:13:22 -07:00

experimental

bump to 36 (#12487)

2023-10-28 08:51:23 -07:00

langchain

Evaluation Callback Multi Response (#12505)

2023-10-28 23:18:29 -07:00