Enhanced pairwise error (#11131)

This commit is contained in:
William FH
2023-09-27 16:04:43 -07:00
committed by GitHub
parent b7e9db5e73
commit d3c2ca5656
3 changed files with 21 additions and 3 deletions

View File

@@ -16,6 +16,10 @@ Here's a summary of the key methods and properties of a comparison evaluator:
- `requires_input`: This property indicates whether this evaluator requires an input string.
- `requires_reference`: This property specifies whether this evaluator requires a reference label.
:::note LangSmith Support
The [run_on_dataset](https://api.python.langchain.com/en/latest/api_reference.html#module-langchain.smith) evaluation method is designed to evaluate only a single model at a time, and thus, doesn't support these evaluators.
:::
Detailed information about creating custom evaluators and the available built-in comparison evaluators is provided in the following sections.
import DocCardList from "@theme/DocCardList";

View File

@@ -17,7 +17,7 @@
"source": [
"## Installation\n",
"\n",
"To get started with Iris Takeoff, all you need is to have docker and python installed on your local system. If you wish to use the server with gpu suport, then you will need to install docker with cuda support.\n",
"To get started with Iris Takeoff, all you need is to have docker and python installed on your local system. If you wish to use the server with gpu support, then you will need to install docker with cuda support.\n",
"\n",
"For Mac and Windows users, make sure you have the docker daemon running! You can check this by running docker ps in your terminal. To start the daemon, open the docker desktop app.\n",
"\n",
@@ -157,7 +157,8 @@
"metadata": {},
"outputs": [],
"source": [
"from langchain.prompts import PromptTemplate\nfrom langchain.chains import LLMChain\n",
"from langchain.prompts import PromptTemplate\n",
"from langchain.chains import LLMChain\n",
"\n",
"llm = TitanTakeoff()\n",
"\n",