github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-06-03 13:43:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

reorder eval docs (#11738)

Browse Source 
cc @leo-gan
This commit is contained in:
Bagatur 2023-10-12 15:46:55 -07:00 committed by GitHub
parent 35965df20d
commit 01b7b46908
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 919 additions and 890 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
docs/docs/guides/evaluation/comparison/custom.ipynb
View File

@ -1,11 +1,21 @@
{
"cells": [
{
"cell_type": "raw",
"id": "5046d96f-d578-4d5b-9a7e-43b28cafe61d",
"metadata": {},
"source": [
"---\n",
"sidebar_position: 2\n",
"title: Custom pairwise evaluator\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "657d2c8c-54b4-42a3-9f02-bdefa0ed6728",
"metadata": {},
"source": [
"# Custom Pairwise Evaluator\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/guides/evaluation/comparison/custom.ipynb)\n",
"\n",
"You can make your own pairwise string evaluators by inheriting from `PairwiseStringEvaluator` class and overwriting the `_evaluate_string_pairs` method (and the `_aevaluate_string_pairs` method if you want to use the evaluator asynchronously).\n",
@ -273,7 +283,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.2"
"version": "3.9.1"
}
},
"nbformat": 4,

13
docs/docs/guides/evaluation/comparison/pairwise_embedding_distance.ipynb
View File

@ -1,5 +1,15 @@
{
"cells": [
{
"cell_type": "raw",
"metadata": {},
"source": [
"---\n",
"sidebar_position: 1\n",
"title: Pairwise embedding distance\n",
"---"
]
},
{
"attachments": {},
"cell_type": "markdown",
@ -7,7 +17,6 @@
"tags": []
},
"source": [
"# Pairwise Embedding Distance \n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/guides/evaluation/comparison/pairwise_embedding_distance.ipynb)\n",
"\n",
"One way to measure the similarity (or dissimilarity) between two predictions on a shared or similar input is to embed the predictions and compute a vector distance between the two embeddings.<a name=\"cite_ref-1\"></a>[<sup>[1]</sup>](#cite_note-1)\n",
@ -225,7 +234,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.2"
"version": "3.9.1"
}
},
"nbformat": 4,

14
docs/docs/guides/evaluation/comparison/pairwise_string.ipynb
View File

@ -1,11 +1,21 @@
{
"cells": [
{
"cell_type": "raw",
"id": "dcfcf124-78fe-4d67-85a4-cfd3409a1ff6",
"metadata": {},
"source": [
"---\n",
"sidebar_position: 0\n",
"title: Pairwise string comparison\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "2da95378",
"metadata": {},
"source": [
"# Pairwise String Comparison\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/guides/evaluation/comparison/pairwise_string.ipynb)\n",
"\n",
"Often you will want to compare predictions of an LLM, Chain, or Agent for a given input. The `StringComparison` evaluators facilitate this so you can answer questions like:\n",
@ -374,7 +384,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.2"
"version": "3.9.1"
}
},
"nbformat": 4,