Harrison/mmr support for opensearch (#6349)

Co-authored-by: Mehmet Öner Yalçın <oneryalcin@gmail.com>
This commit is contained in:
Harrison Chase
2023-06-17 12:22:37 -07:00
committed by GitHub
parent 2eea5d4cb4
commit a2bbe3dda4
2 changed files with 140 additions and 60 deletions

View File

@@ -129,11 +129,7 @@
"cell_type": "code",
"execution_count": null,
"id": "db3fa309",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"query = \"What did the president say about Ketanji Brown Jackson\"\n",
@@ -144,11 +140,7 @@
"cell_type": "code",
"execution_count": null,
"id": "c160d5bb",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"print(docs[0].page_content)"
@@ -158,11 +150,7 @@
"cell_type": "code",
"execution_count": null,
"id": "96215c90",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"docsearch = OpenSearchVectorSearch.from_documents(\n",
@@ -183,11 +171,7 @@
"cell_type": "code",
"execution_count": null,
"id": "62a7cea0",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"print(docs[0].page_content)"
@@ -207,11 +191,7 @@
"cell_type": "code",
"execution_count": null,
"id": "0a8e3c0e",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"docsearch = OpenSearchVectorSearch.from_documents(\n",
@@ -230,11 +210,7 @@
"cell_type": "code",
"execution_count": null,
"id": "92bc40db",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"print(docs[0].page_content)"
@@ -254,11 +230,7 @@
"cell_type": "code",
"execution_count": null,
"id": "6d9f436e",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"docsearch = OpenSearchVectorSearch.from_documents(\n",
@@ -278,16 +250,34 @@
"cell_type": "code",
"execution_count": null,
"id": "8ca50bce",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"print(docs[0].page_content)"
]
},
{
"cell_type": "markdown",
"source": [
"### Maximum marginal relevance search (MMR)\n",
"If youd like to look up for some similar documents, but youd also like to receive diverse results, MMR is method you should consider. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents."
],
"metadata": {
"collapsed": false
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"query = \"What did the president say about Ketanji Brown Jackson\"\n",
"docs = docsearch.max_marginal_relevance_search(query, k=2, fetch_k=10, lambda_param=0.5)"
],
"metadata": {
"collapsed": false
}
},
{
"cell_type": "markdown",
"id": "73264864",