mirror of
https://github.com/hwchase17/langchain.git
synced 2025-08-09 13:00:34 +00:00
docs(langchain): complete PDF embedding example for OpenAI, also some minor doc fixes (#32426)
For OpenAI PDF attaching, note the needed metadata. Also some minor doc updates.
This commit is contained in:
parent
ba83f58141
commit
2543007436
@ -24,7 +24,7 @@
|
|||||||
"\n",
|
"\n",
|
||||||
":::tip\n",
|
":::tip\n",
|
||||||
"\n",
|
"\n",
|
||||||
"The **default** implementation does **not** provide support for token-by-token streaming, but it ensures that the the model can be swapped in for any other model as it supports the same standard interface.\n",
|
"The **default** implementation does **not** provide support for token-by-token streaming, but it ensures that the model can be swapped in for any other model as it supports the same standard interface.\n",
|
||||||
"\n",
|
"\n",
|
||||||
":::\n",
|
":::\n",
|
||||||
"\n",
|
"\n",
|
||||||
|
@ -323,7 +323,7 @@
|
|||||||
"source": [
|
"source": [
|
||||||
"## RAG based approach\n",
|
"## RAG based approach\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Another simple idea is to chunk up the text, but instead of extracting information from every chunk, just focus on the the most relevant chunks.\n",
|
"Another simple idea is to chunk up the text, but instead of extracting information from every chunk, just focus on the most relevant chunks.\n",
|
||||||
"\n",
|
"\n",
|
||||||
":::caution\n",
|
":::caution\n",
|
||||||
"It can be difficult to identify which chunks are relevant.\n",
|
"It can be difficult to identify which chunks are relevant.\n",
|
||||||
|
@ -104,7 +104,7 @@
|
|||||||
"source": [
|
"source": [
|
||||||
"## Chaining\n",
|
"## Chaining\n",
|
||||||
"\n",
|
"\n",
|
||||||
"`filter_messages` can be used in an imperatively (like above) or declaratively, making it easy to compose with other components in a chain:"
|
"`filter_messages` can be used imperatively (like above) or declaratively, making it easy to compose with other components in a chain:"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
@ -199,7 +199,7 @@
|
|||||||
"outputs": [],
|
"outputs": [],
|
||||||
"source": [
|
"source": [
|
||||||
"def _clear():\n",
|
"def _clear():\n",
|
||||||
" \"\"\"Hacky helper method to clear content. See the `full` mode section to to understand why it works.\"\"\"\n",
|
" \"\"\"Hacky helper method to clear content. See the `full` mode section to understand why it works.\"\"\"\n",
|
||||||
" index([], record_manager, vectorstore, cleanup=\"full\", source_id_key=\"source\")"
|
" index([], record_manager, vectorstore, cleanup=\"full\", source_id_key=\"source\")"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
|
@ -88,7 +88,7 @@
|
|||||||
"source": [
|
"source": [
|
||||||
"## Chaining\n",
|
"## Chaining\n",
|
||||||
"\n",
|
"\n",
|
||||||
"`merge_message_runs` can be used in an imperatively (like above) or declaratively, making it easy to compose with other components in a chain:"
|
"`merge_message_runs` can be used imperatively (like above) or declaratively, making it easy to compose with other components in a chain:"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
@ -310,7 +310,7 @@
|
|||||||
"from langchain_openai import OpenAI\n",
|
"from langchain_openai import OpenAI\n",
|
||||||
"\n",
|
"\n",
|
||||||
"chain = load_qa_chain(llm=OpenAI(), chain_type=\"map_reduce\")\n",
|
"chain = load_qa_chain(llm=OpenAI(), chain_type=\"map_reduce\")\n",
|
||||||
"query = [\"Who are the autors?\"]\n",
|
"query = [\"Who are the authors?\"]\n",
|
||||||
"\n",
|
"\n",
|
||||||
"chain.run(input_documents=documents, question=query)"
|
"chain.run(input_documents=documents, question=query)"
|
||||||
]
|
]
|
||||||
|
@ -15,7 +15,7 @@
|
|||||||
"source": [
|
"source": [
|
||||||
"[Socrata](https://dev.socrata.com/foundry/data.sfgov.org/vw6y-z8j6) provides an API for city open data. \n",
|
"[Socrata](https://dev.socrata.com/foundry/data.sfgov.org/vw6y-z8j6) provides an API for city open data. \n",
|
||||||
"\n",
|
"\n",
|
||||||
"For a dataset such as [SF crime](https://data.sfgov.org/Public-Safety/Police-Department-Incident-Reports-Historical-2003/tmnf-yvry), to to the `API` tab on top right. \n",
|
"For a dataset such as [SF crime](https://data.sfgov.org/Public-Safety/Police-Department-Incident-Reports-Historical-2003/tmnf-yvry), see the `API` tab on top right. \n",
|
||||||
"\n",
|
"\n",
|
||||||
"That provides you with the `dataset identifier`.\n",
|
"That provides you with the `dataset identifier`.\n",
|
||||||
"\n",
|
"\n",
|
||||||
|
@ -117,7 +117,7 @@
|
|||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"The fields:\n",
|
"The fields:\n",
|
||||||
" - `es_host_url` is the endpoint to to MetadataIQ Elasticsearch database\n",
|
" - `es_host_url` is the endpoint to MetadataIQ Elasticsearch database\n",
|
||||||
" - `es_index_index` is the name of the index where PowerScale writes it file system metadata\n",
|
" - `es_index_index` is the name of the index where PowerScale writes it file system metadata\n",
|
||||||
" - `es_api_key` is the **encoded** version of your elasticsearch API key\n",
|
" - `es_api_key` is the **encoded** version of your elasticsearch API key\n",
|
||||||
" - `folder_path` is the path on PowerScale to be queried for changes"
|
" - `folder_path` is the path on PowerScale to be queried for changes"
|
||||||
|
@ -11,7 +11,7 @@
|
|||||||
"\n",
|
"\n",
|
||||||
"This loader fetches the text from the Posts of Subreddits or Reddit users, using the `praw` Python package.\n",
|
"This loader fetches the text from the Posts of Subreddits or Reddit users, using the `praw` Python package.\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Make a [Reddit Application](https://www.reddit.com/prefs/apps/) and initialize the loader with with your Reddit API credentials."
|
"Make a [Reddit Application](https://www.reddit.com/prefs/apps/) and initialize the loader with your Reddit API credentials."
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
@ -151,10 +151,10 @@
|
|||||||
"Red arrow magic !\n",
|
"Red arrow magic !\n",
|
||||||
"Something white\n",
|
"Something white\n",
|
||||||
"Something Red\n",
|
"Something Red\n",
|
||||||
"This a a completly useless diagramm, cool !!\n",
|
"This a completely useless diagram, cool !!\n",
|
||||||
"\n",
|
"\n",
|
||||||
"But this is for example !\n",
|
"But this is for example !\n",
|
||||||
"This diagramm is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
"This diagram is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
||||||
"This is a page with something...\n",
|
"This is a page with something...\n",
|
||||||
"\n",
|
"\n",
|
||||||
"WAW I have learned something !\n",
|
"WAW I have learned something !\n",
|
||||||
@ -183,10 +183,10 @@
|
|||||||
"This is a title\n",
|
"This is a title\n",
|
||||||
"Something white\n",
|
"Something white\n",
|
||||||
"Something Red\n",
|
"Something Red\n",
|
||||||
"This a a completly useless diagramm, cool !!\n",
|
"This a completely useless diagram, cool !!\n",
|
||||||
"\n",
|
"\n",
|
||||||
"But this is for example !\n",
|
"But this is for example !\n",
|
||||||
"This diagramm is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
"This diagram is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
||||||
"Another RED arrow wow\n",
|
"Another RED arrow wow\n",
|
||||||
"Arrow with point but red\n",
|
"Arrow with point but red\n",
|
||||||
"Green line\n",
|
"Green line\n",
|
||||||
@ -219,10 +219,10 @@
|
|||||||
"Red arrow magic !\n",
|
"Red arrow magic !\n",
|
||||||
"Something white\n",
|
"Something white\n",
|
||||||
"Something Red\n",
|
"Something Red\n",
|
||||||
"This a a completly useless diagramm, cool !!\n",
|
"This a completely useless diagram, cool !!\n",
|
||||||
"\n",
|
"\n",
|
||||||
"But this is for example !\n",
|
"But this is for example !\n",
|
||||||
"This diagramm is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
"This diagram is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
||||||
"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor\n",
|
"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor\n",
|
||||||
"\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0-\\u00a0incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in\n",
|
"\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0\\u00a0-\\u00a0incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in\n",
|
||||||
"\n",
|
"\n",
|
||||||
@ -252,10 +252,10 @@
|
|||||||
"This is a title\n",
|
"This is a title\n",
|
||||||
"Something white\n",
|
"Something white\n",
|
||||||
"Something Red\n",
|
"Something Red\n",
|
||||||
"This a a completly useless diagramm, cool !!\n",
|
"This a completely useless diagram, cool !!\n",
|
||||||
"\n",
|
"\n",
|
||||||
"But this is for example !\n",
|
"But this is for example !\n",
|
||||||
"This diagramm is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
"This diagram is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
||||||
"\n",
|
"\n",
|
||||||
"------ Page 7 ------\n",
|
"------ Page 7 ------\n",
|
||||||
"Title page : Useful ↔ Useless page\n",
|
"Title page : Useful ↔ Useless page\n",
|
||||||
@ -276,10 +276,10 @@
|
|||||||
"This is a title\n",
|
"This is a title\n",
|
||||||
"Something white\n",
|
"Something white\n",
|
||||||
"Something Red\n",
|
"Something Red\n",
|
||||||
"This a a completly useless diagramm, cool !!\n",
|
"This a completely useless diagram, cool !!\n",
|
||||||
"\n",
|
"\n",
|
||||||
"But this is for example !\n",
|
"But this is for example !\n",
|
||||||
"This diagramm is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
"This diagram is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
||||||
"Title of this document : BLABLABLA\n",
|
"Title of this document : BLABLABLA\n",
|
||||||
"\n",
|
"\n",
|
||||||
"------ Page 8 ------\n",
|
"------ Page 8 ------\n",
|
||||||
@ -359,10 +359,10 @@
|
|||||||
"Red arrow magic !\n",
|
"Red arrow magic !\n",
|
||||||
"Something white\n",
|
"Something white\n",
|
||||||
"Something Red\n",
|
"Something Red\n",
|
||||||
"This a a completly useless diagramm, cool !!\n",
|
"This a completely useless diagram, cool !!\n",
|
||||||
"\n",
|
"\n",
|
||||||
"But this is for example !\n",
|
"But this is for example !\n",
|
||||||
"This diagramm is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
"This diagram is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
||||||
"Useful\\u2194 Useless page\\u00a0\n",
|
"Useful\\u2194 Useless page\\u00a0\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Tests of some exotics characters :\\u00a0\\u00e3\\u00e4\\u00e5\\u0101\\u0103 \\u00fc\\u2554\\u00a0\\u00a0\\u00bc \\u00c7 \\u25d8\\u25cb\\u2642\\u266b\\u2640\\u00ee\\u2665\n",
|
"Tests of some exotics characters :\\u00a0\\u00e3\\u00e4\\u00e5\\u0101\\u0103 \\u00fc\\u2554\\u00a0\\u00a0\\u00bc \\u00c7 \\u25d8\\u25cb\\u2642\\u266b\\u2640\\u00ee\\u2665\n",
|
||||||
@ -444,10 +444,10 @@
|
|||||||
"Red arrow magic !\n",
|
"Red arrow magic !\n",
|
||||||
"Something white\n",
|
"Something white\n",
|
||||||
"Something Red\n",
|
"Something Red\n",
|
||||||
"This a a completly useless diagramm, cool !!\n",
|
"This a completely useless diagram, cool !!\n",
|
||||||
"\n",
|
"\n",
|
||||||
"But this is for example !\n",
|
"But this is for example !\n",
|
||||||
"This diagramm is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
"This diagram is a base of many pages in this file. But it is editable in file \\\"BG WITH CONTENT\\\"\n",
|
||||||
"Only connectors on this page. This is the CoNNeCtor page\n"
|
"Only connectors on this page. This is the CoNNeCtor page\n"
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
@ -19,7 +19,7 @@
|
|||||||
"\n",
|
"\n",
|
||||||
"> [Firestore](https://cloud.google.com/firestore) is a serverless document-oriented database that scales to meet any demand. Extend your database application to build AI-powered experiences leveraging Firestore's Langchain integrations.\n",
|
"> [Firestore](https://cloud.google.com/firestore) is a serverless document-oriented database that scales to meet any demand. Extend your database application to build AI-powered experiences leveraging Firestore's Langchain integrations.\n",
|
||||||
"\n",
|
"\n",
|
||||||
"This notebook goes over how to use [Firestore](https://cloud.google.com/firestore) to to store vectors and query them using the `FirestoreVectorStore` class.\n",
|
"This notebook goes over how to use [Firestore](https://cloud.google.com/firestore) to store vectors and query them using the `FirestoreVectorStore` class.\n",
|
||||||
"\n",
|
"\n",
|
||||||
"[](https://colab.research.google.com/github/googleapis/langchain-google-firestore-python/blob/main/docs/vectorstores.ipynb)"
|
"[](https://colab.research.google.com/github/googleapis/langchain-google-firestore-python/blob/main/docs/vectorstores.ipynb)"
|
||||||
]
|
]
|
||||||
|
@ -43,7 +43,7 @@
|
|||||||
"id": "0dd87fcc",
|
"id": "0dd87fcc",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"You can run the following command to spin up a a postgres container with the `pgvector` extension:"
|
"You can run the following command to spin up a postgres container with the `pgvector` extension:"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
@ -193,7 +193,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"You should additionally not pass `ToolMessages` back to to a model if they are not preceded by an `AIMessage` with tool calls. For example, this will fail:"
|
"You should additionally not pass `ToolMessages` back to a model if they are not preceded by an `AIMessage` with tool calls. For example, this will fail:"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
Loading…
Reference in New Issue
Block a user