mirror of
https://github.com/hwchase17/langchain.git
synced 2026-02-21 06:33:41 +00:00
db7e7b69e3
**Description:** There is one unwanted duplicate cell in refine section of summarization documentation, i have removed it.
708 lines
28 KiB
Plaintext
708 lines
28 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "raw",
|
|
"id": "2aca8168-62ec-4bba-93f0-73da08cd1920",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"title: Summarize Text\n",
|
|
"sidebar_class_name: hidden\n",
|
|
"---"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cf13f702",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Summarize Text\n",
|
|
"\n",
|
|
"Suppose you have a set of documents (PDFs, Notion pages, customer questions, etc.) and you want to summarize the content. \n",
|
|
"\n",
|
|
"LLMs are a great tool for this given their proficiency in understanding and synthesizing text.\n",
|
|
"\n",
|
|
"In the context of [retrieval-augmented generation](/docs/tutorials/rag), summarizing text can help distill the information in a large number of retrieved documents to provide context for a LLM.\n",
|
|
"\n",
|
|
"In this walkthrough we'll go over how to summarize content from multiple documents using LLMs."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8e233997",
|
|
"metadata": {},
|
|
"source": [
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cc8c5f87-3239-44e1-8772-a97cb6138cc5",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Concepts\n",
|
|
"\n",
|
|
"Concepts we will cover are:\n",
|
|
"\n",
|
|
"- Using [language models](/docs/concepts/#chat-models).\n",
|
|
"\n",
|
|
"- Using [document loaders](/docs/concepts/#document-loaders), specifically the [WebBaseLoader](https://api.python.langchain.com/en/latest/document_loaders/langchain_community.document_loaders.web_base.WebBaseLoader.html) to load content from an HTML webpage.\n",
|
|
"\n",
|
|
"- Three ways to summarize or otherwise combine documents.\n",
|
|
" 1. [Stuff](/docs/tutorials/summarization#stuff), which simply concatenates documents into a prompt;\n",
|
|
" 2. [Map-reduce](/docs/tutorials/summarization#map-reduce), which splits documents into batches, summarizes those, and then summarizes the summaries;\n",
|
|
" 3. [Refine](/docs/tutorials/summarization#refine), which updates a rolling summary be iterating over the documents in a sequence.\n",
|
|
"\n",
|
|
"That's a fair amount to cover! Let's dive in.\n",
|
|
"\n",
|
|
"## Setup\n",
|
|
"\n",
|
|
"### Jupyter Notebook\n",
|
|
"\n",
|
|
"This guide (and most of the other guides in the documentation) uses [Jupyter notebooks](https://jupyter.org/) and assumes the reader is as well. Jupyter notebooks are perfect for learning how to work with LLM systems because oftentimes things can go wrong (unexpected output, API down, etc) and going through guides in an interactive environment is a great way to better understand them.\n",
|
|
"\n",
|
|
"This and other tutorials are perhaps most conveniently run in a Jupyter notebook. See [here](https://jupyter.org/install) for instructions on how to install.\n",
|
|
"\n",
|
|
"### Installation\n",
|
|
"\n",
|
|
"To install LangChain run:\n",
|
|
"\n",
|
|
"```{=mdx}\n",
|
|
"import Tabs from '@theme/Tabs';\n",
|
|
"import TabItem from '@theme/TabItem';\n",
|
|
"import CodeBlock from \"@theme/CodeBlock\";\n",
|
|
"\n",
|
|
"<Tabs>\n",
|
|
" <TabItem value=\"pip\" label=\"Pip\" default>\n",
|
|
" <CodeBlock language=\"bash\">pip install langchain</CodeBlock>\n",
|
|
" </TabItem>\n",
|
|
" <TabItem value=\"conda\" label=\"Conda\">\n",
|
|
" <CodeBlock language=\"bash\">conda install langchain -c conda-forge</CodeBlock>\n",
|
|
" </TabItem>\n",
|
|
"</Tabs>\n",
|
|
"\n",
|
|
"```\n",
|
|
"\n",
|
|
"\n",
|
|
"For more details, see our [Installation guide](/docs/how_to/installation).\n",
|
|
"\n",
|
|
"### LangSmith\n",
|
|
"\n",
|
|
"Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls.\n",
|
|
"As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent.\n",
|
|
"The best way to do this is with [LangSmith](https://smith.langchain.com).\n",
|
|
"\n",
|
|
"After you sign up at the link above, make sure to set your environment variables to start logging traces:\n",
|
|
"\n",
|
|
"```shell\n",
|
|
"export LANGCHAIN_TRACING_V2=\"true\"\n",
|
|
"export LANGCHAIN_API_KEY=\"...\"\n",
|
|
"```\n",
|
|
"\n",
|
|
"Or, if in a notebook, you can set them with:\n",
|
|
"\n",
|
|
"```python\n",
|
|
"import getpass\n",
|
|
"import os\n",
|
|
"\n",
|
|
"os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n",
|
|
"os.environ[\"LANGCHAIN_API_KEY\"] = getpass.getpass()\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4715b4ff",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Overview\n",
|
|
"\n",
|
|
"A central question for building a summarizer is how to pass your documents into the LLM's context window. Three common approaches for this are:\n",
|
|
"\n",
|
|
"1. `Stuff`: Simply \"stuff\" all your documents into a single prompt. This is the simplest approach (see [here](/docs/tutorials/rag#built-in-chains) for more on the `create_stuff_documents_chain` constructor, which is used for this method).\n",
|
|
"\n",
|
|
"2. `Map-reduce`: Summarize each document on its own in a \"map\" step and then \"reduce\" the summaries into a final summary (see [here](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.map_reduce.MapReduceDocumentsChain.html) for more on the `MapReduceDocumentsChain`, which is used for this method).\n",
|
|
"\n",
|
|
"3. `Refine`: Update a rolling summary be iterating over the documents in a sequence.\n",
|
|
" \n",
|
|
" "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "08ec66bc",
|
|
"metadata": {},
|
|
"source": [
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "bea785ac",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Quickstart\n",
|
|
"\n",
|
|
"To give you a sneak preview, either pipeline can be wrapped in a single object: `load_summarize_chain`. \n",
|
|
"\n",
|
|
"Suppose we want to summarize a blog post. We can create this in a few lines of code.\n",
|
|
"\n",
|
|
"First set environment variables and install packages:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "928585ec-6f6f-4b67-b2c8-0fc87186342b",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install --upgrade --quiet langchain-openai tiktoken chromadb langchain\n",
|
|
"\n",
|
|
"# Set env var OPENAI_API_KEY or load from a .env file\n",
|
|
"# import dotenv\n",
|
|
"\n",
|
|
"# dotenv.load_dotenv()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "36138740",
|
|
"metadata": {},
|
|
"source": [
|
|
"We can use `chain_type=\"stuff\"`, especially if using larger context window models such as:\n",
|
|
"\n",
|
|
"* 128k token OpenAI `gpt-4-turbo-2024-04-09` \n",
|
|
"* 200k token Anthropic `claude-3-sonnet-20240229`\n",
|
|
"\n",
|
|
"We can also supply `chain_type=\"map_reduce\"` or `chain_type=\"refine\"`.\n",
|
|
"\n",
|
|
"First we load in our documents. We will use [WebBaseLoader](https://api.python.langchain.com/en/latest/document_loaders/langchain_community.document_loaders.web_base.WebBaseLoader.html) to load a blog post:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"id": "d6276d52-d33f-4b6a-aae3-2682df9eb8a7",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"\n",
|
|
"os.environ[\"LANGCHAIN_TRACING_V2\"] = \"True\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "23154e97-c4cb-4bcb-a742-f0c9d06639da",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"The article discusses the concept of LLM-powered autonomous agents, with a focus on the components of planning, memory, and tool use. It includes case studies and proof-of-concept examples, as well as challenges and references to related research. The author emphasizes the potential of LLMs in creating powerful problem-solving agents, while also highlighting limitations such as finite context length and reliability of natural language interfaces.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain.chains.summarize import load_summarize_chain\n",
|
|
"from langchain_community.document_loaders import WebBaseLoader\n",
|
|
"from langchain_openai import ChatOpenAI\n",
|
|
"\n",
|
|
"loader = WebBaseLoader(\"https://lilianweng.github.io/posts/2023-06-23-agent/\")\n",
|
|
"docs = loader.load()\n",
|
|
"\n",
|
|
"llm = ChatOpenAI(temperature=0, model_name=\"gpt-3.5-turbo-1106\")\n",
|
|
"chain = load_summarize_chain(llm, chain_type=\"stuff\")\n",
|
|
"\n",
|
|
"result = chain.invoke(docs)\n",
|
|
"\n",
|
|
"print(result[\"output_text\"])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "615b36e1",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Option 1. Stuff {#stuff}\n",
|
|
"\n",
|
|
"When we use `load_summarize_chain` with `chain_type=\"stuff\"`, we will use the [StuffDocumentsChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.stuff.StuffDocumentsChain.html#langchain.chains.combine_documents.stuff.StuffDocumentsChain).\n",
|
|
"\n",
|
|
"The chain will take a list of documents, insert them all into a prompt, and pass that prompt to an LLM:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"id": "ef45585d",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"The article discusses the concept of building autonomous agents powered by large language models (LLMs). It explores the components of such agents, including planning, memory, and tool use. The article provides case studies and examples of proof-of-concept demos, highlighting the challenges and limitations of LLM-powered agents. It also includes references to related research papers and projects.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain.chains.combine_documents.stuff import StuffDocumentsChain\n",
|
|
"from langchain.chains.llm import LLMChain\n",
|
|
"from langchain_core.prompts import PromptTemplate\n",
|
|
"\n",
|
|
"# Define prompt\n",
|
|
"prompt_template = \"\"\"Write a concise summary of the following:\n",
|
|
"\"{text}\"\n",
|
|
"CONCISE SUMMARY:\"\"\"\n",
|
|
"prompt = PromptTemplate.from_template(prompt_template)\n",
|
|
"\n",
|
|
"# Define LLM chain\n",
|
|
"llm = ChatOpenAI(temperature=0, model_name=\"gpt-3.5-turbo-16k\")\n",
|
|
"llm_chain = LLMChain(llm=llm, prompt=prompt)\n",
|
|
"\n",
|
|
"# Define StuffDocumentsChain\n",
|
|
"stuff_chain = StuffDocumentsChain(llm_chain=llm_chain, document_variable_name=\"text\")\n",
|
|
"\n",
|
|
"docs = loader.load()\n",
|
|
"print(stuff_chain.invoke(docs)[\"output_text\"])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4e4e4a43",
|
|
"metadata": {},
|
|
"source": [
|
|
"Great! We can see that we reproduce the earlier result using the `load_summarize_chain`.\n",
|
|
"\n",
|
|
"### Go deeper\n",
|
|
"\n",
|
|
"* You can easily customize the prompt. \n",
|
|
"* You can easily try different LLMs, (e.g., [Claude](/docs/integrations/chat/anthropic)) via the `llm` parameter."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ad6cabee",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Option 2. Map-Reduce {#map-reduce}\n",
|
|
"\n",
|
|
"Let's unpack the map reduce approach. For this, we'll first map each document to an individual summary using an `LLMChain`. Then we'll use a `ReduceDocumentsChain` to combine those summaries into a single global summary.\n",
|
|
" \n",
|
|
"First, we specify the LLMChain to use for mapping each document to an individual summary:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"id": "a1e6773c",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain.chains import MapReduceDocumentsChain, ReduceDocumentsChain\n",
|
|
"from langchain_text_splitters import CharacterTextSplitter\n",
|
|
"\n",
|
|
"llm = ChatOpenAI(temperature=0)\n",
|
|
"\n",
|
|
"# Map\n",
|
|
"map_template = \"\"\"The following is a set of documents\n",
|
|
"{docs}\n",
|
|
"Based on this list of docs, please identify the main themes \n",
|
|
"Helpful Answer:\"\"\"\n",
|
|
"map_prompt = PromptTemplate.from_template(map_template)\n",
|
|
"map_chain = LLMChain(llm=llm, prompt=map_prompt)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "272ce8ce-919d-4ded-bbd5-a53a8a30bc66",
|
|
"metadata": {},
|
|
"source": [
|
|
"We can also use the Prompt Hub to store and fetch prompts.\n",
|
|
"\n",
|
|
"This will work with your [LangSmith API key](https://docs.smith.langchain.com/).\n",
|
|
"\n",
|
|
"For example, see the map prompt [here](https://smith.langchain.com/hub/rlm/map-prompt)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"id": "ce48b805-d98b-4e0f-8b9e-3b3e72cad3d3",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain import hub\n",
|
|
"\n",
|
|
"map_prompt = hub.pull(\"rlm/map-prompt\")\n",
|
|
"map_chain = LLMChain(llm=llm, prompt=map_prompt)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "bee3c331",
|
|
"metadata": {},
|
|
"source": [
|
|
"The `ReduceDocumentsChain` handles taking the document mapping results and reducing them into a single output. It wraps a generic `CombineDocumentsChain` (like `StuffDocumentsChain`) but adds the ability to collapse documents before passing it to the `CombineDocumentsChain` if their cumulative size exceeds `token_max`. In this example, we can actually re-use our chain for combining our docs to also collapse our docs.\n",
|
|
"\n",
|
|
"So if the cumulative number of tokens in our mapped documents exceeds 4000 tokens, then we'll recursively pass in the documents in batches of < 4000 tokens to our `StuffDocumentsChain` to create batched summaries. And once those batched summaries are cumulatively less than 4000 tokens, we'll pass them all one last time to the `StuffDocumentsChain` to create the final summary."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"id": "6a718890-99ab-439a-8f79-b9ae9c58ad24",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Reduce\n",
|
|
"reduce_template = \"\"\"The following is set of summaries:\n",
|
|
"{docs}\n",
|
|
"Take these and distill it into a final, consolidated summary of the main themes. \n",
|
|
"Helpful Answer:\"\"\"\n",
|
|
"reduce_prompt = PromptTemplate.from_template(reduce_template)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"id": "f189184a-673e-4530-8a6b-57b091045d87",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Note we can also get this from the prompt hub, as noted above\n",
|
|
"reduce_prompt = hub.pull(\"rlm/reduce-prompt\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"id": "c9d1da97-d590-4a96-82b2-8002d27fd7f6",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"ChatPromptTemplate(input_variables=['docs'], metadata={'lc_hub_owner': 'rlm', 'lc_hub_repo': 'map-prompt', 'lc_hub_commit_hash': 'de4fba345f211a462584fc25b7077e69c1ba6cdcf4e21b7ec9abe457ddb16c87'}, messages=[HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['docs'], template='The following is a set of documents:\\n{docs}\\nBased on this list of docs, please identify the main themes \\nHelpful Answer:'))])"
|
|
]
|
|
},
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"reduce_prompt"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"id": "1edb1b0d",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Run chain\n",
|
|
"reduce_chain = LLMChain(llm=llm, prompt=reduce_prompt)\n",
|
|
"\n",
|
|
"# Takes a list of documents, combines them into a single string, and passes this to an LLMChain\n",
|
|
"combine_documents_chain = StuffDocumentsChain(\n",
|
|
" llm_chain=reduce_chain, document_variable_name=\"docs\"\n",
|
|
")\n",
|
|
"\n",
|
|
"# Combines and iteratively reduces the mapped documents\n",
|
|
"reduce_documents_chain = ReduceDocumentsChain(\n",
|
|
" # This is final chain that is called.\n",
|
|
" combine_documents_chain=combine_documents_chain,\n",
|
|
" # If documents exceed context for `StuffDocumentsChain`\n",
|
|
" collapse_documents_chain=combine_documents_chain,\n",
|
|
" # The maximum number of tokens to group documents into.\n",
|
|
" token_max=4000,\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fdb5ae1a",
|
|
"metadata": {},
|
|
"source": [
|
|
"Combining our map and reduce chains into one:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"id": "22f1cdc2",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Created a chunk of size 1003, which is longer than the specified 1000\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Combining documents by mapping a chain over them, then combining results\n",
|
|
"map_reduce_chain = MapReduceDocumentsChain(\n",
|
|
" # Map chain\n",
|
|
" llm_chain=map_chain,\n",
|
|
" # Reduce chain\n",
|
|
" reduce_documents_chain=reduce_documents_chain,\n",
|
|
" # The variable name in the llm_chain to put the documents in\n",
|
|
" document_variable_name=\"docs\",\n",
|
|
" # Return the results of the map steps in the output\n",
|
|
" return_intermediate_steps=False,\n",
|
|
")\n",
|
|
"\n",
|
|
"text_splitter = CharacterTextSplitter.from_tiktoken_encoder(\n",
|
|
" chunk_size=1000, chunk_overlap=0\n",
|
|
")\n",
|
|
"split_docs = text_splitter.split_documents(docs)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"id": "d7e53f93-c5aa-456a-85f4-a6b3301a34ed",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"The main themes identified in the list of documents provided are related to large language models (LLMs), autonomous agents, prompting, steering language models, natural language processing (NLP), the use of tools to augment language models, reinforcement learning, reasoning, acting, self-reflection, and the integration of language models with external knowledge sources.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"result = map_reduce_chain.invoke(split_docs)\n",
|
|
"\n",
|
|
"print(result[\"output_text\"])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e62c21cf",
|
|
"metadata": {},
|
|
"source": [
|
|
"If we follow the [Langsmith Trace](https://smith.langchain.com/public/3a1a6d51-68e5-4805-8d90-78920ce60a51/r), we can see the the individual LLM summarizations, including the [final call](https://smith.langchain.com/public/69482813-f0b7-46b0-a99f-86d56fc9644a/r) that summarizes the summaries.\n",
|
|
"\n",
|
|
"### Go deeper\n",
|
|
" \n",
|
|
"**Customization** \n",
|
|
"\n",
|
|
"* As shown above, you can customize the LLMs and prompts for map and reduce stages.\n",
|
|
"\n",
|
|
"**Real-world use-case**\n",
|
|
"\n",
|
|
"* See [this blog post](https://blog.langchain.dev/llms-to-improve-documentation/) case-study on analyzing user interactions (questions about LangChain documentation)! \n",
|
|
"* The blog post and associated [repo](https://github.com/mendableai/QA_clustering) also introduce clustering as a means of summarization.\n",
|
|
"* This opens up another path beyond the `stuff` or `map-reduce` approaches that is worth considering.\n",
|
|
"\n",
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f08ff365",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Option 3. Refine {#refine}\n",
|
|
" \n",
|
|
"[RefineDocumentsChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.refine.RefineDocumentsChain.html) is similar to map-reduce:\n",
|
|
"\n",
|
|
"> The refine documents chain constructs a response by looping over the input documents and iteratively updating its answer. For each document, it passes all non-document inputs, the current document, and the latest intermediate answer to an LLM chain to get a new answer.\n",
|
|
"\n",
|
|
"This can be easily run with the `chain_type=\"refine\"` specified."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 21,
|
|
"id": "de1dc10e",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"The existing summary provides detailed instructions for implementing a project's architecture through code, focusing on creating core classes, functions, and methods in different files following best practices for the chosen language and framework. Assumptions about the model, view, and controller components are also outlined. The additional context highlights challenges in long-term planning and task decomposition, as well as the reliability issues with natural language interfaces in LLM-powered autonomous agents. These insights shed light on the limitations and potential pitfalls of using LLMs in agent systems, with references to recent research on LLM-powered autonomous agents and related technologies.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"chain = load_summarize_chain(llm, chain_type=\"refine\")\n",
|
|
"result = chain.invoke(split_docs)\n",
|
|
"\n",
|
|
"print(result[\"output_text\"])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b5dc3052-5873-4ef2-b633-3709ede4131a",
|
|
"metadata": {},
|
|
"source": [
|
|
"Following the [Langsmith trace](https://smith.langchain.com/public/38017fa7-b190-4635-992c-e8554227a4bb/r), we can see the summaries iteratively updated with new information."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5b46f44d",
|
|
"metadata": {},
|
|
"source": [
|
|
"It's also possible to supply a prompt and return intermediate steps."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"id": "f86c8072",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"prompt_template = \"\"\"Write a concise summary of the following:\n",
|
|
"{text}\n",
|
|
"CONCISE SUMMARY:\"\"\"\n",
|
|
"prompt = PromptTemplate.from_template(prompt_template)\n",
|
|
"\n",
|
|
"refine_template = (\n",
|
|
" \"Your job is to produce a final summary\\n\"\n",
|
|
" \"We have provided an existing summary up to a certain point: {existing_answer}\\n\"\n",
|
|
" \"We have the opportunity to refine the existing summary\"\n",
|
|
" \"(only if needed) with some more context below.\\n\"\n",
|
|
" \"------------\\n\"\n",
|
|
" \"{text}\\n\"\n",
|
|
" \"------------\\n\"\n",
|
|
" \"Given the new context, refine the original summary in Italian\"\n",
|
|
" \"If the context isn't useful, return the original summary.\"\n",
|
|
")\n",
|
|
"refine_prompt = PromptTemplate.from_template(refine_template)\n",
|
|
"chain = load_summarize_chain(\n",
|
|
" llm=llm,\n",
|
|
" chain_type=\"refine\",\n",
|
|
" question_prompt=prompt,\n",
|
|
" refine_prompt=refine_prompt,\n",
|
|
" return_intermediate_steps=True,\n",
|
|
" input_key=\"input_documents\",\n",
|
|
" output_key=\"output_text\",\n",
|
|
")\n",
|
|
"result = chain.invoke({\"input_documents\": split_docs}, return_only_outputs=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"id": "d9600b67-79d4-4f85-aba2-9fe81fa29f49",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Il presente articolo discute il concetto di costruire agenti autonomi utilizzando LLM (large language model) come controller principale. Esplora i diversi componenti di un sistema di agenti alimentato da LLM, tra cui la pianificazione, la memoria e l'uso degli strumenti. Dimostrazioni di concetto come AutoGPT mostrano il potenziale di LLM come risolutore generale di problemi. Approcci come Chain of Thought, Tree of Thoughts, LLM+P, ReAct e Reflexion consentono agli agenti autonomi di pianificare, riflettere su se stessi e migliorarsi iterativamente. Tuttavia, ci sono sfide da affrontare, come la limitata capacità di contesto che limita l'inclusione di informazioni storiche dettagliate e la difficoltà di pianificazione a lungo termine e decomposizione delle attività. Inoltre, l'affidabilità dell'interfaccia di linguaggio naturale tra LLM e componenti esterni come la memoria e gli strumenti è incerta, poiché i LLM possono commettere errori di formattazione e mostrare comportamenti ribelli. Nonostante ciò, il sistema AutoGPT viene menzionato come esempio di dimostrazione di concetto che utilizza LLM come controller principale per agenti autonomi. Questo articolo fa riferimento a diverse fonti che esplorano approcci e applicazioni specifiche di LLM nell'ambito degli agenti autonomi.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(result[\"output_text\"])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"id": "5f91a8eb-daa5-4191-ace4-01765801db3e",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"This article discusses the concept of building autonomous agents using LLM (large language model) as the core controller. The article explores the different components of an LLM-powered agent system, including planning, memory, and tool use. It also provides examples of proof-of-concept demos and highlights the potential of LLM as a general problem solver.\n",
|
|
"\n",
|
|
"Questo articolo discute del concetto di costruire agenti autonomi utilizzando LLM (large language model) come controller principale. L'articolo esplora i diversi componenti di un sistema di agenti alimentato da LLM, inclusa la pianificazione, la memoria e l'uso degli strumenti. Vengono forniti anche esempi di dimostrazioni di proof-of-concept e si evidenzia il potenziale di LLM come risolutore generale di problemi. Inoltre, vengono presentati approcci come Chain of Thought, Tree of Thoughts, LLM+P, ReAct e Reflexion che consentono agli agenti autonomi di pianificare, riflettere su se stessi e migliorare iterativamente.\n",
|
|
"\n",
|
|
"Questo articolo discute del concetto di costruire agenti autonomi utilizzando LLM (large language model) come controller principale. L'articolo esplora i diversi componenti di un sistema di agenti alimentato da LLM, inclusa la pianificazione, la memoria e l'uso degli strumenti. Vengono forniti anche esempi di dimostrazioni di proof-of-concept e si evidenzia il potenziale di LLM come risolutore generale di problemi. Inoltre, vengono presentati approcci come Chain of Thought, Tree of Thoughts, LLM+P, ReAct e Reflexion che consentono agli agenti autonomi di pianificare, riflettere su se stessi e migliorare iterativamente. Il nuovo contesto riguarda l'approccio Chain of Hindsight (CoH) che permette al modello di migliorare autonomamente i propri output attraverso un processo di apprendimento supervisionato. Viene anche presentato l'approccio Algorithm Distillation (AD) che applica lo stesso concetto alle traiettorie di apprendimento per compiti di reinforcement learning.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(\"\\n\\n\".join(result[\"intermediate_steps\"][:3]))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "0d8a8398-a43c-4f14-933c-c0743ae6ec40",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Splitting and summarizing in a single chain\n",
|
|
"For convenience, we can wrap both the text splitting of our long document and summarizing in a single `AnalyzeDocumentsChain`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "0ddd522e-30dc-4f6a-b993-c4f97e656c4f",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain.chains import AnalyzeDocumentChain\n",
|
|
"\n",
|
|
"summarize_document_chain = AnalyzeDocumentChain(\n",
|
|
" combine_docs_chain=chain, text_splitter=text_splitter\n",
|
|
")\n",
|
|
"summarize_document_chain.invoke(docs[0].page_content)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a41e4a81-3e26-4753-95bd-f80633620121",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Next steps\n",
|
|
"\n",
|
|
"We encourage you to check out the [how-to guides](/docs/how_to) for more detail on: \n",
|
|
"\n",
|
|
"- Built-in [document loaders](/docs/how_to/#document-loaders) and [text-splitters](/docs/how_to/#text-splitters)\n",
|
|
"- Integrating various combine-document chains into a [RAG application](/docs/tutorials/rag/)\n",
|
|
"- Incorporating retrieval into a [chatbot](/docs/how_to/chatbots_retrieval/)\n",
|
|
"\n",
|
|
"and other concepts."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "db01bcf3-0186-4689-8f79-1a577e551cb1",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.4"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|