mirror of
https://github.com/hwchase17/langchain.git
synced 2025-06-08 07:57:07 +00:00
580fbd9ada
``` https://api\.python\.langchain\.com/en/latest/([^/]*)/langchain_([^.]*)\.(.*)\.html([^"]*) https://python.langchain.com/v0.2/api_reference/$2/$1/langchain_$2.$3.html$4 ``` --------- Co-authored-by: Bagatur <baskaryan@gmail.com>
478 lines
17 KiB
Plaintext
478 lines
17 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "raw",
|
|
"id": "beba2e0e",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"sidebar_position: 2\n",
|
|
"---"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "bb0735c0",
|
|
"metadata": {},
|
|
"source": [
|
|
"# How to use few shot examples in chat models\n",
|
|
"\n",
|
|
":::info Prerequisites\n",
|
|
"\n",
|
|
"This guide assumes familiarity with the following concepts:\n",
|
|
"- [Prompt templates](/docs/concepts/#prompt-templates)\n",
|
|
"- [Example selectors](/docs/concepts/#example-selectors)\n",
|
|
"- [Chat models](/docs/concepts/#chat-model)\n",
|
|
"- [Vectorstores](/docs/concepts/#vector-stores)\n",
|
|
"\n",
|
|
":::\n",
|
|
"\n",
|
|
"This guide covers how to prompt a chat model with example inputs and outputs. Providing the model with a few such examples is called few-shotting, and is a simple yet powerful way to guide generation and in some cases drastically improve model performance.\n",
|
|
"\n",
|
|
"There does not appear to be solid consensus on how best to do few-shot prompting, and the optimal prompt compilation will likely vary by model. Because of this, we provide few-shot prompt templates like the [FewShotChatMessagePromptTemplate](https://python.langchain.com/v0.2/api_reference/core/prompts/langchain_core.prompts.few_shot.FewShotChatMessagePromptTemplate.html?highlight=fewshot#langchain_core.prompts.few_shot.FewShotChatMessagePromptTemplate) as a flexible starting point, and you can modify or replace them as you see fit.\n",
|
|
"\n",
|
|
"The goal of few-shot prompt templates are to dynamically select examples based on an input, and then format the examples in a final prompt to provide for the model.\n",
|
|
"\n",
|
|
"**Note:** The following code examples are for chat models only, since `FewShotChatMessagePromptTemplates` are designed to output formatted [chat messages](/docs/concepts/#message-types) rather than pure strings. For similar few-shot prompt examples for pure string templates compatible with completion models (LLMs), see the [few-shot prompt templates](/docs/how_to/few_shot_examples/) guide."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d716f2de-cc29-4823-9360-a808c7bfdb86",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Fixed Examples\n",
|
|
"\n",
|
|
"The most basic (and common) few-shot prompting technique is to use fixed prompt examples. This way you can select a chain, evaluate it, and avoid worrying about additional moving parts in production.\n",
|
|
"\n",
|
|
"The basic components of the template are:\n",
|
|
"- `examples`: A list of dictionary examples to include in the final prompt.\n",
|
|
"- `example_prompt`: converts each example into 1 or more messages through its [`format_messages`](https://python.langchain.com/v0.2/api_reference/core/prompts/langchain_core.prompts.chat.ChatPromptTemplate.html?highlight=format_messages#langchain_core.prompts.chat.ChatPromptTemplate.format_messages) method. A common example would be to convert each example into one human message and one AI message response, or a human message followed by a function call message.\n",
|
|
"\n",
|
|
"Below is a simple demonstration. First, define the examples you'd like to include. Let's give the LLM an unfamiliar mathematical operator, denoted by the \"🦜\" emoji:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "5b79e400",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install -qU langchain langchain-openai langchain-chroma\n",
|
|
"\n",
|
|
"import os\n",
|
|
"from getpass import getpass\n",
|
|
"\n",
|
|
"os.environ[\"OPENAI_API_KEY\"] = getpass()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "30856d92",
|
|
"metadata": {},
|
|
"source": [
|
|
"If we try to ask the model what the result of this expression is, it will fail:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"id": "174dec5b",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"AIMessage(content='The expression \"2 🦜 9\" is not a standard mathematical operation or equation. It appears to be a combination of the number 2 and the parrot emoji 🦜 followed by the number 9. It does not have a specific mathematical meaning.', response_metadata={'token_usage': {'completion_tokens': 54, 'prompt_tokens': 17, 'total_tokens': 71}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-aad12dda-5c47-4a1e-9949-6fe94e03242a-0', usage_metadata={'input_tokens': 17, 'output_tokens': 54, 'total_tokens': 71})"
|
|
]
|
|
},
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain_openai import ChatOpenAI\n",
|
|
"\n",
|
|
"model = ChatOpenAI(model=\"gpt-3.5-turbo-0125\", temperature=0.0)\n",
|
|
"\n",
|
|
"model.invoke(\"What is 2 🦜 9?\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e6d58385",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's see what happens if we give the LLM some examples to work with. We'll define some below:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"id": "0fc5a02a-6249-4e92-95c3-30fff9671e8b",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain_core.prompts import ChatPromptTemplate, FewShotChatMessagePromptTemplate\n",
|
|
"\n",
|
|
"examples = [\n",
|
|
" {\"input\": \"2 🦜 2\", \"output\": \"4\"},\n",
|
|
" {\"input\": \"2 🦜 3\", \"output\": \"5\"},\n",
|
|
"]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e8710ecc-2aa0-4172-a74c-250f6bc3d9e2",
|
|
"metadata": {},
|
|
"source": [
|
|
"Next, assemble them into the few-shot prompt template."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"id": "65e72ad1-9060-47d0-91a1-bc130c8b98ac",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[HumanMessage(content='2 🦜 2'), AIMessage(content='4'), HumanMessage(content='2 🦜 3'), AIMessage(content='5')]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# This is a prompt template used to format each individual example.\n",
|
|
"example_prompt = ChatPromptTemplate.from_messages(\n",
|
|
" [\n",
|
|
" (\"human\", \"{input}\"),\n",
|
|
" (\"ai\", \"{output}\"),\n",
|
|
" ]\n",
|
|
")\n",
|
|
"few_shot_prompt = FewShotChatMessagePromptTemplate(\n",
|
|
" example_prompt=example_prompt,\n",
|
|
" examples=examples,\n",
|
|
")\n",
|
|
"\n",
|
|
"print(few_shot_prompt.invoke({}).to_messages())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5490bd59-b28f-46a4-bbdf-0191802dd3c5",
|
|
"metadata": {},
|
|
"source": [
|
|
"Finally, we assemble the final prompt as shown below, passing `few_shot_prompt` directly into the `from_messages` factory method, and use it with a model:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"id": "9f86d6d9-50de-41b6-b6c7-0f9980cc0187",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"final_prompt = ChatPromptTemplate.from_messages(\n",
|
|
" [\n",
|
|
" (\"system\", \"You are a wondrous wizard of math.\"),\n",
|
|
" few_shot_prompt,\n",
|
|
" (\"human\", \"{input}\"),\n",
|
|
" ]\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "dd8029c5",
|
|
"metadata": {},
|
|
"source": [
|
|
"And now let's ask the model the initial question and see how it does:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"id": "97d443b1-6fae-4b36-bede-3ff7306288a3",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"AIMessage(content='11', response_metadata={'token_usage': {'completion_tokens': 1, 'prompt_tokens': 60, 'total_tokens': 61}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-5ec4e051-262f-408e-ad00-3f2ebeb561c3-0', usage_metadata={'input_tokens': 60, 'output_tokens': 1, 'total_tokens': 61})"
|
|
]
|
|
},
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain_openai import ChatOpenAI\n",
|
|
"\n",
|
|
"chain = final_prompt | model\n",
|
|
"\n",
|
|
"chain.invoke({\"input\": \"What is 2 🦜 9?\"})"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "70ab7114-f07f-46be-8874-3705a25aba5f",
|
|
"metadata": {},
|
|
"source": [
|
|
"And we can see that the model has now inferred that the parrot emoji means addition from the given few-shot examples!\n",
|
|
"\n",
|
|
"## Dynamic few-shot prompting\n",
|
|
"\n",
|
|
"Sometimes you may want to select only a few examples from your overall set to show based on the input. For this, you can replace the `examples` passed into `FewShotChatMessagePromptTemplate` with an `example_selector`. The other components remain the same as above! Our dynamic few-shot prompt template would look like:\n",
|
|
"\n",
|
|
"- `example_selector`: responsible for selecting few-shot examples (and the order in which they are returned) for a given input. These implement the [BaseExampleSelector](https://python.langchain.com/v0.2/api_reference/core/example_selectors/langchain_core.example_selectors.base.BaseExampleSelector.html?highlight=baseexampleselector#langchain_core.example_selectors.base.BaseExampleSelector) interface. A common example is the vectorstore-backed [SemanticSimilarityExampleSelector](https://python.langchain.com/v0.2/api_reference/core/example_selectors/langchain_core.example_selectors.semantic_similarity.SemanticSimilarityExampleSelector.html?highlight=semanticsimilarityexampleselector#langchain_core.example_selectors.semantic_similarity.SemanticSimilarityExampleSelector)\n",
|
|
"- `example_prompt`: convert each example into 1 or more messages through its [`format_messages`](https://python.langchain.com/v0.2/api_reference/core/prompts/langchain_core.prompts.chat.ChatPromptTemplate.html?highlight=chatprompttemplate#langchain_core.prompts.chat.ChatPromptTemplate.format_messages) method. A common example would be to convert each example into one human message and one AI message response, or a human message followed by a function call message.\n",
|
|
"\n",
|
|
"These once again can be composed with other messages and chat templates to assemble your final prompt.\n",
|
|
"\n",
|
|
"Let's walk through an example with the `SemanticSimilarityExampleSelector`. Since this implementation uses a vectorstore to select examples based on semantic similarity, we will want to first populate the store. Since the basic idea here is that we want to search for and return examples most similar to the text input, we embed the `values` of our prompt examples rather than considering the keys:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"id": "ad66f06a-66fd-4fcc-8166-5d0e3c801e57",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain_chroma import Chroma\n",
|
|
"from langchain_core.example_selectors import SemanticSimilarityExampleSelector\n",
|
|
"from langchain_openai import OpenAIEmbeddings\n",
|
|
"\n",
|
|
"examples = [\n",
|
|
" {\"input\": \"2 🦜 2\", \"output\": \"4\"},\n",
|
|
" {\"input\": \"2 🦜 3\", \"output\": \"5\"},\n",
|
|
" {\"input\": \"2 🦜 4\", \"output\": \"6\"},\n",
|
|
" {\"input\": \"What did the cow say to the moon?\", \"output\": \"nothing at all\"},\n",
|
|
" {\n",
|
|
" \"input\": \"Write me a poem about the moon\",\n",
|
|
" \"output\": \"One for the moon, and one for me, who are we to talk about the moon?\",\n",
|
|
" },\n",
|
|
"]\n",
|
|
"\n",
|
|
"to_vectorize = [\" \".join(example.values()) for example in examples]\n",
|
|
"embeddings = OpenAIEmbeddings()\n",
|
|
"vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=examples)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2f7e384a-2031-432b-951c-7ea8cf9262f1",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Create the `example_selector`\n",
|
|
"\n",
|
|
"With a vectorstore created, we can create the `example_selector`. Here we will call it in isolation, and set `k` on it to only fetch the two example closest to the input."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"id": "7790303a-f722-452e-8921-b14bdf20bdff",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[{'input': 'What did the cow say to the moon?', 'output': 'nothing at all'},\n",
|
|
" {'input': '2 🦜 4', 'output': '6'}]"
|
|
]
|
|
},
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"example_selector = SemanticSimilarityExampleSelector(\n",
|
|
" vectorstore=vectorstore,\n",
|
|
" k=2,\n",
|
|
")\n",
|
|
"\n",
|
|
"# The prompt template will load examples by passing the input do the `select_examples` method\n",
|
|
"example_selector.select_examples({\"input\": \"horse\"})"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cc77c40f-3f58-40a2-b757-a2a2ea43f24a",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Create prompt template\n",
|
|
"\n",
|
|
"We now assemble the prompt template, using the `example_selector` created above."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"id": "253c255e-41d7-45f6-9d88-c7a0ced4b1bd",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[HumanMessage(content='2 🦜 3'), AIMessage(content='5'), HumanMessage(content='2 🦜 4'), AIMessage(content='6')]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain_core.prompts import ChatPromptTemplate, FewShotChatMessagePromptTemplate\n",
|
|
"\n",
|
|
"# Define the few-shot prompt.\n",
|
|
"few_shot_prompt = FewShotChatMessagePromptTemplate(\n",
|
|
" # The input variables select the values to pass to the example_selector\n",
|
|
" input_variables=[\"input\"],\n",
|
|
" example_selector=example_selector,\n",
|
|
" # Define how each example will be formatted.\n",
|
|
" # In this case, each example will become 2 messages:\n",
|
|
" # 1 human, and 1 AI\n",
|
|
" example_prompt=ChatPromptTemplate.from_messages(\n",
|
|
" [(\"human\", \"{input}\"), (\"ai\", \"{output}\")]\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"print(few_shot_prompt.invoke(input=\"What's 3 🦜 3?\").to_messages())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "339cae7d-0eb0-44a6-852f-0267c5ff72b3",
|
|
"metadata": {},
|
|
"source": [
|
|
"And we can pass this few-shot chat message prompt template into another chat prompt template:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"id": "e731cb45-f0ea-422c-be37-42af2a6cb2c4",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"messages=[HumanMessage(content='2 🦜 3'), AIMessage(content='5'), HumanMessage(content='2 🦜 4'), AIMessage(content='6')]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"final_prompt = ChatPromptTemplate.from_messages(\n",
|
|
" [\n",
|
|
" (\"system\", \"You are a wondrous wizard of math.\"),\n",
|
|
" few_shot_prompt,\n",
|
|
" (\"human\", \"{input}\"),\n",
|
|
" ]\n",
|
|
")\n",
|
|
"\n",
|
|
"print(few_shot_prompt.invoke(input=\"What's 3 🦜 3?\"))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2408ea69-1880-4ef5-a0fa-ffa8d2026aa9",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Use with an chat model\n",
|
|
"\n",
|
|
"Finally, you can connect your model to the few-shot prompt."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"id": "0568cbc6-5354-47f1-ab4d-dfcc616cf583",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"AIMessage(content='6', response_metadata={'token_usage': {'completion_tokens': 1, 'prompt_tokens': 60, 'total_tokens': 61}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-d1863e5e-17cd-4e9d-bf7a-b9f118747a65-0', usage_metadata={'input_tokens': 60, 'output_tokens': 1, 'total_tokens': 61})"
|
|
]
|
|
},
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"chain = final_prompt | ChatOpenAI(model=\"gpt-3.5-turbo-0125\", temperature=0.0)\n",
|
|
"\n",
|
|
"chain.invoke({\"input\": \"What's 3 🦜 3?\"})"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c87fad3c",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Next steps\n",
|
|
"\n",
|
|
"You've now learned how to add few-shot examples to your chat prompts.\n",
|
|
"\n",
|
|
"Next, check out the other how-to guides on prompt templates in this section, the related how-to guide on [few shotting with text completion models](/docs/how_to/few_shot_examples), or the other [example selector how-to guides](/docs/how_to/example_selectors/)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "46e26b53",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.5"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|