mirror of
https://github.com/hwchase17/langchain.git
synced 2025-05-13 02:57:22 +00:00
350 lines
13 KiB
Plaintext
350 lines
13 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "raw",
|
|
"id": "afaf8039",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"sidebar_label: Azure OpenAI\n",
|
|
"---"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e49f1e0d",
|
|
"metadata": {},
|
|
"source": [
|
|
"# AzureChatOpenAI\n",
|
|
"\n",
|
|
"This guide will help you get started with AzureOpenAI [chat models](/docs/concepts/chat_models). For detailed documentation of all AzureChatOpenAI features and configurations head to the [API reference](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html).\n",
|
|
"\n",
|
|
"Azure OpenAI has several chat models. You can find information about their latest models and their costs, context windows, and supported input types in the [Azure docs](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models).\n",
|
|
"\n",
|
|
":::info Azure OpenAI vs OpenAI\n",
|
|
"\n",
|
|
"Azure OpenAI refers to OpenAI models hosted on the [Microsoft Azure platform](https://azure.microsoft.com/en-us/products/ai-services/openai-service). OpenAI also provides its own model APIs. To access OpenAI services directly, use the [ChatOpenAI integration](/docs/integrations/chat/openai/).\n",
|
|
"\n",
|
|
":::\n",
|
|
"\n",
|
|
"## Overview\n",
|
|
"### Integration details\n",
|
|
"\n",
|
|
"| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/azure) | Package downloads | Package latest |\n",
|
|
"| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n",
|
|
"| [AzureChatOpenAI](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html) | [langchain-openai](https://python.langchain.com/api_reference/openai/index.html) | ❌ | beta | ✅ |  |  |\n",
|
|
"\n",
|
|
"### Model features\n",
|
|
"| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",
|
|
"| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n",
|
|
"| ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | \n",
|
|
"\n",
|
|
"## Setup\n",
|
|
"\n",
|
|
"To access AzureOpenAI models you'll need to create an Azure account, create a deployment of an Azure OpenAI model, get the name and endpoint for your deployment, get an Azure OpenAI API key, and install the `langchain-openai` integration package.\n",
|
|
"\n",
|
|
"### Credentials\n",
|
|
"\n",
|
|
"Head to the [Azure docs](https://learn.microsoft.com/en-us/azure/ai-services/openai/chatgpt-quickstart?tabs=command-line%2Cpython-new&pivots=programming-language-python) to create your deployment and generate an API key. Once you've done this set the AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT environment variables:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import getpass\n",
|
|
"import os\n",
|
|
"\n",
|
|
"if \"AZURE_OPENAI_API_KEY\" not in os.environ:\n",
|
|
" os.environ[\"AZURE_OPENAI_API_KEY\"] = getpass.getpass(\n",
|
|
" \"Enter your AzureOpenAI API key: \"\n",
|
|
" )\n",
|
|
"os.environ[\"AZURE_OPENAI_ENDPOINT\"] = \"https://YOUR-ENDPOINT.openai.azure.com/\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "72ee0c4b-9764-423a-9dbf-95129e185210",
|
|
"metadata": {},
|
|
"source": [
|
|
"If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "a15d341e-3e26-4ca3-830b-5aab30ed66de",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n",
|
|
"# os.environ[\"LANGSMITH_TRACING\"] = \"true\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "0730d6a1-c893-4840-9817-5e5251676d5d",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Installation\n",
|
|
"\n",
|
|
"The LangChain AzureOpenAI integration lives in the `langchain-openai` package:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "652d6238-1f87-422a-b135-f5abbb8652fc",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install -qU langchain-openai"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a38cde65-254d-4219-a441-068766c0d4b5",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Instantiation\n",
|
|
"\n",
|
|
"Now we can instantiate our model object and generate chat completions.\n",
|
|
"- Replace `azure_deployment` with the name of your deployment,\n",
|
|
"- You can find the latest supported `api_version` here: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain_openai import AzureChatOpenAI\n",
|
|
"\n",
|
|
"llm = AzureChatOpenAI(\n",
|
|
" azure_deployment=\"gpt-35-turbo\", # or your deployment\n",
|
|
" api_version=\"2023-06-01-preview\", # or your api version\n",
|
|
" temperature=0,\n",
|
|
" max_tokens=None,\n",
|
|
" timeout=None,\n",
|
|
" max_retries=2,\n",
|
|
" # other params...\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2b4f3e15",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Invocation"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"id": "62e0dbc3",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"AIMessage(content=\"J'adore la programmation.\", response_metadata={'token_usage': {'completion_tokens': 8, 'prompt_tokens': 31, 'total_tokens': 39}, 'model_name': 'gpt-35-turbo', 'system_fingerprint': None, 'prompt_filter_results': [{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}], 'finish_reason': 'stop', 'logprobs': None, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}, id='run-bea4b46c-e3e1-4495-9d3a-698370ad963d-0', usage_metadata={'input_tokens': 31, 'output_tokens': 8, 'total_tokens': 39})"
|
|
]
|
|
},
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"messages = [\n",
|
|
" (\n",
|
|
" \"system\",\n",
|
|
" \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n",
|
|
" ),\n",
|
|
" (\"human\", \"I love programming.\"),\n",
|
|
"]\n",
|
|
"ai_msg = llm.invoke(messages)\n",
|
|
"ai_msg"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"id": "d86145b3-bfef-46e8-b227-4dda5c9c2705",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"J'adore la programmation.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(ai_msg.content)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "18e2bfc0-7e78-4528-a73f-499ac150dca8",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Chaining\n",
|
|
"\n",
|
|
"We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'completion_tokens': 6, 'prompt_tokens': 26, 'total_tokens': 32}, 'model_name': 'gpt-35-turbo', 'system_fingerprint': None, 'prompt_filter_results': [{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}], 'finish_reason': 'stop', 'logprobs': None, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}, id='run-cbc44038-09d3-40d4-9da2-c5910ee636ca-0', usage_metadata={'input_tokens': 26, 'output_tokens': 6, 'total_tokens': 32})"
|
|
]
|
|
},
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain_core.prompts import ChatPromptTemplate\n",
|
|
"\n",
|
|
"prompt = ChatPromptTemplate.from_messages(\n",
|
|
" [\n",
|
|
" (\n",
|
|
" \"system\",\n",
|
|
" \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n",
|
|
" ),\n",
|
|
" (\"human\", \"{input}\"),\n",
|
|
" ]\n",
|
|
")\n",
|
|
"\n",
|
|
"chain = prompt | llm\n",
|
|
"chain.invoke(\n",
|
|
" {\n",
|
|
" \"input_language\": \"English\",\n",
|
|
" \"output_language\": \"German\",\n",
|
|
" \"input\": \"I love programming.\",\n",
|
|
" }\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Specifying model version\n",
|
|
"\n",
|
|
"Azure OpenAI responses contain `model_name` response metadata property, which is name of the model used to generate the response. However unlike native OpenAI responses, it does not contain the specific version of the model, which is set on the deployment in Azure. E.g. it does not distinguish between `gpt-35-turbo-0125` and `gpt-35-turbo-0301`. This makes it tricky to know which version of the model was used to generate the response, which as result can lead to e.g. wrong total cost calculation with `OpenAICallbackHandler`.\n",
|
|
"\n",
|
|
"To solve this problem, you can pass `model_version` parameter to `AzureChatOpenAI` class, which will be added to the model name in the llm output. This way you can easily distinguish between different versions of the model."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "04b36e75-e8b7-4721-899e-76301ac2ecd9",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install -qU langchain-community"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"id": "2ca02d23-60d0-43eb-8d04-070f61f8fefd",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Total Cost (USD): $0.000063\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain_community.callbacks import get_openai_callback\n",
|
|
"\n",
|
|
"with get_openai_callback() as cb:\n",
|
|
" llm.invoke(messages)\n",
|
|
" print(\n",
|
|
" f\"Total Cost (USD): ${format(cb.total_cost, '.6f')}\"\n",
|
|
" ) # without specifying the model version, flat-rate 0.002 USD per 1k input and output tokens is used"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"id": "e1b07ae2-3de7-44bd-bfdc-b76f4ba45a35",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Total Cost (USD): $0.000074\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"llm_0301 = AzureChatOpenAI(\n",
|
|
" azure_deployment=\"gpt-35-turbo\", # or your deployment\n",
|
|
" api_version=\"2023-06-01-preview\", # or your api version\n",
|
|
" model_version=\"0301\",\n",
|
|
")\n",
|
|
"with get_openai_callback() as cb:\n",
|
|
" llm_0301.invoke(messages)\n",
|
|
" print(f\"Total Cost (USD): ${format(cb.total_cost, '.6f')}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3",
|
|
"metadata": {},
|
|
"source": [
|
|
"## API reference\n",
|
|
"\n",
|
|
"For detailed documentation of all AzureChatOpenAI features and configurations head to the API reference: https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.4"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|