mirror of
https://github.com/hwchase17/langchain.git
synced 2025-09-01 19:12:42 +00:00
openai[patch]: release 0.2.6 (#27924)
Some additions in support of [predicted outputs](https://platform.openai.com/docs/guides/latency-optimization#use-predicted-outputs) feature: - Bump openai sdk version - Add integration test - Add example to integration docs The `prediction` kwarg is already plumbed through model invocation.
This commit is contained in:
@@ -509,6 +509,101 @@
|
||||
"output_message.content"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5c35d0a4-a6b8-4d35-a02b-a37a8bda5692",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Predicted output\n",
|
||||
"\n",
|
||||
":::info\n",
|
||||
"Requires `langchain-openai>=0.2.6`\n",
|
||||
":::\n",
|
||||
"\n",
|
||||
"Some OpenAI models (such as their `gpt-4o` and `gpt-4o-mini` series) support [Predicted Outputs](https://platform.openai.com/docs/guides/latency-optimization#use-predicted-outputs), which allow you to pass in a known portion of the LLM's expected output ahead of time to reduce latency. This is useful for cases such as editing text or code, where only a small part of the model's output will change.\n",
|
||||
"\n",
|
||||
"Here's an example:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"id": "88fee1e9-58c1-42ad-ae23-24b882e175e7",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/// <summary>\n",
|
||||
"/// Represents a user with a first name, last name, and email.\n",
|
||||
"/// </summary>\n",
|
||||
"public class User\n",
|
||||
"{\n",
|
||||
" /// <summary>\n",
|
||||
" /// Gets or sets the user's first name.\n",
|
||||
" /// </summary>\n",
|
||||
" public string FirstName { get; set; }\n",
|
||||
"\n",
|
||||
" /// <summary>\n",
|
||||
" /// Gets or sets the user's last name.\n",
|
||||
" /// </summary>\n",
|
||||
" public string LastName { get; set; }\n",
|
||||
"\n",
|
||||
" /// <summary>\n",
|
||||
" /// Gets or sets the user's email.\n",
|
||||
" /// </summary>\n",
|
||||
" public string Email { get; set; }\n",
|
||||
"}\n",
|
||||
"{'token_usage': {'completion_tokens': 226, 'prompt_tokens': 166, 'total_tokens': 392, 'completion_tokens_details': {'accepted_prediction_tokens': 49, 'audio_tokens': None, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 107}, 'prompt_tokens_details': {'audio_tokens': None, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-2024-08-06', 'system_fingerprint': 'fp_45cf54deae', 'finish_reason': 'stop', 'logprobs': None}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"code = \"\"\"\n",
|
||||
"/// <summary>\n",
|
||||
"/// Represents a user with a first name, last name, and username.\n",
|
||||
"/// </summary>\n",
|
||||
"public class User\n",
|
||||
"{\n",
|
||||
" /// <summary>\n",
|
||||
" /// Gets or sets the user's first name.\n",
|
||||
" /// </summary>\n",
|
||||
" public string FirstName { get; set; }\n",
|
||||
"\n",
|
||||
" /// <summary>\n",
|
||||
" /// Gets or sets the user's last name.\n",
|
||||
" /// </summary>\n",
|
||||
" public string LastName { get; set; }\n",
|
||||
"\n",
|
||||
" /// <summary>\n",
|
||||
" /// Gets or sets the user's username.\n",
|
||||
" /// </summary>\n",
|
||||
" public string Username { get; set; }\n",
|
||||
"}\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"llm = ChatOpenAI(model=\"gpt-4o\")\n",
|
||||
"query = (\n",
|
||||
" \"Replace the Username property with an Email property. \"\n",
|
||||
" \"Respond only with code, and with no markdown formatting.\"\n",
|
||||
")\n",
|
||||
"response = llm.invoke(\n",
|
||||
" [{\"role\": \"user\", \"content\": query}, {\"role\": \"user\", \"content\": code}],\n",
|
||||
" prediction={\"type\": \"content\", \"content\": code},\n",
|
||||
")\n",
|
||||
"print(response.content)\n",
|
||||
"print(response.response_metadata)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "2ee1b26d-a388-4e7c-9f40-bfd1388ecc03",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Note that currently predictions are billed as additional tokens and may increase your usage and costs in exchange for this reduced latency."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "feb4a499",
|
||||
@@ -601,7 +696,7 @@
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": ".venv",
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
@@ -615,7 +710,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.4"
|
||||
"version": "3.10.4"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
Reference in New Issue
Block a user