mirror of
https://github.com/hwchase17/langchain.git
synced 2026-04-04 03:14:33 +00:00
99 lines
3.1 KiB
Plaintext
99 lines
3.1 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "raw",
|
|
"id": "e9437c8a-d8b7-4bf6-8ff4-54068a5a266c",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"sidebar_position: 1.5\n",
|
|
"---"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d0df7646-b1e1-4014-a841-6dae9b3c50d9",
|
|
"metadata": {},
|
|
"source": [
|
|
"# How to stream chat model responses\n",
|
|
"\n",
|
|
"All ChatModels implement the Runnable interface, which comes with default implementations of all methods, ie. ainvoke, batch, abatch, stream, astream. This gives all ChatModels basic support for streaming.\n",
|
|
"\n",
|
|
"Streaming support defaults to returning an Iterator (or AsyncIterator in the case of async streaming) of a single value, the final result returned by the underlying ChatModel provider. This obviously doesn't give you token-by-token streaming, which requires native support from the ChatModel provider, but ensures your code that expects an iterator of tokens can work for any of our ChatModel integrations.\n",
|
|
"\n",
|
|
"See which [integrations support token-by-token streaming here](/docs/integrations/chat/)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "03080a2c-45e8-45b9-a367-62816eae54c4",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain_community.chat_models import ChatAnthropic"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"id": "975c4f32-21f6-4a71-9091-f87b56347c33",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
" Here's a song I just improvised about goldfish on the moon:\n",
|
|
"\n",
|
|
"Floating in space, looking for a place \n",
|
|
"To call their home, all alone\n",
|
|
"Swimming through stars, these goldfish from Mars\n",
|
|
"Left their fishbowl behind, a new life to find\n",
|
|
"On the moon, where the craters loom\n",
|
|
"Searching for food, maybe some lunar food\n",
|
|
"Out of their depth, close to death\n",
|
|
"How they wish, for just one small fish\n",
|
|
"To join them up here, their future unclear\n",
|
|
"On the moon, where the Earth looms\n",
|
|
"Dreaming of home, filled with foam\n",
|
|
"Their bodies adapt, continuing to last \n",
|
|
"On the moon, where they learn to swoon\n",
|
|
"Over cheese that astronauts tease\n",
|
|
"As they stare back at Earth, the planet of birth\n",
|
|
"These goldfish out of water, swim on and on\n",
|
|
"Lunar pioneers, conquering their fears\n",
|
|
"On the moon, where they happily swoon"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"chat = ChatAnthropic(model=\"claude-2\")\n",
|
|
"for chunk in chat.stream(\"Write me a song about goldfish on the moon\"):\n",
|
|
" print(chunk.content, end=\"\", flush=True)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.1"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|