mirror of
https://github.com/hwchase17/langchain.git
synced 2025-06-05 06:33:20 +00:00
113 lines
3.5 KiB
Plaintext
113 lines
3.5 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fc37c39a-7406-4c13-a754-b8e95fd970a0",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Streaming\n",
|
|
"\n",
|
|
"All `LLM`s implement the `Runnable` interface, which comes with default implementations of all methods, ie. ainvoke, batch, abatch, stream, astream. This gives all `LLM`s basic support for streaming.\n",
|
|
"\n",
|
|
"Streaming support defaults to returning an Iterator (or AsyncIterator in the case of async streaming) of a single value, the final result returned by the underlying `LLM` provider. This obviously doesn't give you token-by-token streaming, which requires native support from the `LLM` provider, but ensures your code that expects an iterator of tokens can work for any of our `LLM` integrations.\n",
|
|
"\n",
|
|
"See which [integrations support token-by-token streaming here](/docs/integrations/llms/)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"id": "9baa0527-b97d-41d3-babd-472ec5e59e3e",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\n",
|
|
"\n",
|
|
"Verse 1:\n",
|
|
"Bubbles dancing in my glass\n",
|
|
"Clear and crisp, it's such a blast\n",
|
|
"Refreshing taste, it's like a dream\n",
|
|
"Sparkling water, you make me beam\n",
|
|
"\n",
|
|
"Chorus:\n",
|
|
"Oh sparkling water, you're my delight\n",
|
|
"With every sip, you make me feel so right\n",
|
|
"You're like a party in my mouth\n",
|
|
"I can't get enough, I'm hooked no doubt\n",
|
|
"\n",
|
|
"Verse 2:\n",
|
|
"No sugar, no calories, just pure bliss\n",
|
|
"You're the perfect drink, I must confess\n",
|
|
"From lemon to lime, so many flavors to choose\n",
|
|
"Sparkling water, you never fail to amuse\n",
|
|
"\n",
|
|
"Chorus:\n",
|
|
"Oh sparkling water, you're my delight\n",
|
|
"With every sip, you make me feel so right\n",
|
|
"You're like a party in my mouth\n",
|
|
"I can't get enough, I'm hooked no doubt\n",
|
|
"\n",
|
|
"Bridge:\n",
|
|
"Some may say you're just plain water\n",
|
|
"But to me, you're so much more\n",
|
|
"You bring a sparkle to my day\n",
|
|
"In every single way\n",
|
|
"\n",
|
|
"Chorus:\n",
|
|
"Oh sparkling water, you're my delight\n",
|
|
"With every sip, you make me feel so right\n",
|
|
"You're like a party in my mouth\n",
|
|
"I can't get enough, I'm hooked no doubt\n",
|
|
"\n",
|
|
"Outro:\n",
|
|
"So here's to you, my dear sparkling water\n",
|
|
"You'll always be my go-to drink forever\n",
|
|
"With your effervescence and refreshing taste\n",
|
|
"You'll always have a special place."
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain.llms import OpenAI\n",
|
|
"\n",
|
|
"\n",
|
|
"llm = OpenAI(model=\"gpt-3.5-turbo-instruct\", temperature=0, max_tokens=512)\n",
|
|
"for chunk in llm.stream(\"Write me a song about sparkling water.\"):\n",
|
|
" print(chunk, end=\"\", flush=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "d81140f2-384b-4470-bf93-957013c6620b",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.9.1"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|