community[patch]: Add maritalk streaming (sync and async) (#19203)

Co-authored-by: RosevalJr <rdmalajr@gmail.com>
Co-authored-by: Roseval Donisete Malaquias Junior <roseval@maritaca.ai>
This commit is contained in:
Rodrigo Nogueira
2024-04-29 18:31:14 -03:00
committed by GitHub
parent cc6191cb90
commit 90f19028e5
2 changed files with 258 additions and 12 deletions

View File

@@ -33,7 +33,7 @@
"metadata": {},
"outputs": [],
"source": [
"!pip install langchain langchain-core langchain-community"
"!pip install langchain langchain-core langchain-community httpx"
]
},
{
@@ -89,6 +89,58 @@
"print(response) # should answer something like \"1. Max\\n2. Bella\\n3. Charlie\\n4. Rocky\""
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Stream Generation\n",
"\n",
"For tasks involving the generation of long text, such as creating an extensive article or translating a large document, it can be advantageous to receive the response in parts, as the text is generated, instead of waiting for the complete text. This makes the application more responsive and efficient, especially when the generated text is extensive. We offer two approaches to meet this need: one synchronous and another asynchronous.\n",
"\n",
"#### Synchronous:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_core.messages import HumanMessage\n",
"\n",
"messages = [HumanMessage(content=\"Suggest 3 names for my dog\")]\n",
"\n",
"for chunk in llm.stream(messages):\n",
" print(chunk.content, end=\"\", flush=True)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Asynchronous:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_core.messages import HumanMessage\n",
"\n",
"\n",
"async def async_invoke_chain(animal: str):\n",
" messages = [HumanMessage(content=f\"Suggest 3 names for my {animal}\")]\n",
" async for chunk in llm._astream(messages):\n",
" print(chunk.message.content, end=\"\", flush=True)\n",
"\n",
"\n",
"await async_invoke_chain(\"dog\")"
]
},
{
"cell_type": "markdown",
"metadata": {},