{
 "cells": [
  {
   "cell_type": "raw",
   "id": "e9437c8a-d8b7-4bf6-8ff4-54068a5a266c",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_position: 1.5\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0df7646-b1e1-4014-a841-6dae9b3c50d9",
   "metadata": {},
   "source": [
    "# How to stream chat model responses\n",
    "\n",
    "All ChatModels implement the Runnable interface, which comes with default implementations of all methods, ie. ainvoke, batch, abatch, stream, astream. This gives all ChatModels basic support for streaming.\n",
    "\n",
    "Streaming support defaults to returning an Iterator (or AsyncIterator in the case of async streaming) of a single value, the final result returned by the underlying ChatModel provider. This obviously doesn't give you token-by-token streaming, which requires native support from the ChatModel provider, but ensures your code that expects an iterator of tokens can work for any of our ChatModel integrations.\n",
    "\n",
    "See which [integrations support token-by-token streaming here](/docs/integrations/chat/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "03080a2c-45e8-45b9-a367-62816eae54c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.chat_models import ChatAnthropic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "975c4f32-21f6-4a71-9091-f87b56347c33",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Here's a song I just improvised about goldfish on the moon:\n",
      "\n",
      "Floating in space, looking for a place \n",
      "To call their home, all alone\n",
      "Swimming through stars, these goldfish from Mars\n",
      "Left their fishbowl behind, a new life to find\n",
      "On the moon, where the craters loom\n",
      "Searching for food, maybe some lunar food\n",
      "Out of their depth, close to death\n",
      "How they wish, for just one small fish\n",
      "To join them up here, their future unclear\n",
      "On the moon, where the Earth looms\n",
      "Dreaming of home, filled with foam\n",
      "Their bodies adapt, continuing to last \n",
      "On the moon, where they learn to swoon\n",
      "Over cheese that astronauts tease\n",
      "As they stare back at Earth, the planet of birth\n",
      "These goldfish out of water, swim on and on\n",
      "Lunar pioneers, conquering their fears\n",
      "On the moon, where they happily swoon"
     ]
    }
   ],
   "source": [
    "chat = ChatAnthropic(model=\"claude-2\")\n",
    "for chunk in chat.stream(\"Write me a song about goldfish on the moon\"):\n",
    "    print(chunk.content, end=\"\", flush=True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}