From e787249af1823fe482c0812a61b80b6640fc8ee6 Mon Sep 17 00:00:00 2001 From: Bagatur <22008038+baskaryan@users.noreply.github.com> Date: Wed, 3 Jul 2024 10:33:05 -0400 Subject: [PATCH] docs: fireworks standard page (#23816) --- docs/docs/integrations/chat/fireworks.ipynb | 334 +++++++++++--------- 1 file changed, 190 insertions(+), 144 deletions(-) diff --git a/docs/docs/integrations/chat/fireworks.ipynb b/docs/docs/integrations/chat/fireworks.ipynb index da519e8ba55..1e83aea7e3c 100644 --- a/docs/docs/integrations/chat/fireworks.ipynb +++ b/docs/docs/integrations/chat/fireworks.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "raw", - "id": "529aeba9", + "id": "afaf8039", "metadata": {}, "source": [ "---\n", @@ -11,197 +11,243 @@ ] }, { - "attachments": {}, "cell_type": "markdown", - "id": "642fd21c-600a-47a1-be96-6e1438b421a9", + "id": "e49f1e0d", "metadata": {}, "source": [ "# ChatFireworks\n", "\n", - ">[Fireworks](https://app.fireworks.ai/) accelerates product development on generative AI by creating an innovative AI experiment and production platform. \n", + "This doc help you get started with Fireworks AI [chat models](/docs/concepts/#chat-models). For detailed documentation of all ChatFireworks features and configurations head to the [API reference](https://api.python.langchain.com/en/latest/chat_models/langchain_fireworks.chat_models.ChatFireworks.html).\n", "\n", - "This example goes over how to use LangChain to interact with `ChatFireworks` models." - ] - }, - { - "cell_type": "raw", - "id": "4a7c795e", - "metadata": {}, - "source": [ - "%pip install langchain-fireworks" + "Fireworks AI is an AI inference platform to run and customize models. For a list of all models served by Fireworks see the [Fireworks docs](https://fireworks.ai/models).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/v0.2/docs/integrations/chat/fireworks) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatFireworks](https://api.python.langchain.com/en/latest/chat_models/langchain_fireworks.chat_models.ChatFireworks.html) | [langchain-fireworks](https://api.python.langchain.com/en/latest/fireworks_api_reference.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-fireworks?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-fireworks?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ | \n", + "\n", + "## Setup\n", + "\n", + "To access Fireworks models you'll need to create a Fireworks account, get an API key, and install the `langchain-fireworks` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to (ttps://fireworks.ai/login to sign up to Fireworks and generate an API key. Once you've done this set the FIREWORKS_API_KEY environment variable:" ] }, { "cell_type": "code", - "execution_count": 1, - "id": "d00d850917865298", - "metadata": { - "collapsed": false, - "jupyter": { - "outputs_hidden": false - } - }, - "outputs": [], - "source": [ - "from langchain_core.messages import HumanMessage, SystemMessage\n", - "from langchain_fireworks import ChatFireworks" - ] - }, - { - "cell_type": "markdown", - "id": "f28ebf8b-f14f-46c7-9962-8b8dc42e31be", - "metadata": {}, - "source": [ - "# Setup\n", - "\n", - "1. Make sure the `langchain-fireworks` package is installed in your environment.\n", - "2. Sign in to [Fireworks AI](http://fireworks.ai) for the an API Key to access our models, and make sure it is set as the `FIREWORKS_API_KEY` environment variable.\n", - "3. Set up your model using a model id. If the model is not set, the default model is fireworks-llama-v2-7b-chat. See the full, most up-to-date model list on [app.fireworks.ai](https://app.fireworks.ai)." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "d096fb14-8acc-4047-9cd0-c842430c3a1d", + "execution_count": null, + "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", "metadata": {}, "outputs": [], "source": [ "import getpass\n", "import os\n", "\n", - "if \"FIREWORKS_API_KEY\" not in os.environ:\n", - " os.environ[\"FIREWORKS_API_KEY\"] = getpass.getpass(\"Fireworks API Key:\")\n", - "\n", - "# Initialize a Fireworks chat model\n", - "chat = ChatFireworks(model=\"accounts/fireworks/models/mixtral-8x7b-instruct\")" + "os.environ[\"FIREWORKS_API_KEY\"] = getpass.getpass(\"Enter your Fireworks API key: \")" ] }, { "cell_type": "markdown", - "id": "d8f13144-37cf-47a5-b5a0-e3cdf76d9a72", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", "metadata": {}, "source": [ - "# Calling the Model Directly\n", + "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", "\n", - "You can call the model directly with a system and human message to get answers." + "The LangChain Fireworks integration lives in the `langchain-fireworks` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-fireworks" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:\n", + "\n", + "- TODO: Update model instantiation with relevant params." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_fireworks import ChatFireworks\n", + "\n", + "llm = ChatFireworks(\n", + " model=\"accounts/fireworks/models/llama-v3-70b-instruct\",\n", + " temperature=0,\n", + " max_tokens=None,\n", + " timeout=None,\n", + " max_retries=2,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"J'adore la programmation.\", response_metadata={'token_usage': {'prompt_tokens': 35, 'total_tokens': 44, 'completion_tokens': 9}, 'model_name': 'accounts/fireworks/models/llama-v3-70b-instruct', 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None}, id='run-df28e69a-ff30-457e-a743-06eb14d01cb0-0', usage_metadata={'input_tokens': 35, 'output_tokens': 9, 'total_tokens': 44})" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" ] }, { "cell_type": "code", "execution_count": 3, - "id": "72340871-ae2f-415f-b399-0777d32dc379", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content=\"Hello! I'm an AI language model, a helpful assistant designed to chat and assist you with any questions or information you might need. I'm here to make your experience as smooth and enjoyable as possible. How can I assist you today?\")" - ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# ChatFireworks Wrapper\n", - "system_message = SystemMessage(content=\"You are to chat with the user.\")\n", - "human_message = HumanMessage(content=\"Who are you?\")\n", - "\n", - "chat.invoke([system_message, human_message])" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "68c6b1fa-2ff7-4a63-8d88-3cec302180b8", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content=\"I'm an AI and do not have the ability to experience the weather firsthand. However,\")" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Setting additional parameters: temperature, max_tokens, top_p\n", - "chat = ChatFireworks(\n", - " model=\"accounts/fireworks/models/mixtral-8x7b-instruct\",\n", - " temperature=1,\n", - " max_tokens=20,\n", - ")\n", - "system_message = SystemMessage(content=\"You are to chat with the user.\")\n", - "human_message = HumanMessage(content=\"How's the weather today?\")\n", - "chat.invoke([system_message, human_message])" - ] - }, - { - "cell_type": "markdown", - "id": "8c44cb36", - "metadata": {}, - "source": [ - "# Tool Calling\n", - "\n", - "Fireworks offers the `FireFunction-v2` tool calling model. You can use it for structured output and function calling use cases:" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "ee2db682", + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "{'function': {'arguments': '{\"name\": \"Erick\", \"age\": 27}',\n", - " 'name': 'ExtractFields'},\n", - " 'id': 'call_J0WYP2TLenaFw3UeVU0UnWqx',\n", - " 'index': 0,\n", - " 'type': 'function'}\n" + "J'adore la programmation.\n" ] } ], "source": [ - "from pprint import pprint\n", + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", "\n", - "from langchain_core.pydantic_v1 import BaseModel\n", - "\n", - "\n", - "class ExtractFields(BaseModel):\n", - " name: str\n", - " age: int\n", - "\n", - "\n", - "chat = ChatFireworks(\n", - " model=\"accounts/fireworks/models/firefunction-v2\",\n", - ").bind_tools([ExtractFields])\n", - "\n", - "result = chat.invoke(\"I am a 27 year old named Erick\")\n", - "\n", - "pprint(result.additional_kwargs[\"tool_calls\"][0])" + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" ] }, { "cell_type": "code", - "execution_count": null, - "id": "2321a4e6", + "execution_count": 4, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", "metadata": {}, - "outputs": [], - "source": [] + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'prompt_tokens': 30, 'total_tokens': 37, 'completion_tokens': 7}, 'model_name': 'accounts/fireworks/models/llama-v3-70b-instruct', 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None}, id='run-ff3f91ad-ed81-4acf-9f59-7490dc8d8f48-0', usage_metadata={'input_tokens': 30, 'output_tokens': 7, 'total_tokens': 37})" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatFireworks features and configurations head to the API reference: https://api.python.langchain.com/en/latest/chat_models/langchain_fireworks.chat_models.ChatFireworks.html" + ] } ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "poetry-venv-2", "language": "python", - "name": "python3" + "name": "poetry-venv-2" }, "language_info": { "codemirror_mode": { @@ -213,7 +259,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.4" + "version": "3.11.9" } }, "nbformat": 4,