diff --git a/docs/extras/modules/callbacks/integrations/promptlayer.ipynb b/docs/extras/modules/callbacks/integrations/promptlayer.ipynb new file mode 100644 index 00000000000..a5fc2d40f06 --- /dev/null +++ b/docs/extras/modules/callbacks/integrations/promptlayer.ipynb @@ -0,0 +1,210 @@ +{ + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# PromptLayer\n", + "\n", + "![PromptLayer](https://promptlayer.com/text_logo.png)\n", + "\n", + "[PromptLayer](https://promptlayer.com) is a an LLM observability platform that lets you visualize requests, version prompts, and track usage. In this guide we will go over how to setup the `PromptLayerCallbackHandler`. \n", + "\n", + "While PromptLayer does have LLMs that integrate directly with LangChain (eg [`PromptLayerOpenAI`](https://python.langchain.com/docs/modules/model_io/models/llms/integrations/promptlayer_openai)), this callback is the recommended way to integrate PromptLayer with LangChain.\n", + "\n", + "See [our docs](https://docs.promptlayer.com/languages/langchain) for more information." + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Installation and Setup" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install promptlayer --upgrade" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Getting API Credentials\n", + "\n", + "If you do not have a PromptLayer account, create one on [promptlayer.com](https://www.promptlayer.com). Then get an API key by clicking on the settings cog in the navbar and\n", + "set it as an environment variabled called `PROMPTLAYER_API_KEY`\n" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Usage\n", + "\n", + "Getting started with `PromptLayerCallbackHandler` is fairly simple, it takes two optional arguments:\n", + "1. `pl_tags` - an optional list of strings that will be tracked as tags on PromptLayer.\n", + "2. `pl_id_callback` - an optional function that will take `promptlayer_request_id` as an argument. This ID can be used with all of PromptLayer's tracking features to track, metadata, scores, and prompt usage." + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Simple OpenAI Example\n", + "\n", + "In this simple example we use `PromptLayerCallbackHandler` with `ChatOpenAI`. We add a PromptLayer tag named `chatopenai`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import promptlayer # Don't forget this 🍰\n", + "from langchain.callbacks import PromptLayerCallbackHandler\n", + "from langchain.chat_models import ChatOpenAI\n", + "from langchain.schema import (\n", + " HumanMessage,\n", + ")\n", + "\n", + "chat_llm = ChatOpenAI(\n", + " temperature=0,\n", + " callbacks=[PromptLayerCallbackHandler(pl_tags=[\"chatopenai\"])],\n", + ")\n", + "llm_results = chat_llm(\n", + " [\n", + " HumanMessage(content=\"What comes after 1,2,3 ?\"),\n", + " HumanMessage(content=\"Tell me another joke?\"),\n", + " ]\n", + ")\n", + "print(llm_results)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### GPT4All Example" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import promptlayer # Don't forget this 🍰\n", + "from langchain.callbacks import PromptLayerCallbackHandler\n", + "\n", + "from langchain.llms import GPT4All\n", + "\n", + "model = GPT4All(model=\"./models/gpt4all-model.bin\", n_ctx=512, n_threads=8)\n", + "\n", + "response = model(\n", + " \"Once upon a time, \",\n", + " callbacks=[PromptLayerCallbackHandler(pl_tags=[\"langchain\", \"gpt4all\"])],\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Full Featured Example\n", + "\n", + "In this example we unlock more of the power of PromptLayer.\n", + "\n", + "PromptLayer allows you to visually create, version, and track prompt templates. Using the [Prompt Registry](https://docs.promptlayer.com/features/prompt-registry), we can programatically fetch the prompt template called `example`.\n", + "\n", + "We also define a `pl_id_callback` function which takes in the `promptlayer_request_id` and logs a score, metadata and links the prompt template used. Read more about tracking on [our docs](https://docs.promptlayer.com/features/prompt-history/request-id)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import promptlayer # Don't forget this 🍰\n", + "from langchain.callbacks import PromptLayerCallbackHandler\n", + "from langchain.llms import OpenAI\n", + "\n", + "\n", + "def pl_id_callback(promptlayer_request_id):\n", + " print(\"prompt layer id \", promptlayer_request_id)\n", + " promptlayer.track.score(\n", + " request_id=promptlayer_request_id, score=100\n", + " ) # score is an integer 0-100\n", + " promptlayer.track.metadata(\n", + " request_id=promptlayer_request_id, metadata={\"foo\": \"bar\"}\n", + " ) # metadata is a dictionary of key value pairs that is tracked on PromptLayer\n", + " promptlayer.track.prompt(\n", + " request_id=promptlayer_request_id,\n", + " prompt_name=\"example\",\n", + " prompt_input_variables={\"product\": \"toasters\"},\n", + " version=1,\n", + " ) # link the request to a prompt template\n", + "\n", + "\n", + "openai_llm = OpenAI(\n", + " model_name=\"text-davinci-002\",\n", + " callbacks=[PromptLayerCallbackHandler(pl_id_callback=pl_id_callback)],\n", + ")\n", + "\n", + "example_prompt = promptlayer.prompts.get(\"example\", version=1, langchain=True)\n", + "openai_llm(example_prompt.format(product=\"toasters\"))" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "That is all it takes! After setup all your requests will show up on the PromptLayer dashboard.\n", + "This callback also works with any LLM implemented on LangChain." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "base", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8 (default, Apr 13 2021, 12:59:45) \n[Clang 10.0.0 ]" + }, + "vscode": { + "interpreter": { + "hash": "c4fe2cd85a8d9e8baaec5340ce66faff1c77581a9f43e6c45e85e09b6fced008" + } + } + }, + "nbformat": 4, + "nbformat_minor": 4 +}