langchain/docs/extras/integrations/callbacks/promptlayer.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PromptLayer\n",
    "\n",
    "![PromptLayer](https://promptlayer.com/text_logo.png)\n",
    "\n",
    "[PromptLayer](https://promptlayer.com) is a an LLM observability platform that lets you visualize requests, version prompts, and track usage. In this guide we will go over how to setup the `PromptLayerCallbackHandler`. \n",
    "\n",
    "While PromptLayer does have LLMs that integrate directly with LangChain (e.g. [`PromptLayerOpenAI`](https://python.langchain.com/docs/integrations/llms/promptlayer_openai)), this callback is the recommended way to integrate PromptLayer with LangChain.\n",
    "\n",
    "See [our docs](https://docs.promptlayer.com/languages/langchain) for more information."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Installation and Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install promptlayer --upgrade"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting API Credentials\n",
    "\n",
    "If you do not have a PromptLayer account, create one on [promptlayer.com](https://www.promptlayer.com). Then get an API key by clicking on the settings cog in the navbar and\n",
    "set it as an environment variabled called `PROMPTLAYER_API_KEY`\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Usage\n",
    "\n",
    "Getting started with `PromptLayerCallbackHandler` is fairly simple, it takes two optional arguments:\n",
    "1. `pl_tags` - an optional list of strings that will be tracked as tags on PromptLayer.\n",
    "2. `pl_id_callback` - an optional function that will take `promptlayer_request_id` as an argument. This ID can be used with all of PromptLayer's tracking features to track, metadata, scores, and prompt usage."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Simple OpenAI Example\n",
    "\n",
    "In this simple example we use `PromptLayerCallbackHandler` with `ChatOpenAI`. We add a PromptLayer tag named `chatopenai`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import promptlayer  # Don't forget this 🍰\n",
    "from langchain.callbacks import PromptLayerCallbackHandler\n",
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain.schema import (\n",
    "    HumanMessage,\n",
    ")\n",
    "\n",
    "chat_llm = ChatOpenAI(\n",
    "    temperature=0,\n",
    "    callbacks=[PromptLayerCallbackHandler(pl_tags=[\"chatopenai\"])],\n",
    ")\n",
    "llm_results = chat_llm(\n",
    "    [\n",
    "        HumanMessage(content=\"What comes after 1,2,3 ?\"),\n",
    "        HumanMessage(content=\"Tell me another joke?\"),\n",
    "    ]\n",
    ")\n",
    "print(llm_results)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GPT4All Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import promptlayer  # Don't forget this 🍰\n",
    "from langchain.callbacks import PromptLayerCallbackHandler\n",
    "\n",
    "from langchain.llms import GPT4All\n",
    "\n",
    "model = GPT4All(model=\"./models/gpt4all-model.bin\", n_ctx=512, n_threads=8)\n",
    "\n",
    "response = model(\n",
    "    \"Once upon a time, \",\n",
    "    callbacks=[PromptLayerCallbackHandler(pl_tags=[\"langchain\", \"gpt4all\"])],\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Full Featured Example\n",
    "\n",
    "In this example we unlock more of the power of PromptLayer.\n",
    "\n",
    "PromptLayer allows you to visually create, version, and track prompt templates. Using the [Prompt Registry](https://docs.promptlayer.com/features/prompt-registry), we can programatically fetch the prompt template called `example`.\n",
    "\n",
    "We also define a `pl_id_callback` function which takes in the `promptlayer_request_id` and logs a score, metadata and links the prompt template used. Read more about tracking on [our docs](https://docs.promptlayer.com/features/prompt-history/request-id)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import promptlayer  # Don't forget this 🍰\n",
    "from langchain.callbacks import PromptLayerCallbackHandler\n",
    "from langchain.llms import OpenAI\n",
    "\n",
    "\n",
    "def pl_id_callback(promptlayer_request_id):\n",
    "    print(\"prompt layer id \", promptlayer_request_id)\n",
    "    promptlayer.track.score(\n",
    "        request_id=promptlayer_request_id, score=100\n",
    "    )  # score is an integer 0-100\n",
    "    promptlayer.track.metadata(\n",
    "        request_id=promptlayer_request_id, metadata={\"foo\": \"bar\"}\n",
    "    )  # metadata is a dictionary of key value pairs that is tracked on PromptLayer\n",
    "    promptlayer.track.prompt(\n",
    "        request_id=promptlayer_request_id,\n",
    "        prompt_name=\"example\",\n",
    "        prompt_input_variables={\"product\": \"toasters\"},\n",
    "        version=1,\n",
    "    )  # link the request to a prompt template\n",
    "\n",
    "\n",
    "openai_llm = OpenAI(\n",
    "    model_name=\"text-davinci-002\",\n",
    "    callbacks=[PromptLayerCallbackHandler(pl_id_callback=pl_id_callback)],\n",
    ")\n",
    "\n",
    "example_prompt = promptlayer.prompts.get(\"example\", version=1, langchain=True)\n",
    "openai_llm(example_prompt.format(product=\"toasters\"))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That is all it takes! After setup all your requests will show up on the PromptLayer dashboard.\n",
    "This callback also works with any LLM implemented on LangChain."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8 (default, Apr 13 2021, 12:59:45) \n[Clang 10.0.0 ]"
  },
  "vscode": {
   "interpreter": {
    "hash": "c4fe2cd85a8d9e8baaec5340ce66faff1c77581a9f43e6c45e85e09b6fced008"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}