diff --git a/docs/docs/contributing/how_to/integrations/package.mdx b/docs/docs/contributing/how_to/integrations/package.mdx index 89050715e5e..ec8e08bbd9b 100644 --- a/docs/docs/contributing/how_to/integrations/package.mdx +++ b/docs/docs/contributing/how_to/integrations/package.mdx @@ -29,10 +29,11 @@ pip install poetry Next, come up with a name for your package. For this guide, we'll use `langchain-parrot-link`. You can confirm that the name is available on PyPi by searching for it on the [PyPi website](https://pypi.org/). -Next, create your new Python package with Poetry: +Next, create your new Python package with Poetry, and navigate into the new directory with `cd`: ```bash poetry new langchain-parrot-link +cd langchain-parrot-link ``` Add main dependencies using Poetry, which will add them to your `pyproject.toml` file: @@ -48,10 +49,10 @@ with your published package, or just installing them separately when you run tes `langchain-tests` will provide the [standard tests](../standard_tests) we will use later. We recommended pinning these to the latest version: -Note: Replace `{latest version}` with the latest version of `langchain-tests` below. +Note: Replace `` with the latest version of `langchain-tests` below. ```bash -poetry add --group test pytest pytest-socket langchain-tests=={latest version} +poetry add --group test pytest pytest-socket langchain-tests== ``` You're now ready to start writing your integration package! diff --git a/docs/docs/contributing/how_to/integrations/standard_tests.ipynb b/docs/docs/contributing/how_to/integrations/standard_tests.ipynb index c8935838874..f73dbcf01df 100644 --- a/docs/docs/contributing/how_to/integrations/standard_tests.ipynb +++ b/docs/docs/contributing/how_to/integrations/standard_tests.ipynb @@ -10,10 +10,12 @@ "---\n", "# How to add standard tests to an integration\n", "\n", - "When creating either a custom class for yourself or a new tool to publish in a LangChain integration, it is important to add standard tests to ensure it works as expected. This guide will show you how to add standard tests to a tool, and you can **[Skip to the test templates](#standard-test-templates-per-component)** for implementing tests for each integration.\n", + "When creating either a custom class for yourself or to publish in a LangChain integration, it is important to add standard tests to ensure it works as expected. This guide will show you how to add standard tests to a custom chat model, and you can **[Skip to the test templates](#standard-test-templates-per-component)** for implementing tests for each integration type.\n", "\n", "## Setup\n", "\n", + "If you're coming from the [previous guide](../package), you have already installed these dependencies, and you can skip this section.\n", + "\n", "First, let's install 2 dependencies:\n", "\n", "- `langchain-core` will define the interfaces we want to import to define our custom tool.\n", @@ -53,29 +55,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's say we're publishing a package, `langchain_parrot_link`, that exposes a\n", - "tool called `ParrotMultiplyTool`:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# title=\"langchain_parrot_link/tools.py\"\n", - "from langchain_core.tools import BaseTool\n", - "\n", - "\n", - "class ParrotMultiplyTool(BaseTool):\n", - " name: str = \"ParrotMultiplyTool\"\n", - " description: str = (\n", - " \"Multiply two numbers like a parrot. Parrots always add \"\n", - " \"eighty for their matey.\"\n", - " )\n", - "\n", - " def _run(self, a: int, b: int) -> int:\n", - " return a * b + 80" + "Let's say we're publishing a package, `langchain_parrot_link`, that exposes the chat model from the [guide on implementing the package](../package). We can add the standard tests to the package by following the steps below." ] }, { @@ -85,70 +65,33 @@ "And we'll assume you've structured your package the same way as the main LangChain\n", "packages:\n", "\n", - "```\n", - "/\n", + "```plaintext\n", + "langchain-parrot-link/\n", "├── langchain_parrot_link/\n", - "│ └── tools.py\n", - "└── tests/\n", - " ├── unit_tests/\n", - " │ └── test_tools.py\n", - " └── integration_tests/\n", - " └── test_tools.py\n", + "│ ├── __init__.py\n", + "│ └── chat_models.py\n", + "├── tests/\n", + "│ ├── __init__.py\n", + "│ └── test_chat_models.py\n", + "├── pyproject.toml\n", + "└── README.md\n", "```\n", "\n", "## Add and configure standard tests\n", "\n", "There are 2 namespaces in the `langchain-tests` package: \n", "\n", - "- [unit tests](../../../concepts/testing.mdx#unit-tests) (`langchain_tests.unit_tests`): designed to be used to test the tool in isolation and without access to external services\n", - "- [integration tests](../../../concepts/testing.mdx#unit-tests) (`langchain_tests.integration_tests`): designed to be used to test the tool with access to external services (in particular, the external service that the tool is designed to interact with).\n", + "- [unit tests](../../../concepts/testing.mdx#unit-tests) (`langchain_tests.unit_tests`): designed to be used to test the component in isolation and without access to external services\n", + "- [integration tests](../../../concepts/testing.mdx#unit-tests) (`langchain_tests.integration_tests`): designed to be used to test the component with access to external services (in particular, the external service that the component is designed to interact with).\n", "\n", "Both types of tests are implemented as [`pytest` class-based test suites](https://docs.pytest.org/en/7.1.x/getting-started.html#group-multiple-tests-in-a-class).\n", "\n", "By subclassing the base classes for each type of standard test (see below), you get all of the standard tests for that type, and you\n", "can override the properties that the test suite uses to configure the tests.\n", "\n", - "### Standard tools tests\n", + "### Standard chat model tests\n", "\n", - "Here's how you would configure the standard unit tests for the custom tool, e.g. in `tests/test_tools.py`:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "title": "tests/test_custom_tool.py" - }, - "outputs": [], - "source": [ - "# title=\"tests/unit_tests/test_tools.py\"\n", - "from typing import Type\n", - "\n", - "from langchain_parrot_link.tools import ParrotMultiplyTool\n", - "from langchain_tests.unit_tests import ToolsUnitTests\n", - "\n", - "\n", - "class TestParrotMultiplyToolUnit(ToolsUnitTests):\n", - " @property\n", - " def tool_constructor(self) -> Type[ParrotMultiplyTool]:\n", - " return ParrotMultiplyTool\n", - "\n", - " @property\n", - " def tool_constructor_params(self) -> dict:\n", - " # if your tool constructor instead required initialization arguments like\n", - " # `def __init__(self, some_arg: int):`, you would return those here\n", - " # as a dictionary, e.g.: `return {'some_arg': 42}`\n", - " return {}\n", - "\n", - " @property\n", - " def tool_invoke_params_example(self) -> dict:\n", - " \"\"\"\n", - " Returns a dictionary representing the \"args\" of an example tool call.\n", - "\n", - " This should NOT be a ToolCall dict - i.e. it should not\n", - " have {\"name\", \"id\", \"args\"} keys.\n", - " \"\"\"\n", - " return {\"a\": 2, \"b\": 3}" + "Here's how you would configure the standard unit tests for the custom chat model:" ] }, { @@ -157,34 +100,52 @@ "metadata": {}, "outputs": [], "source": [ - "# title=\"tests/integration_tests/test_tools.py\"\n", + "# title=\"tests/unit_tests/test_chat_models.py\"\n", + "from typing import Tuple, Type\n", + "\n", + "from langchain_parrot_link.chat_models import ChatParrotLink\n", + "from langchain_tests.unit_tests import ChatModelUnitTests\n", + "\n", + "\n", + "class TestChatParrotLinkUnit(ChatModelUnitTests):\n", + " @property\n", + " def chat_model_class(self) -> Type[ChatParrotLink]:\n", + " return ChatParrotLink\n", + "\n", + " @property\n", + " def chat_model_params(self) -> dict:\n", + " return {\n", + " \"model\": \"bird-brain-001\",\n", + " \"temperature\": 0,\n", + " \"parrot_buffer_length\": 50,\n", + " }" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# title=\"tests/integration_tests/test_chat_models.py\"\n", "from typing import Type\n", "\n", - "from langchain_parrot_link.tools import ParrotMultiplyTool\n", - "from langchain_tests.integration_tests import ToolsIntegrationTests\n", + "from langchain_parrot_link.chat_models import ChatParrotLink\n", + "from langchain_tests.integration_tests import ChatModelIntegrationTests\n", "\n", "\n", - "class TestParrotMultiplyToolIntegration(ToolsIntegrationTests):\n", + "class TestChatParrotLinkIntegration(ChatModelIntegrationTests):\n", " @property\n", - " def tool_constructor(self) -> Type[ParrotMultiplyTool]:\n", - " return ParrotMultiplyTool\n", + " def chat_model_class(self) -> Type[ChatParrotLink]:\n", + " return ChatParrotLink\n", "\n", " @property\n", - " def tool_constructor_params(self) -> dict:\n", - " # if your tool constructor instead required initialization arguments like\n", - " # `def __init__(self, some_arg: int):`, you would return those here\n", - " # as a dictionary, e.g.: `return {'some_arg': 42}`\n", - " return {}\n", - "\n", - " @property\n", - " def tool_invoke_params_example(self) -> dict:\n", - " \"\"\"\n", - " Returns a dictionary representing the \"args\" of an example tool call.\n", - "\n", - " This should NOT be a ToolCall dict - i.e. it should not\n", - " have {\"name\", \"id\", \"args\"} keys.\n", - " \"\"\"\n", - " return {\"a\": 2, \"b\": 3}" + " def chat_model_params(self) -> dict:\n", + " return {\n", + " \"model\": \"bird-brain-001\",\n", + " \"temperature\": 0,\n", + " \"parrot_buffer_length\": 50,\n", + " }" ] }, { @@ -228,7 +189,8 @@ "Above, we implement the **unit** and **integration** standard tests for a tool. Below are the templates for implementing the standard tests for each component:\n", "\n", "
\n", - " Chat Models" + " Chat Models\n", + "

Note: The standard tests for chat models are implemented in the example in the main body of this guide too.

" ] }, { @@ -251,7 +213,11 @@ "\n", " @property\n", " def chat_model_params(self) -> dict:\n", - " return {\"model\": \"bird-brain-001\", \"temperature\": 0}" + " return {\n", + " \"model\": \"bird-brain-001\",\n", + " \"temperature\": 0,\n", + " \"parrot_buffer_length\": 50,\n", + " }" ] }, { @@ -274,7 +240,11 @@ "\n", " @property\n", " def chat_model_params(self) -> dict:\n", - " return {\"model\": \"bird-brain-001\", \"temperature\": 0}" + " return {\n", + " \"model\": \"bird-brain-001\",\n", + " \"temperature\": 0,\n", + " \"parrot_buffer_length\": 50,\n", + " }" ] }, { @@ -338,8 +308,7 @@ "source": [ "
\n", "
\n", - " Tools/Toolkits\n", - "

Note: The standard tests for tools/toolkits are implemented in the example in the main body of this guide too.

" + " Tools/Toolkits" ] }, {