From 34e94755af66c1d00e95c8d67db7609fa8af07fd Mon Sep 17 00:00:00 2001 From: Dharshan A Date: Mon, 10 Mar 2025 18:26:07 +0530 Subject: [PATCH 01/14] Fix typo in astream_events in streaming docs (#30195) Thank you for contributing to LangChain! - [ ] **PR title**: "package: description" - Where "package" is whichever of langchain, community, core, etc. is being modified. Use "docs: ..." for purely docs changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [ ] **PR message**: ***Delete this entire checklist*** and replace with - **Description:** a description of the change - **Issue:** the issue # it fixes, if applicable - **Dependencies:** any dependencies required for this change - **Twitter handle:** if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [ ] **Add tests and docs**: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, eyurtsev, ccurme, vbarda, hwchase17. --- docs/docs/concepts/streaming.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs/concepts/streaming.mdx b/docs/docs/concepts/streaming.mdx index 8a02537102b..c2dc400e23e 100644 --- a/docs/docs/concepts/streaming.mdx +++ b/docs/docs/concepts/streaming.mdx @@ -91,7 +91,7 @@ For more information, please see: #### Usage with LCEL -If you compose multiple Runnables using [LangChain’s Expression Language (LCEL)](/docs/concepts/lcel), the `stream()` and `astream()` methods will, by convention, stream the output of the last step in the chain. This allows the final processed result to be streamed incrementally. **LCEL** tries to optimize streaming latency in pipelines such that the streaming results from the last step are available as soon as possible. +If you compose multiple Runnables using [LangChain’s Expression Language (LCEL)](/docs/concepts/lcel), the `stream()` and `astream()` methods will, by convention, stream the output of the last step in the chain. This allows the final processed result to be streamed incrementally. **LCEL** tries to optimize streaming latency in pipelines so that the streaming results from the last step are available as soon as possible. @@ -104,7 +104,7 @@ Use the `astream_events` API to access custom data and intermediate outputs from While this API is available for use with [LangGraph](/docs/concepts/architecture#langgraph) as well, it is usually not necessary when working with LangGraph, as the `stream` and `astream` methods provide comprehensive streaming capabilities for LangGraph graphs. ::: -For chains constructed using **LCEL**, the `.stream()` method only streams the output of the final step from te chain. This might be sufficient for some applications, but as you build more complex chains of several LLM calls together, you may want to use the intermediate values of the chain alongside the final output. For example, you may want to return sources alongside the final generation when building a chat-over-documents app. +For chains constructed using **LCEL**, the `.stream()` method only streams the output of the final step from the chain. This might be sufficient for some applications, but as you build more complex chains of several LLM calls together, you may want to use the intermediate values of the chain alongside the final output. For example, you may want to return sources alongside the final generation when building a chat-over-documents app. There are ways to do this [using callbacks](/docs/concepts/callbacks), or by constructing your chain in such a way that it passes intermediate values to the end with something like chained [`.assign()`](/docs/how_to/passthrough/) calls, but LangChain also includes an From aa6dae4a5bd6f62f82497f5c29ec72a55367bbd1 Mon Sep 17 00:00:00 2001 From: Hugh Gao Date: Mon, 10 Mar 2025 20:58:40 +0800 Subject: [PATCH 02/14] community: Remove the system message count limit for ChatTongyi. (#30192) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Description The models in DashScope support multiple SystemMessage. Here is the [Doc](https://bailian.console.aliyun.com/model_experience_center/text#/model-market/detail/qwen-long?tabKey=sdk), and the example code on the document page: ```python import os from openai import OpenAI client = OpenAI( api_key=os.getenv("DASHSCOPE_API_KEY"), # 如果您没有配置环境变量,请在此处替换您的API-KEY base_url="https://dashscope.aliyuncs.com/compatible-mode/v1", # 填写DashScope服务base_url ) # 初始化messages列表 completion = client.chat.completions.create( model="qwen-long", messages=[ {'role': 'system', 'content': 'You are a helpful assistant.'}, # 请将 'file-fe-xxx'替换为您实际对话场景所使用的 file-id。 {'role': 'system', 'content': 'fileid://file-fe-xxx'}, {'role': 'user', 'content': '这篇文章讲了什么?'} ], stream=True, stream_options={"include_usage": True} ) full_content = "" for chunk in completion: if chunk.choices and chunk.choices[0].delta.content: # 拼接输出内容 full_content += chunk.choices[0].delta.content print(chunk.model_dump()) print({full_content}) ``` Tip: The example code is for OpenAI, but the document said that it also supports the DataScope API, and I tested it, and it works. ``` Is the Dashscope SDK invocation method compatible? Yes, the Dashscope SDK remains compatible for model invocation. However, file uploads and file-ID retrieval are currently only supported via the OpenAI SDK. The file-ID obtained through this method is also compatible with Dashscope for model invocation. ``` --- libs/community/langchain_community/chat_models/tongyi.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/libs/community/langchain_community/chat_models/tongyi.py b/libs/community/langchain_community/chat_models/tongyi.py index de02213c580..4c44272f271 100644 --- a/libs/community/langchain_community/chat_models/tongyi.py +++ b/libs/community/langchain_community/chat_models/tongyi.py @@ -783,8 +783,6 @@ class ChatTongyi(BaseChatModel): ] if len(system_message_indices) == 1 and system_message_indices[0] != 0: raise ValueError("System message can only be the first message.") - elif len(system_message_indices) > 1: - raise ValueError("There can be only one system message at most.") params["messages"] = message_dicts From 7b8f266039397bdb8cec4535ed745d8d54145a18 Mon Sep 17 00:00:00 2001 From: ccurme Date: Mon, 10 Mar 2025 08:59:59 -0400 Subject: [PATCH 03/14] infra: additional testing on core release (#30180) Here we add a job to the release workflow that, when releasing `langchain-core`, tests prior published versions of select packages against the new version of core. We limit the testing to the most recent published versions of langchain-anthropic and langchain-openai. This is designed to catch backward-incompatible updates to core. We sometimes update core and downstream packages simultaneously, so there may not be any commit in the history at which tests would fail. So although core and latest downstream packages could be consistent, we can benefit from testing prior versions of downstream packages against core. I tested the workflow by simulating a [breaking change](https://github.com/langchain-ai/langchain/pull/30180/commits/d7287248cf6edd0c4224f3b2dea65805802f8248) in core and running it with publishing steps disabled: https://github.com/langchain-ai/langchain/actions/runs/13741876345. The workflow correctly caught the issue. --- .github/workflows/_release.yml | 67 ++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/.github/workflows/_release.yml b/.github/workflows/_release.yml index 92dd272b769..e72ed2bce79 100644 --- a/.github/workflows/_release.yml +++ b/.github/workflows/_release.yml @@ -312,12 +312,79 @@ jobs: run: make integration_tests working-directory: ${{ inputs.working-directory }} + # Test select published packages against new core + test-prior-published-packages-against-new-core: + needs: + - build + - release-notes + - test-pypi-publish + - pre-release-checks + if: ${{ startsWith(inputs.working-directory, 'libs/core') }} + runs-on: ubuntu-latest + strategy: + matrix: + partner: [openai, anthropic] + fail-fast: false # Continue testing other partners if one fails + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} + AZURE_OPENAI_API_VERSION: ${{ secrets.AZURE_OPENAI_API_VERSION }} + AZURE_OPENAI_API_BASE: ${{ secrets.AZURE_OPENAI_API_BASE }} + AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }} + AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_CHAT_DEPLOYMENT_NAME }} + AZURE_OPENAI_LEGACY_CHAT_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_LEGACY_CHAT_DEPLOYMENT_NAME }} + AZURE_OPENAI_LLM_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_LLM_DEPLOYMENT_NAME }} + AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT_NAME }} + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uv + uses: "./.github/actions/uv_setup" + with: + python-version: ${{ env.PYTHON_VERSION }} + + - uses: actions/download-artifact@v4 + with: + name: dist + path: ${{ inputs.working-directory }}/dist/ + + - name: Test against ${{ matrix.partner }} + run: | + # Identify latest tag + LATEST_PACKAGE_TAG="$( + git ls-remote --tags origin "langchain-${{ matrix.partner }}*" \ + | awk '{print $2}' \ + | sed 's|refs/tags/||' \ + | sort -Vr \ + | head -n 1 + )" + echo "Latest package tag: $LATEST_PACKAGE_TAG" + + # Shallow-fetch just that single tag + git fetch --depth=1 origin tag "$LATEST_PACKAGE_TAG" + + # Navigate to the partner directory + cd $GITHUB_WORKSPACE/libs/partners/${{ matrix.partner }} + + # Checkout the latest package files + git checkout "$LATEST_PACKAGE_TAG" -- . + + # Print as a sanity check + echo "Version number from pyproject.toml: " + cat pyproject.toml | grep "version = " + + # Run tests + uv sync --group test --group test_integration + uv pip install ../../core/dist/*.whl + make integration_tests + publish: needs: - build - release-notes - test-pypi-publish - pre-release-checks + - test-prior-published-packages-against-new-core runs-on: ubuntu-latest permissions: # This permission is used for trusted publishing: From f896e701eb420cd295f26618b4a855a4bf52ffde Mon Sep 17 00:00:00 2001 From: ccurme Date: Mon, 10 Mar 2025 12:58:17 -0400 Subject: [PATCH 04/14] deepseek: install local langchain-tests in test deps (#30198) --- libs/partners/deepseek/pyproject.toml | 3 ++- libs/partners/deepseek/uv.lock | 35 ++++++++++++++++++++------- 2 files changed, 28 insertions(+), 10 deletions(-) diff --git a/libs/partners/deepseek/pyproject.toml b/libs/partners/deepseek/pyproject.toml index b8b10514d18..e7fbbc55246 100644 --- a/libs/partners/deepseek/pyproject.toml +++ b/libs/partners/deepseek/pyproject.toml @@ -26,7 +26,7 @@ test = [ "pytest-asyncio<1.0.0,>=0.23.2", "pytest-socket<1.0.0,>=0.7.0", "pytest-watcher<1.0.0,>=0.3.4", - "langchain-tests<1.0.0,>=0.3.5", + "langchain-tests", "langchain-openai", "pytest-timeout<3.0.0,>=2.3.1", ] @@ -40,6 +40,7 @@ typing = ["mypy<2.0,>=1.10"] [tool.uv.sources] langchain-openai = { path = "../openai", editable = true } langchain-core = { path = "../../core", editable = true } +langchain-tests = { path = "../../standard-tests", editable = true } [tool.mypy] disallow_untyped_defs = "True" diff --git a/libs/partners/deepseek/uv.lock b/libs/partners/deepseek/uv.lock index 4820cc01c05..fd30e79a64d 100644 --- a/libs/partners/deepseek/uv.lock +++ b/libs/partners/deepseek/uv.lock @@ -367,7 +367,7 @@ wheels = [ [[package]] name = "langchain-core" -version = "0.3.35" +version = "0.3.43" source = { editable = "../../core" } dependencies = [ { name = "jsonpatch" }, @@ -399,7 +399,7 @@ dev = [ ] lint = [{ name = "ruff", specifier = ">=0.9.2,<1.0.0" }] test = [ - { name = "blockbuster", specifier = "~=1.5.11" }, + { name = "blockbuster", specifier = "~=1.5.18" }, { name = "freezegun", specifier = ">=1.2.2,<2.0.0" }, { name = "grandalf", specifier = ">=0.8,<1.0" }, { name = "langchain-tests", directory = "../../standard-tests" }, @@ -464,7 +464,7 @@ dev = [] lint = [{ name = "ruff", specifier = ">=0.5,<1.0" }] test = [ { name = "langchain-openai", editable = "../openai" }, - { name = "langchain-tests", specifier = ">=0.3.5,<1.0.0" }, + { name = "langchain-tests", editable = "../../standard-tests" }, { name = "pytest", specifier = ">=7.4.3,<8.0.0" }, { name = "pytest-asyncio", specifier = ">=0.23.2,<1.0.0" }, { name = "pytest-socket", specifier = ">=0.7.0,<1.0.0" }, @@ -476,7 +476,7 @@ typing = [{ name = "mypy", specifier = ">=1.10,<2.0" }] [[package]] name = "langchain-openai" -version = "0.3.5" +version = "0.3.8" source = { editable = "../openai" } dependencies = [ { name = "langchain-core" }, @@ -524,8 +524,8 @@ typing = [ [[package]] name = "langchain-tests" -version = "0.3.10" -source = { registry = "https://pypi.org/simple" } +version = "0.3.14" +source = { editable = "../../standard-tests" } dependencies = [ { name = "httpx" }, { name = "langchain-core" }, @@ -536,9 +536,26 @@ dependencies = [ { name = "pytest-socket" }, { name = "syrupy" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/80/24/b1ef0d74222d04c4196e673e3ae8bac9f89481c17c4e6a72c67f61b403c7/langchain_tests-0.3.10.tar.gz", hash = "sha256:ba0ce038cb633e906961efc85591dd86b28d5c84a7880e7e0cd4dcb833d604a8", size = 31022 } -wheels = [ - { url = "https://files.pythonhosted.org/packages/d0/c3/2f2f2e919bbb9f8608389ac926c6cf8f717c3965956f0e5f139372742fb9/langchain_tests-0.3.10-py3-none-any.whl", hash = "sha256:393e15990b9d1d12b52ee832257e874beb4299891d98ec7682b7fba12c0f8fe1", size = 37521 }, + +[package.metadata] +requires-dist = [ + { name = "httpx", specifier = ">=0.25.0,<1" }, + { name = "langchain-core", editable = "../../core" }, + { name = "numpy", specifier = ">=1.26.2,<3" }, + { name = "pytest", specifier = ">=7,<9" }, + { name = "pytest-asyncio", specifier = ">=0.20,<1" }, + { name = "pytest-socket", specifier = ">=0.6.0,<1" }, + { name = "syrupy", specifier = ">=4,<5" }, +] + +[package.metadata.requires-dev] +codespell = [{ name = "codespell", specifier = ">=2.2.0,<3.0.0" }] +lint = [{ name = "ruff", specifier = ">=0.9.2,<1.0.0" }] +test = [{ name = "langchain-core", editable = "../../core" }] +test-integration = [] +typing = [ + { name = "langchain-core", editable = "../../core" }, + { name = "mypy", specifier = ">=1,<2" }, ] [[package]] From 38420ee76e44d1b1b01f6d0d5781ffb148efca2c Mon Sep 17 00:00:00 2001 From: ccurme Date: Mon, 10 Mar 2025 15:17:20 -0400 Subject: [PATCH 05/14] docs: add note on Deepseek R1 (#30201) --- docs/docs/integrations/chat/deepseek.ipynb | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/docs/integrations/chat/deepseek.ipynb b/docs/docs/integrations/chat/deepseek.ipynb index b00408c1afe..5004dc85212 100644 --- a/docs/docs/integrations/chat/deepseek.ipynb +++ b/docs/docs/integrations/chat/deepseek.ipynb @@ -38,6 +38,12 @@ "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", "| ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | \n", "\n", + ":::note\n", + "\n", + "DeepSeek-R1, specified via `model=\"deepseek-reasoner\"`, does not support tool calling or structured output. Those features [are supported](https://api-docs.deepseek.com/guides/function_calling) by DeepSeek-V3 (specified via `model=\"deepseek-chat\"`).\n", + "\n", + ":::\n", + "\n", "## Setup\n", "\n", "To access DeepSeek models you'll need to create a/an DeepSeek account, get an API key, and install the `langchain-deepseek` integration package.\n", From 62c570dd77eb23bb289d3a84181d12fe98018bcd Mon Sep 17 00:00:00 2001 From: ccurme Date: Mon, 10 Mar 2025 15:22:24 -0400 Subject: [PATCH 06/14] standard-tests, openai: bump core (#30202) --- libs/partners/openai/pyproject.toml | 2 +- libs/partners/openai/uv.lock | 4 ++-- libs/standard-tests/pyproject.toml | 2 +- libs/standard-tests/uv.lock | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/libs/partners/openai/pyproject.toml b/libs/partners/openai/pyproject.toml index 56453295858..72d2eb4be97 100644 --- a/libs/partners/openai/pyproject.toml +++ b/libs/partners/openai/pyproject.toml @@ -7,7 +7,7 @@ authors = [] license = { text = "MIT" } requires-python = "<4.0,>=3.9" dependencies = [ - "langchain-core<1.0.0,>=0.3.42", + "langchain-core<1.0.0,>=0.3.43", "openai<2.0.0,>=1.58.1", "tiktoken<1,>=0.7", ] diff --git a/libs/partners/openai/uv.lock b/libs/partners/openai/uv.lock index f6011fbe366..3ab88aaa485 100644 --- a/libs/partners/openai/uv.lock +++ b/libs/partners/openai/uv.lock @@ -462,7 +462,7 @@ wheels = [ [[package]] name = "langchain-core" -version = "0.3.42" +version = "0.3.43" source = { editable = "../../core" } dependencies = [ { name = "jsonpatch" }, @@ -603,7 +603,7 @@ typing = [ [[package]] name = "langchain-tests" -version = "0.3.13" +version = "0.3.14" source = { editable = "../../standard-tests" } dependencies = [ { name = "httpx" }, diff --git a/libs/standard-tests/pyproject.toml b/libs/standard-tests/pyproject.toml index 619120e4a23..45bbe98931c 100644 --- a/libs/standard-tests/pyproject.toml +++ b/libs/standard-tests/pyproject.toml @@ -7,7 +7,7 @@ authors = [{ name = "Erick Friis", email = "erick@langchain.dev" }] license = { text = "MIT" } requires-python = "<4.0,>=3.9" dependencies = [ - "langchain-core<1.0.0,>=0.3.42", + "langchain-core<1.0.0,>=0.3.43", "pytest<9,>=7", "pytest-asyncio<1,>=0.20", "httpx<1,>=0.25.0", diff --git a/libs/standard-tests/uv.lock b/libs/standard-tests/uv.lock index 3b73955f8d5..2f9a5d4c765 100644 --- a/libs/standard-tests/uv.lock +++ b/libs/standard-tests/uv.lock @@ -288,7 +288,7 @@ wheels = [ [[package]] name = "langchain-core" -version = "0.3.42" +version = "0.3.43" source = { editable = "../core" } dependencies = [ { name = "jsonpatch" }, From 70fc0b8363a59349f4b78bbc514e386e6abc7179 Mon Sep 17 00:00:00 2001 From: ccurme Date: Mon, 10 Mar 2025 16:18:33 -0400 Subject: [PATCH 07/14] infra: update release workflow (#30203) --- .github/workflows/_release.yml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.github/workflows/_release.yml b/.github/workflows/_release.yml index e72ed2bce79..67e9584a625 100644 --- a/.github/workflows/_release.yml +++ b/.github/workflows/_release.yml @@ -385,6 +385,14 @@ jobs: - test-pypi-publish - pre-release-checks - test-prior-published-packages-against-new-core + if: > + ( + startsWith(inputs.working-directory, 'libs/core') + && needs.test-prior-published-packages-against-new-core.result == 'success' + ) + || ( + !startsWith(inputs.working-directory, 'libs/core') + ) runs-on: ubuntu-latest permissions: # This permission is used for trusted publishing: From 27d86d7bc87d38798a0ab39f0b593ff4f890a83a Mon Sep 17 00:00:00 2001 From: ccurme Date: Mon, 10 Mar 2025 17:53:03 -0400 Subject: [PATCH 08/14] infra: update release workflow (#30207) Fix condition --- .github/workflows/_release.yml | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/.github/workflows/_release.yml b/.github/workflows/_release.yml index 67e9584a625..982d9c30be1 100644 --- a/.github/workflows/_release.yml +++ b/.github/workflows/_release.yml @@ -386,12 +386,13 @@ jobs: - pre-release-checks - test-prior-published-packages-against-new-core if: > - ( - startsWith(inputs.working-directory, 'libs/core') - && needs.test-prior-published-packages-against-new-core.result == 'success' - ) - || ( - !startsWith(inputs.working-directory, 'libs/core') + always() && + needs.build.result == 'success' && + needs.release-notes.result == 'success' && + needs.test-pypi-publish.result == 'success' && + needs.pre-release-checks.result == 'success' && ( + (startsWith(inputs.working-directory, 'libs/core') && needs.test-prior-published-packages-against-new-core.result == 'success') + || (!startsWith(inputs.working-directory, 'libs/core')) ) runs-on: ubuntu-latest permissions: From 81d1653a30df570150cccedaa6e91d2ccc5a0ed3 Mon Sep 17 00:00:00 2001 From: Dharshan A Date: Tue, 11 Mar 2025 19:14:20 +0530 Subject: [PATCH 09/14] docs: Fix typo in Generating Examples section of few-shot prompting doc (#30219) Thank you for contributing to LangChain! - [ ] **PR title**: "package: description" - Where "package" is whichever of langchain, community, core, etc. is being modified. Use "docs: ..." for purely docs changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [ ] **PR message**: ***Delete this entire checklist*** and replace with - **Description:** a description of the change - **Issue:** the issue # it fixes, if applicable - **Dependencies:** any dependencies required for this change - **Twitter handle:** if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [ ] **Add tests and docs**: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, eyurtsev, ccurme, vbarda, hwchase17. --- docs/docs/concepts/few_shot_prompting.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/docs/concepts/few_shot_prompting.mdx b/docs/docs/concepts/few_shot_prompting.mdx index e48d76905dc..ad11df48255 100644 --- a/docs/docs/concepts/few_shot_prompting.mdx +++ b/docs/docs/concepts/few_shot_prompting.mdx @@ -30,7 +30,7 @@ At a high-level, the basic ways to generate examples are: - User feedback: users (or labelers) leave feedback on interactions with the application and examples are generated based on that feedback (for example, all interactions with positive feedback could be turned into examples). - LLM feedback: same as user feedback but the process is automated by having models evaluate themselves. -Which approach is best depends on your task. For tasks where a small number core principles need to be understood really well, it can be valuable hand-craft a few really good examples. +Which approach is best depends on your task. For tasks where a small number of core principles need to be understood really well, it can be valuable hand-craft a few really good examples. For tasks where the space of correct behaviors is broader and more nuanced, it can be useful to generate many examples in a more automated fashion so that there's a higher likelihood of there being some highly relevant examples for any runtime input. **Single-turn v.s. multi-turn examples** @@ -39,8 +39,8 @@ Another dimension to think about when generating examples is what the example is The simplest types of examples just have a user input and an expected model output. These are single-turn examples. -One more complex type if example is where the example is an entire conversation, usually in which a model initially responds incorrectly and a user then tells the model how to correct its answer. -This is called a multi-turn example. Multi-turn examples can be useful for more nuanced tasks where its useful to show common errors and spell out exactly why they're wrong and what should be done instead. +One more complex type of example is where the example is an entire conversation, usually in which a model initially responds incorrectly and a user then tells the model how to correct its answer. +This is called a multi-turn example. Multi-turn examples can be useful for more nuanced tasks where it's useful to show common errors and spell out exactly why they're wrong and what should be done instead. ## 2. Number of examples @@ -77,7 +77,7 @@ If we insert our examples as messages, where each example is represented as a se One area where formatting examples as messages can be tricky is when our example outputs have tool calls. This is because different models have different constraints on what types of message sequences are allowed when any tool calls are generated. - Some models require that any AIMessage with tool calls be immediately followed by ToolMessages for every tool call, - Some models additionally require that any ToolMessages be immediately followed by an AIMessage before the next HumanMessage, -- Some models require that tools are passed in to the model if there are any tool calls / ToolMessages in the chat history. +- Some models require that tools are passed into the model if there are any tool calls / ToolMessages in the chat history. These requirements are model-specific and should be checked for the model you are using. If your model requires ToolMessages after tool calls and/or AIMessages after ToolMessages and your examples only include expected tool calls and not the actual tool outputs, you can try adding dummy ToolMessages / AIMessages to the end of each example with generic contents to satisfy the API constraints. In these cases it's especially worth experimenting with inserting your examples as strings versus messages, as having dummy messages can adversely affect certain models. From c7842730efbeddab76f0d04fb76c658258f7c61e Mon Sep 17 00:00:00 2001 From: Vadym Barda Date: Tue, 11 Mar 2025 18:55:45 -0400 Subject: [PATCH 10/14] core[patch]: support single-node subgraphs and put subgraph nodes under the respective subgraphs (#30234) --- .../langchain_core/runnables/graph_mermaid.py | 88 +++++++++++++------ .../runnables/__snapshots__/test_graph.ambr | 39 +++++--- .../tests/unit_tests/runnables/test_graph.py | 17 ++++ 3 files changed, 105 insertions(+), 39 deletions(-) diff --git a/libs/core/langchain_core/runnables/graph_mermaid.py b/libs/core/langchain_core/runnables/graph_mermaid.py index 4693558cc0d..af4f806c1de 100644 --- a/libs/core/langchain_core/runnables/graph_mermaid.py +++ b/libs/core/langchain_core/runnables/graph_mermaid.py @@ -56,37 +56,50 @@ def draw_mermaid( if with_styles else "graph TD;\n" ) + # Group nodes by subgraph + subgraph_nodes: dict[str, dict[str, Node]] = {} + regular_nodes: dict[str, Node] = {} - if with_styles: - # Node formatting templates - default_class_label = "default" - format_dict = {default_class_label: "{0}({1})"} - if first_node is not None: - format_dict[first_node] = "{0}([{1}]):::first" - if last_node is not None: - format_dict[last_node] = "{0}([{1}]):::last" + for key, node in nodes.items(): + if ":" in key: + # For nodes with colons, add them only to their deepest subgraph level + prefix = ":".join(key.split(":")[:-1]) + subgraph_nodes.setdefault(prefix, {})[key] = node + else: + regular_nodes[key] = node - # Add nodes to the graph - for key, node in nodes.items(): - node_name = node.name.split(":")[-1] + # Node formatting templates + default_class_label = "default" + format_dict = {default_class_label: "{0}({1})"} + if first_node is not None: + format_dict[first_node] = "{0}([{1}]):::first" + if last_node is not None: + format_dict[last_node] = "{0}([{1}]):::last" + + def render_node(key: str, node: Node, indent: str = "\t") -> str: + """Helper function to render a node with consistent formatting.""" + node_name = node.name.split(":")[-1] + label = ( + f"

{node_name}

" + if node_name.startswith(tuple(MARKDOWN_SPECIAL_CHARS)) + and node_name.endswith(tuple(MARKDOWN_SPECIAL_CHARS)) + else node_name + ) + if node.metadata: label = ( - f"

{node_name}

" - if node_name.startswith(tuple(MARKDOWN_SPECIAL_CHARS)) - and node_name.endswith(tuple(MARKDOWN_SPECIAL_CHARS)) - else node_name + f"{label}
" + + "\n".join(f"{k} = {value}" for k, value in node.metadata.items()) + + "" ) - if node.metadata: - label = ( - f"{label}
" - + "\n".join( - f"{key} = {value}" for key, value in node.metadata.items() - ) - + "" - ) - node_label = format_dict.get(key, format_dict[default_class_label]).format( - _escape_node_label(key), label - ) - mermaid_graph += f"\t{node_label}\n" + node_label = format_dict.get(key, format_dict[default_class_label]).format( + _escape_node_label(key), label + ) + return f"{indent}{node_label}\n" + + # Add non-subgraph nodes to the graph + if with_styles: + for key, node in regular_nodes.items(): + mermaid_graph += render_node(key, node) # Group edges by their common prefixes edge_groups: dict[str, list[Edge]] = {} @@ -116,6 +129,11 @@ def draw_mermaid( seen_subgraphs.add(subgraph) mermaid_graph += f"\tsubgraph {subgraph}\n" + # Add nodes that belong to this subgraph + if with_styles and prefix in subgraph_nodes: + for key, node in subgraph_nodes[prefix].items(): + mermaid_graph += render_node(key, node) + for edge in edges: source, target = edge.source, edge.target @@ -156,11 +174,25 @@ def draw_mermaid( # Start with the top-level edges (no common prefix) add_subgraph(edge_groups.get("", []), "") - # Add remaining subgraphs + # Add remaining subgraphs with edges for prefix in edge_groups: if ":" in prefix or prefix == "": continue add_subgraph(edge_groups[prefix], prefix) + seen_subgraphs.add(prefix) + + # Add empty subgraphs (subgraphs with no internal edges) + if with_styles: + for prefix in subgraph_nodes: + if ":" not in prefix and prefix not in seen_subgraphs: + mermaid_graph += f"\tsubgraph {prefix}\n" + + # Add nodes that belong to this subgraph + for key, node in subgraph_nodes[prefix].items(): + mermaid_graph += render_node(key, node) + + mermaid_graph += "\tend\n" + seen_subgraphs.add(prefix) # Add custom styles for nodes if with_styles: diff --git a/libs/core/tests/unit_tests/runnables/__snapshots__/test_graph.ambr b/libs/core/tests/unit_tests/runnables/__snapshots__/test_graph.ambr index 4b4568571b3..2e4a19ce5c2 100644 --- a/libs/core/tests/unit_tests/runnables/__snapshots__/test_graph.ambr +++ b/libs/core/tests/unit_tests/runnables/__snapshots__/test_graph.ambr @@ -5,9 +5,6 @@ graph TD; __start__([

__start__

]):::first parent_1(parent_1) - child_child_1_grandchild_1(grandchild_1) - child_child_1_grandchild_2(grandchild_2
__interrupt = before) - child_child_2(child_2) parent_2(parent_2) __end__([

__end__

]):::last __start__ --> parent_1; @@ -15,8 +12,11 @@ parent_1 --> child_child_1_grandchild_1; parent_2 --> __end__; subgraph child + child_child_2(child_2) child_child_1_grandchild_2 --> child_child_2; subgraph child_1 + child_child_1_grandchild_1(grandchild_1) + child_child_1_grandchild_2(grandchild_2
__interrupt = before) child_child_1_grandchild_1 --> child_child_1_grandchild_2; end end @@ -32,10 +32,6 @@ graph TD; __start__([

__start__

]):::first parent_1(parent_1) - child_child_1_grandchild_1(grandchild_1) - child_child_1_grandchild_1_greatgrandchild(greatgrandchild) - child_child_1_grandchild_2(grandchild_2
__interrupt = before) - child_child_2(child_2) parent_2(parent_2) __end__([

__end__

]):::last __start__ --> parent_1; @@ -43,10 +39,14 @@ parent_1 --> child_child_1_grandchild_1; parent_2 --> __end__; subgraph child + child_child_2(child_2) child_child_1_grandchild_2 --> child_child_2; subgraph child_1 + child_child_1_grandchild_1(grandchild_1) + child_child_1_grandchild_2(grandchild_2
__interrupt = before) child_child_1_grandchild_1_greatgrandchild --> child_child_1_grandchild_2; subgraph grandchild_1 + child_child_1_grandchild_1_greatgrandchild(greatgrandchild) child_child_1_grandchild_1 --> child_child_1_grandchild_1_greatgrandchild; end end @@ -1996,10 +1996,6 @@ graph TD; __start__([

__start__

]):::first outer_1(outer_1) - inner_1_inner_1(inner_1) - inner_1_inner_2(inner_2
__interrupt = before) - inner_2_inner_1(inner_1) - inner_2_inner_2(inner_2) outer_2(outer_2) __end__([

__end__

]):::last __start__ --> outer_1; @@ -2009,9 +2005,13 @@ outer_1 --> inner_2_inner_1; outer_2 --> __end__; subgraph inner_1 + inner_1_inner_1(inner_1) + inner_1_inner_2(inner_2
__interrupt = before) inner_1_inner_1 --> inner_1_inner_2; end subgraph inner_2 + inner_2_inner_1(inner_1) + inner_2_inner_2(inner_2) inner_2_inner_1 --> inner_2_inner_2; end classDef default fill:#f2f0ff,line-height:1.2 @@ -2020,6 +2020,23 @@ ''' # --- +# name: test_single_node_subgraph_mermaid[mermaid] + ''' + %%{init: {'flowchart': {'curve': 'linear'}}}%% + graph TD; + __start__([

__start__

]):::first + __end__([

__end__

]):::last + __start__ --> sub_meow; + sub_meow --> __end__; + subgraph sub + sub_meow(meow) + end + classDef default fill:#f2f0ff,line-height:1.2 + classDef first fill-opacity:0 + classDef last fill:#bfb6fc + + ''' +# --- # name: test_trim dict({ 'edges': list([ diff --git a/libs/core/tests/unit_tests/runnables/test_graph.py b/libs/core/tests/unit_tests/runnables/test_graph.py index c2f7ef9b7dc..6f822c1e7c2 100644 --- a/libs/core/tests/unit_tests/runnables/test_graph.py +++ b/libs/core/tests/unit_tests/runnables/test_graph.py @@ -448,6 +448,23 @@ def test_triple_nested_subgraph_mermaid(snapshot: SnapshotAssertion) -> None: assert graph.draw_mermaid() == snapshot(name="mermaid") +def test_single_node_subgraph_mermaid(snapshot: SnapshotAssertion) -> None: + empty_data = BaseModel + nodes = { + "__start__": Node( + id="__start__", name="__start__", data=empty_data, metadata=None + ), + "sub:meow": Node(id="sub:meow", name="meow", data=empty_data, metadata=None), + "__end__": Node(id="__end__", name="__end__", data=empty_data, metadata=None), + } + edges = [ + Edge(source="__start__", target="sub:meow", data=None, conditional=False), + Edge(source="sub:meow", target="__end__", data=None, conditional=False), + ] + graph = Graph(nodes, edges) + assert graph.draw_mermaid() == snapshot(name="mermaid") + + def test_runnable_get_graph_with_invalid_input_type() -> None: """Test that error isn't raised when getting graph with invalid input type.""" From 23fa70f32891bdf54fc73309549a595568958428 Mon Sep 17 00:00:00 2001 From: Vadym Barda Date: Tue, 11 Mar 2025 18:59:02 -0400 Subject: [PATCH 11/14] core[patch]: release 0.3.44 (#30236) --- libs/core/pyproject.toml | 2 +- libs/core/uv.lock | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/libs/core/pyproject.toml b/libs/core/pyproject.toml index 22053fb4ecf..484bfaa165c 100644 --- a/libs/core/pyproject.toml +++ b/libs/core/pyproject.toml @@ -17,7 +17,7 @@ dependencies = [ "pydantic<3.0.0,>=2.7.4; python_full_version >= \"3.12.4\"", ] name = "langchain-core" -version = "0.3.43" +version = "0.3.44" description = "Building applications with LLMs through composability" readme = "README.md" diff --git a/libs/core/uv.lock b/libs/core/uv.lock index f497a14a5e5..bf9653da199 100644 --- a/libs/core/uv.lock +++ b/libs/core/uv.lock @@ -935,7 +935,7 @@ wheels = [ [[package]] name = "langchain-core" -version = "0.3.43" +version = "0.3.44" source = { editable = "." } dependencies = [ { name = "jsonpatch" }, From 49bdd3b6fe07e1ee29c5f2381ef7a563bef15778 Mon Sep 17 00:00:00 2001 From: Jason Zhang <166434281+jayfish0@users.noreply.github.com> Date: Tue, 11 Mar 2025 18:57:40 -0700 Subject: [PATCH 12/14] docs: Add AgentQL provider doc, tool/toolkit doc and documentloader doc (#30144) - **Description:** Added AgentQL docs for the provider page, tools page and documentloader page - **Twitter handle:** @AgentQL Repo: https://github.com/tinyfish-io/agentql-integrations/tree/main/langchain PyPI: https://pypi.org/project/langchain-agentql/ If no one reviews your PR within a few days, please @-mention one of baskaryan, eyurtsev, ccurme, vbarda, hwchase17. --------- Co-authored-by: Chester Curme --- .../document_loaders/agentql.ipynb | 265 ++++ docs/docs/integrations/providers/agentql.mdx | 35 + docs/docs/integrations/tools/agentql.ipynb | 1077 +++++++++++++++++ docs/scripts/tool_feat_table.py | 5 + docs/src/theme/FeatureTables.js | 7 + libs/packages.yml | 3 + 6 files changed, 1392 insertions(+) create mode 100644 docs/docs/integrations/document_loaders/agentql.ipynb create mode 100644 docs/docs/integrations/providers/agentql.mdx create mode 100644 docs/docs/integrations/tools/agentql.ipynb diff --git a/docs/docs/integrations/document_loaders/agentql.ipynb b/docs/docs/integrations/document_loaders/agentql.ipynb new file mode 100644 index 00000000000..0cdfcc39b81 --- /dev/null +++ b/docs/docs/integrations/document_loaders/agentql.ipynb @@ -0,0 +1,265 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "wkUAAcGZNSJ3" + }, + "source": [ + "# AgentQLLoader\n", + "\n", + "[AgentQL](https://www.agentql.com/)'s document loader provides structured data extraction from any web page using an [AgentQL query](https://docs.agentql.com/agentql-query). AgentQL can be used across multiple languages and web pages without breaking over time and change.\n", + "\n", + "## Overview\n", + "\n", + "`AgentQLLoader` requires the following two parameters:\n", + "- `url`: The URL of the web page you want to extract data from.\n", + "- `query`: The AgentQL query to execute. Learn more about [how to write an AgentQL query in the docs](https://docs.agentql.com/agentql-query) or test one out in the [AgentQL Playground](https://dev.agentql.com/playground).\n", + "\n", + "Setting the following parameters are optional:\n", + "- `api_key`: Your AgentQL API key from [dev.agentql.com](https://dev.agentql.com). **`Optional`.**\n", + "- `timeout`: The number of seconds to wait for a request before timing out. **Defaults to `900`.**\n", + "- `is_stealth_mode_enabled`: Whether to enable experimental anti-bot evasion strategies. This feature may not work for all websites at all times. Data extraction may take longer to complete with this mode enabled. **Defaults to `False`.**\n", + "- `wait_for`: The number of seconds to wait for the page to load before extracting data. **Defaults to `0`.**\n", + "- `is_scroll_to_bottom_enabled`: Whether to scroll to bottom of the page before extracting data. **Defaults to `False`.**\n", + "- `mode`: `\"standard\"` uses deep data analysis, while `\"fast\"` trades some depth of analysis for speed and is adequate for most usecases. [Learn more about the modes in this guide.](https://docs.agentql.com/accuracy/standard-mode) **Defaults to `\"fast\"`.**\n", + "- `is_screenshot_enabled`: Whether to take a screenshot before extracting data. Returned in 'metadata' as a Base64 string. **Defaults to `False`.**\n", + "\n", + "AgentQLLoader is implemented with AgentQL's [REST API](https://docs.agentql.com/rest-api/api-reference)\n", + "\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | JS support |\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| AgentQLLoader| langchain-agentql | ✅ | ❌ | ❌ |\n", + "\n", + "### Loader features\n", + "| Source | Document Lazy Loading | Native Async Support\n", + "| :---: | :---: | :---: |\n", + "| AgentQLLoader | ✅ | ❌ |" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "CaKa2QrnwPXq" + }, + "source": [ + "## Setup\n", + "\n", + "To use the AgentQL Document Loader, you will need to configure the `AGENTQL_API_KEY` environment variable, or use the `api_key` parameter. You can acquire an API key from our [Dev Portal](https://dev.agentql.com)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "mZNJvUQBNSJ5" + }, + "source": [ + "### Installation\n", + "\n", + "Install **langchain-agentql**." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "IblRoJJDNSJ5" + }, + "outputs": [], + "source": [ + "%pip install -qU langchain_agentql" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "SNsUT60YvfCm" + }, + "source": [ + "### Set Credentials" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "id": "2D1EN7Egvk1c" + }, + "outputs": [], + "source": [ + "import os\n", + "\n", + "os.environ[\"AGENTQL_API_KEY\"] = \"YOUR_AGENTQL_API_KEY\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "D4hnJV_6NSJ5" + }, + "source": [ + "## Initialization\n", + "\n", + "Next instantiate your model object:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "id": "oMJdxL_KNSJ5" + }, + "outputs": [], + "source": [ + "from langchain_agentql.document_loaders import AgentQLLoader\n", + "\n", + "loader = AgentQLLoader(\n", + " url=\"https://www.agentql.com/blog\",\n", + " query=\"\"\"\n", + " {\n", + " posts[] {\n", + " title\n", + " url\n", + " date\n", + " author\n", + " }\n", + " }\n", + " \"\"\",\n", + " is_scroll_to_bottom_enabled=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "SRxIOx90NSJ5" + }, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "bNnnCZ1oNSJ5", + "outputId": "d0eb8cb4-9742-4f0c-80f1-0509a3af1808" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'request_id': 'bdb9dbe7-8a7f-427f-bc16-839ccc02cae6', 'generated_query': None, 'screenshot': None}, page_content=\"{'posts': [{'title': 'Launch Week Recap—make the web AI-ready', 'url': 'https://www.agentql.com/blog/2024-launch-week-recap', 'date': 'Nov 18, 2024', 'author': 'Rachel-Lee Nabors'}, {'title': 'Accurate data extraction from PDFs and images with AgentQL', 'url': 'https://www.agentql.com/blog/accurate-data-extraction-pdfs-images', 'date': 'Feb 1, 2025', 'author': 'Rachel-Lee Nabors'}, {'title': 'Introducing Scheduled Scraping Workflows', 'url': 'https://www.agentql.com/blog/scheduling', 'date': 'Dec 2, 2024', 'author': 'Rachel-Lee Nabors'}, {'title': 'Updates to Our Pricing Model', 'url': 'https://www.agentql.com/blog/2024-pricing-update', 'date': 'Nov 19, 2024', 'author': 'Rachel-Lee Nabors'}, {'title': 'Get data from any page: AgentQL’s REST API Endpoint—Launch week day 5', 'url': 'https://www.agentql.com/blog/data-rest-api', 'date': 'Nov 15, 2024', 'author': 'Rachel-Lee Nabors'}]}\")" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "wtPMNh72NSJ5", + "outputId": "59d529a4-3c22-445c-f5cf-dc7b24168906" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'request_id': 'bdb9dbe7-8a7f-427f-bc16-839ccc02cae6', 'generated_query': None, 'screenshot': None}\n" + ] + } + ], + "source": [ + "print(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7RMuEwl4NSJ5" + }, + "source": [ + "## Lazy Load\n", + "\n", + "`AgentQLLoader` currently only loads one `Document` at a time. Therefore, `load()` and `lazy_load()` behave the same:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "FIYddZBONSJ5", + "outputId": "c39a7a6d-bc52-4ef9-b36f-e1d138590b79" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[Document(metadata={'request_id': '06273abd-b2ef-4e15-b0ec-901cba7b4825', 'generated_query': None, 'screenshot': None}, page_content=\"{'posts': [{'title': 'Launch Week Recap—make the web AI-ready', 'url': 'https://www.agentql.com/blog/2024-launch-week-recap', 'date': 'Nov 18, 2024', 'author': 'Rachel-Lee Nabors'}, {'title': 'Accurate data extraction from PDFs and images with AgentQL', 'url': 'https://www.agentql.com/blog/accurate-data-extraction-pdfs-images', 'date': 'Feb 1, 2025', 'author': 'Rachel-Lee Nabors'}, {'title': 'Introducing Scheduled Scraping Workflows', 'url': 'https://www.agentql.com/blog/scheduling', 'date': 'Dec 2, 2024', 'author': 'Rachel-Lee Nabors'}, {'title': 'Updates to Our Pricing Model', 'url': 'https://www.agentql.com/blog/2024-pricing-update', 'date': 'Nov 19, 2024', 'author': 'Rachel-Lee Nabors'}, {'title': 'Get data from any page: AgentQL’s REST API Endpoint—Launch week day 5', 'url': 'https://www.agentql.com/blog/data-rest-api', 'date': 'Nov 15, 2024', 'author': 'Rachel-Lee Nabors'}]}\")]" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pages = [doc for doc in loader.lazy_load()]\n", + "pages" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For more information on how to use this integration, please refer to the [git repo](https://github.com/tinyfish-io/agentql-integrations/tree/main/langchain) or the [langchain integration documentation](https://docs.agentql.com/integrations/langchain)" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/docs/docs/integrations/providers/agentql.mdx b/docs/docs/integrations/providers/agentql.mdx new file mode 100644 index 00000000000..20381e1784e --- /dev/null +++ b/docs/docs/integrations/providers/agentql.mdx @@ -0,0 +1,35 @@ +# AgentQL + +[AgentQL](https://www.agentql.com/) provides web interaction and structured data extraction from any web page using an [AgentQL query](https://docs.agentql.com/agentql-query) or a Natural Language prompt. AgentQL can be used across multiple languages and web pages without breaking over time and change. + +## Installation and Setup + +Install the integration package: + +```bash +pip install langchain-agentql +``` + +## API Key + +Get an API Key from our [Dev Portal](https://dev.agentql.com/) and add it to your environment variables: +``` +export AGENTQL_API_KEY="your-api-key-here" +``` + +## DocumentLoader +AgentQL's document loader provides structured data extraction from any web page using an AgentQL query. + +```python +from langchain_agentql.document_loaders import AgentQLLoader +``` +See our [document loader documentation and usage example](/docs/integrations/document_loaders/agentql). + +## Tools and Toolkits +AgentQL tools provides web interaction and structured data extraction from any web page using an AgentQL query or a Natural Language prompt. + +```python +from langchain_agentql.tools import ExtractWebDataTool, ExtractWebDataBrowserTool, GetWebElementBrowserTool +from langchain_agentql import AgentQLBrowserToolkit +``` +See our [tools documentation and usage example](/docs/integrations/tools/agentql). diff --git a/docs/docs/integrations/tools/agentql.ipynb b/docs/docs/integrations/tools/agentql.ipynb new file mode 100644 index 00000000000..96c3c3797cc --- /dev/null +++ b/docs/docs/integrations/tools/agentql.ipynb @@ -0,0 +1,1077 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a6f91f20", + "metadata": { + "id": "a6f91f20" + }, + "source": [ + "# AgentQL\n", + "\n", + "[AgentQL](https://www.agentql.com/) tools provides web interaction and structured data extraction from any web page using an [AgentQL query](https://docs.agentql.com/agentql-query) or a Natural Language prompt. AgentQL can be used across multiple languages and web pages without breaking over time and change.\n", + "\n", + "## Overview\n", + "\n", + "AgentQL provides the following three tools:\n", + "\n", + "- **`ExtractWebDataTool`** extracts structured data as JSON from a web page given a URL using either an [AgentQL query](https://docs.agentql.com/agentql-query/query-intro) or a Natural Language description of the data.\n", + "\n", + "The following two tools are also bundled as `AgentQLBrowserToolkit` and must be used with a `Playwright` browser or a remote browser instance via Chrome DevTools Protocal (CDP):\n", + "\n", + "- **`ExtractWebDataBrowserTool`** extracts structured data as JSON from the active web page in a browser using either an [AgentQL query](https://docs.agentql.com/agentql-query/query-intro) or a Natural Language description.\n", + "\n", + "- **`GetWebElementBrowserTool`** finds a web element on the active web page in a browser using a Natural Language description and returns its CSS selector for further interaction.\n", + "\n", + "### Integration details\n", + "\n", + "| Class | Package | Serializable | [JS support](https://js.langchain.com/docs/integrations/tools/langchain_agentql) | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| AgentQL | langchain-agentql | ❌ | ❌ | 1.0.0 |\n", + "\n", + "### Tool features\n", + "\n", + "| Tool | Web Data Extraction | Web Element Extraction | Use With Local Browser |\n", + "| :--- | :---: | :---: | :---: |\n", + "| ExtractWebDataTool | ✅ | ❌ | ❌\n", + "| ExtractWebDataBrowserTool | ✅ | ❌ | ✅\n", + "| GetWebElementBrowserTool | ❌ | ✅ | ✅" + ] + }, + { + "cell_type": "markdown", + "id": "e0ec39b2", + "metadata": {}, + "source": [ + "## Setup" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f85b4089", + "metadata": { + "id": "f85b4089" + }, + "outputs": [], + "source": [ + "%pip install --quiet -U langchain_agentql" + ] + }, + { + "cell_type": "markdown", + "id": "uQvMedxGWeUV", + "metadata": { + "id": "uQvMedxGWeUV" + }, + "source": [ + "To run this notebook, install `Playwright` browser and configure Jupyter Notebook's `asyncio` loop." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "OMaAdTLYWRfL", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "OMaAdTLYWRfL", + "outputId": "e1472145-dea3-45b1-b8d1-875772bfdfe1" + }, + "outputs": [], + "source": [ + "!playwright install\n", + "\n", + "# This import is required only for jupyter notebooks, since they have their own eventloop\n", + "import nest_asyncio\n", + "\n", + "nest_asyncio.apply()" + ] + }, + { + "cell_type": "markdown", + "id": "b15e9266", + "metadata": { + "id": "b15e9266" + }, + "source": [ + "### Credentials\n", + "\n", + "To use the AgentQL tools, you will need to get your own API key from the [AgentQL Dev Portal](https://dev.agentql.com/) and set the AgentQL environment variable." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "e0b178a2-8816-40ca-b57c-ccdd86dde9c9", + "metadata": { + "id": "e0b178a2-8816-40ca-b57c-ccdd86dde9c9" + }, + "outputs": [], + "source": [ + "import os\n", + "\n", + "os.environ[\"AGENTQL_API_KEY\"] = \"YOUR_AGENTQL_API_KEY\"" + ] + }, + { + "cell_type": "markdown", + "id": "3nk4k9ED8E9Z", + "metadata": { + "id": "3nk4k9ED8E9Z" + }, + "source": [ + "## Instantiation" + ] + }, + { + "cell_type": "markdown", + "id": "1c97218f-f366-479d-8bf7-fe9f2f6df73f", + "metadata": { + "id": "1c97218f-f366-479d-8bf7-fe9f2f6df73f" + }, + "source": [ + "### `ExtractWebDataTool`\n", + "You can instantiate `ExtractWebDataTool` with the following params:\n", + "- `api_key`: Your AgentQL API key from [dev.agentql.com](https://dev.agentql.com). **`Optional`.**\n", + "- `timeout`: The number of seconds to wait for a request before timing out. Increase if data extraction times out. **Defaults to `900`.**\n", + "- `is_stealth_mode_enabled`: Whether to enable experimental anti-bot evasion strategies. This feature may not work for all websites at all times. Data extraction may take longer to complete with this mode enabled. **Defaults to `False`.**\n", + "- `wait_for`: The number of seconds to wait for the page to load before extracting data. **Defaults to `0`.**\n", + "- `is_scroll_to_bottom_enabled`: Whether to scroll to bottom of the page before extracting data. **Defaults to `False`.**\n", + "- `mode`: `\"standard\"` uses deep data analysis, while `\"fast\"` trades some depth of analysis for speed and is adequate for most usecases. [Learn more about the modes in this guide.](https://docs.agentql.com/accuracy/standard-mode) **Defaults to `\"fast\"`.**\n", + "- `is_screenshot_enabled`: Whether to take a screenshot before extracting data. Returned in 'metadata' as a Base64 string. **Defaults to `False`.**\n", + "\n", + "`ExtractWebDataTool` is implemented with AgentQL's REST API, you can view more details about the parameters in the [API Reference docs](https://docs.agentql.com/rest-api/api-reference)." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "8b3ddfe9-ca79-494c-a7ab-1f56d9407a64", + "metadata": { + "id": "8b3ddfe9-ca79-494c-a7ab-1f56d9407a64" + }, + "outputs": [], + "source": [ + "from langchain_agentql.tools import ExtractWebDataTool\n", + "\n", + "extract_web_data_tool = ExtractWebDataTool()" + ] + }, + { + "cell_type": "markdown", + "id": "sSTERzBmCyi3", + "metadata": { + "id": "sSTERzBmCyi3" + }, + "source": [ + "### `ExtractWebDataBrowserTool`\n", + "\n", + "To instantiate **ExtractWebDataBrowserTool**, you need to connect the tool with a browser instance.\n", + "\n", + "You can set the following params:\n", + "- `timeout`: The number of seconds to wait for a request before timing out. Increase if data extraction times out. **Defaults to `900`.**\n", + "- `wait_for_network_idle`: Whether to wait until the network reaches a full idle state before executing. **Defaults to `True`.**\n", + "- `include_hidden`: Whether to take into account visually hidden elements on the page. **Defaults to `True`.**\n", + "- `mode`: `\"standard\"` uses deep data analysis, while `\"fast\"` trades some depth of analysis for speed and is adequate for most usecases. [Learn more about the modes in this guide.](https://docs.agentql.com/accuracy/standard-mode) **Defaults to `\"fast\"`.**\n", + "\n", + "`ExtractWebDataBrowserTool` is implemented with AgentQL's SDK. You can find more details about the parameters and the functions in AgentQL's [API References](https://docs.agentql.com/python-sdk/api-references/agentql-page#querydata)." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "bnhKlXqHE7Z5", + "metadata": { + "id": "bnhKlXqHE7Z5" + }, + "outputs": [], + "source": [ + "from langchain_agentql.tools import ExtractWebDataBrowserTool\n", + "from langchain_agentql.utils import create_async_playwright_browser\n", + "\n", + "async_browser = await create_async_playwright_browser()\n", + "\n", + "extract_web_data_browser_tool = ExtractWebDataBrowserTool(async_browser=async_browser)" + ] + }, + { + "cell_type": "markdown", + "id": "DJjSQwSaJ4Ml", + "metadata": { + "id": "DJjSQwSaJ4Ml" + }, + "source": [ + "### `GetWebElementBrowserTool`\n", + "\n", + "To instantiate **GetWebElementBrowserTool**, you need to connect the tool with a browser instance.\n", + "\n", + "You can set the following params:\n", + "- `timeout`: The number of seconds to wait for a request before timing out. Increase if data extraction times out. **Defaults to `900`.**\n", + "- `wait_for_network_idle`: Whether to wait until the network reaches a full idle state before executing. **Defaults to `True`.**\n", + "- `include_hidden`: Whether to take into account visually hidden elements on the page. **Defaults to `False`.**\n", + "- `mode`: `\"standard\"` uses deep data analysis, while `\"fast\"` trades some depth of analysis for speed and is adequate for most usecases. [Learn more about the modes in this guide.](https://docs.agentql.com/accuracy/standard-mode) **Defaults to `\"fast\"`.**\n", + "\n", + "`GetWebElementBrowserTool` is implemented with AgentQL's SDK. You can find more details about the parameters and the functions in AgentQL's [API References](https://docs.agentql.com/python-sdk/api-references/agentql-page#queryelements).`" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "503PRMZ1Lay7", + "metadata": { + "id": "503PRMZ1Lay7" + }, + "outputs": [], + "source": [ + "from langchain_agentql.tools import GetWebElementBrowserTool\n", + "\n", + "extract_web_element_tool = GetWebElementBrowserTool(async_browser=async_browser)" + ] + }, + { + "cell_type": "markdown", + "id": "RVRZn9Dy8Q65", + "metadata": { + "id": "RVRZn9Dy8Q65" + }, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "markdown", + "id": "74147a1a", + "metadata": { + "id": "74147a1a" + }, + "source": [ + "### `ExtractWebDataTool`\n", + "\n", + "This tool uses AgentQL's REST API under the hood, sending the publically available web page's URL to AgentQL's endpoint. This will not work with private pages or logged in sessions. Use `ExtractWebDataBrowserTool` for those usecases.\n", + "\n", + "- `url`: The URL of the web page you want to extract data from.\n", + "- `query`: The AgentQL query to execute. Use AgentQL query if you want to extract precisely structured data. Learn more about [how to write an AgentQL query in the docs](https://docs.agentql.com/agentql-query) or test one out in the [AgentQL Playground](https://dev.agentql.com/playground).\n", + "- `prompt`: A Natural Language description of the data to extract from the page. AgentQL will infer the data’s structure from your prompt. Use `prompt` if you want to extract data defined by free-form language without defining a particular structure. \n", + "\n", + "**Note:** You must define either a `query` or a `prompt` to use AgentQL." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "65310a8b-eb0c-4d9e-a618-4f4abe2414fc", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "65310a8b-eb0c-4d9e-a618-4f4abe2414fc", + "outputId": "48996c37-b61e-487f-a618-719f75afc4db" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "{'data': {'posts': [{'title': 'Launch Week Recap—make the web AI-ready',\n", + " 'url': 'https://www.agentql.com/blog/2024-launch-week-recap',\n", + " 'date': 'Nov 18, 2024',\n", + " 'author': 'Rachel-Lee Nabors'},\n", + " {'title': 'Accurate data extraction from PDFs and images with AgentQL',\n", + " 'url': 'https://www.agentql.com/blog/accurate-data-extraction-pdfs-images',\n", + " 'date': 'Feb 1, 2025',\n", + " 'author': 'Rachel-Lee Nabors'},\n", + " {'title': 'Introducing Scheduled Scraping Workflows',\n", + " 'url': 'https://www.agentql.com/blog/scheduling',\n", + " 'date': 'Dec 2, 2024',\n", + " 'author': 'Rachel-Lee Nabors'},\n", + " {'title': 'Updates to Our Pricing Model',\n", + " 'url': 'https://www.agentql.com/blog/2024-pricing-update',\n", + " 'date': 'Nov 19, 2024',\n", + " 'author': 'Rachel-Lee Nabors'},\n", + " {'title': 'Get data from any page: AgentQL’s REST API Endpoint—Launch week day 5',\n", + " 'url': 'https://www.agentql.com/blog/data-rest-api',\n", + " 'date': 'Nov 15, 2024',\n", + " 'author': 'Rachel-Lee Nabors'}]},\n", + " 'metadata': {'request_id': '0dc1f89c-1b6a-46fe-8089-6cd0f082f094',\n", + " 'generated_query': None,\n", + " 'screenshot': None}}" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# You can invoke the tool with either a query or a prompt\n", + "\n", + "# extract_web_data_tool.invoke(\n", + "# {\n", + "# \"url\": \"https://www.agentql.com/blog\",\n", + "# \"prompt\": \"the blog posts with title, url, date of post and author\",\n", + "# }\n", + "# )\n", + "\n", + "extract_web_data_tool.invoke(\n", + " {\n", + " \"url\": \"https://www.agentql.com/blog\",\n", + " \"query\": \"{ posts[] { title url date author } }\",\n", + " },\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "71zmio-iEEDU", + "metadata": { + "id": "71zmio-iEEDU" + }, + "source": [ + "### `ExtractWebDataBrowserTool`\n", + "- `query`: The AgentQL query to execute. Use AgentQL query if you want to extract precisely structured data. Learn more about [how to write an AgentQL query in the docs](https://docs.agentql.com/agentql-query) or test one out in the [AgentQL Playground](https://dev.agentql.com/playground).\n", + "- `prompt`: A Natural Language description of the data to extract from the page. AgentQL will infer the data’s structure from your prompt. Use `prompt` if you want to extract data defined by free-form language without defining a particular structure. \n", + "\n", + "**Note:** You must define either a `query` or a `prompt` to use AgentQL.\n", + "\n", + "To extract data, first you must navigate to a web page using LangChain's [Playwright](https://python.langchain.com/docs/integrations/tools/playwright/) tool." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "Cp9LxO8MaPN1", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "id": "Cp9LxO8MaPN1", + "outputId": "08b98158-e451-428f-f2c0-4c7eec9924cd" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + }, + "text/plain": [ + "'Navigating to https://www.agentql.com/blog returned status code 200'" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_community.tools.playwright import NavigateTool\n", + "\n", + "navigate_tool = NavigateTool(async_browser=async_browser)\n", + "await navigate_tool.ainvoke({\"url\": \"https://www.agentql.com/blog\"})" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "ia5qf2RFEQR9", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ia5qf2RFEQR9", + "outputId": "ee9c1396-82c2-4e94-a79e-730319610033" + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.11/dist-packages/agentql/_core/_utils.py:167: UserWarning: \u001b[31m🚨 The function get_data_by_prompt_experimental is experimental and may not work as expected 🚨\u001b[0m\n", + " warnings.warn(\n" + ] + }, + { + "data": { + "text/plain": [ + "{'blog_posts': [{'title': 'Launch Week Recap—make the web AI-ready',\n", + " 'url': 'https://www.agentql.com/blog/2024-launch-week-recap',\n", + " 'date': 'Nov 18, 2024',\n", + " 'author': 'Rachel-Lee Nabors'},\n", + " {'title': 'Accurate data extraction from PDFs and images with AgentQL',\n", + " 'url': 'https://www.agentql.com/blog/accurate-data-extraction-pdfs-images',\n", + " 'date': 'Feb 1, 2025',\n", + " 'author': 'Rachel-Lee Nabors'},\n", + " {'title': 'Introducing Scheduled Scraping Workflows',\n", + " 'url': 'https://www.agentql.com/blog/scheduling',\n", + " 'date': 'Dec 2, 2024',\n", + " 'author': 'Rachel-Lee Nabors'},\n", + " {'title': 'Updates to Our Pricing Model',\n", + " 'url': 'https://www.agentql.com/blog/2024-pricing-update',\n", + " 'date': 'Nov 19, 2024',\n", + " 'author': 'Rachel-Lee Nabors'},\n", + " {'title': 'Get data from any page: AgentQL’s REST API Endpoint—Launch week day 5',\n", + " 'url': 'https://www.agentql.com/blog/data-rest-api',\n", + " 'date': 'Nov 15, 2024',\n", + " 'author': 'Rachel-Lee Nabors'}]}" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# You can invoke the tool with either a query or a prompt\n", + "\n", + "# await extract_web_data_browser_tool.ainvoke(\n", + "# {'query': '{ blogs[] { title url date author } }'}\n", + "# )\n", + "\n", + "await extract_web_data_browser_tool.ainvoke(\n", + " {\"prompt\": \"the blog posts with title, url, date of post and author\"}\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "xg8rKW6jEXEI", + "metadata": { + "id": "xg8rKW6jEXEI" + }, + "source": [ + "### `GetWebElementBrowserTool`\n", + "- `prompt`: A Natural Language description of the web element to find on the page." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "XgktygByEnas", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "id": "XgktygByEnas", + "outputId": "23091977-4b37-415a-97f9-8ed1154de495" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + }, + "text/plain": [ + "\"[tf623_id='194']\"" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "selector = await extract_web_element_tool.ainvoke({\"prompt\": \"Next page button\"})\n", + "selector" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "O8QJ_NyFfcdh", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "id": "O8QJ_NyFfcdh", + "outputId": "ff0033ca-b230-4fc1-b25f-383f7b5e2ca0" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + }, + "text/plain": [ + "\"Clicked element '[tf623_id='194']'\"" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_community.tools.playwright import ClickTool\n", + "\n", + "# Disabling 'visible_only' will allow us to click on elements that are not visible on the page\n", + "await ClickTool(async_browser=async_browser, visible_only=False).ainvoke(\n", + " {\"selector\": selector}\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "dgsTLAcifoJO", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "id": "dgsTLAcifoJO", + "outputId": "376d0bce-3595-4643-c616-764e4f6bb0ed" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + }, + "text/plain": [ + "'https://www.agentql.com/blog/page/2'" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_community.tools.playwright import CurrentWebPageTool\n", + "\n", + "await CurrentWebPageTool(async_browser=async_browser).ainvoke({})" + ] + }, + { + "cell_type": "markdown", + "id": "ed9fda2c", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "You can use AgentQL tools in a chain by first binding one to a [tool-calling model](/docs/how_to/tool_calling/) and then calling it:\n" + ] + }, + { + "cell_type": "markdown", + "id": "7fd5dc6f", + "metadata": {}, + "source": [ + "### Instantiate LLM" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ef351fb1", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "\n", + "os.environ[\"OPENAI_API_KEY\"] = \"YOUR_OPENAI_API_KEY\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ac535776", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.chat_models import init_chat_model\n", + "\n", + "llm = init_chat_model(model=\"gpt-4o\", model_provider=\"openai\")" + ] + }, + { + "cell_type": "markdown", + "id": "4727f685", + "metadata": {}, + "source": [ + "### Execute Tool Chain" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "434b4678", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'messages': [ToolMessage(content='{\"data\": {\"posts\": [{\"title\": \"Launch Week Recap—make the web AI-ready\", \"url\": \"https://www.agentql.com/blog/2024-launch-week-recap\", \"date\": \"Nov 18, 2024\", \"author\": \"Rachel-Lee Nabors\"}, {\"title\": \"Accurate data extraction from PDFs and images with AgentQL\", \"url\": \"https://www.agentql.com/blog/accurate-data-extraction-pdfs-images\", \"date\": \"Feb 1, 2025\", \"author\": \"Rachel-Lee Nabors\"}, {\"title\": \"Introducing Scheduled Scraping Workflows\", \"url\": \"https://www.agentql.com/blog/scheduling\", \"date\": \"Dec 2, 2024\", \"author\": \"Rachel-Lee Nabors\"}, {\"title\": \"Updates to Our Pricing Model\", \"url\": \"https://www.agentql.com/blog/2024-pricing-update\", \"date\": \"Nov 19, 2024\", \"author\": \"Rachel-Lee Nabors\"}, {\"title\": \"Get data from any page: AgentQL’s REST API Endpoint—Launch week day 5\", \"url\": \"https://www.agentql.com/blog/data-rest-api\", \"date\": \"Nov 15, 2024\", \"author\": \"Rachel-Lee Nabors\"}]}, \"metadata\": {\"request_id\": \"1a84ed12-d02a-497d-b09d-21fe49342fa3\", \"generated_query\": null, \"screenshot\": null}}', name='extract_web_data_with_rest_api', tool_call_id='call_z4Rl1MpjJZNcbLlq1OCneoMF')]}" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "from langchain_core.runnables import RunnableConfig, chain\n", + "\n", + "prompt = ChatPromptTemplate(\n", + " [\n", + " (\"system\", \"You are a helpful assistant in extracting data from website.\"),\n", + " (\"human\", \"{user_input}\"),\n", + " (\"placeholder\", \"{messages}\"),\n", + " ]\n", + ")\n", + "\n", + "# specifying tool_choice will force the model to call this tool.\n", + "llm_with_tools = llm.bind_tools(\n", + " [extract_web_data_tool], tool_choice=\"extract_web_data_with_rest_api\"\n", + ")\n", + "\n", + "llm_chain = prompt | llm_with_tools\n", + "\n", + "\n", + "@chain\n", + "def tool_chain(user_input: str, config: RunnableConfig):\n", + " input_ = {\"user_input\": user_input}\n", + " ai_msg = llm_chain.invoke(input_, config=config)\n", + " tool_msgs = extract_web_data_tool.batch(ai_msg.tool_calls, config=config)\n", + " return {\"messages\": tool_msgs}\n", + "\n", + "\n", + "tool_chain.invoke(\n", + " \"Extract data from https://www.agentql.com/blog using the following agentql query: { posts[] { title url date author } }\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "-SknEjzZXQWg", + "metadata": { + "id": "-SknEjzZXQWg" + }, + "source": [ + "## Use within an agent\n", + "\n", + "You can use AgentQL tools with an AI Agent using the `AgentQLBrowserToolkit` . This toolkit includes `ExtractDataBrowserTool` and `GetWebElementBrowserTool`. Here's an example of agentic browser actions that combine AgentQL's toolkit with the Playwright tools." + ] + }, + { + "cell_type": "markdown", + "id": "VLzyKpfAmvv7", + "metadata": { + "id": "VLzyKpfAmvv7" + }, + "source": [ + "### Instantiate Toolkit\n" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "UGsFDL1atP3y", + "metadata": { + "id": "UGsFDL1atP3y" + }, + "outputs": [], + "source": [ + "from langchain_agentql.utils import create_async_playwright_browser\n", + "\n", + "async_agent_browser = await create_async_playwright_browser()" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "nCWN9X118rtF", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "nCWN9X118rtF", + "outputId": "0ef8160e-f2a3-4ad5-f53d-8cb0f0d71367" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[ExtractWebDataBrowserTool(async_browser= version=133.0.6943.16>),\n", + " GetWebElementBrowserTool(async_browser= version=133.0.6943.16>)]" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_agentql import AgentQLBrowserToolkit\n", + "\n", + "agentql_toolkit = AgentQLBrowserToolkit(async_browser=async_agent_browser)\n", + "agentql_toolkit.get_tools()" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "rBYb-I6Tp56C", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "rBYb-I6Tp56C", + "outputId": "b1ccaa79-425b-4137-cd4d-bffbc32fc395" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[NavigateTool(async_browser= version=133.0.6943.16>),\n", + " ClickTool(async_browser= version=133.0.6943.16>, visible_only=False)]" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_community.tools.playwright import ClickTool, NavigateTool\n", + "\n", + "# we hand pick the following tools to allow more precise agentic browser actions\n", + "playwright_toolkit = [\n", + " NavigateTool(async_browser=async_agent_browser),\n", + " ClickTool(async_browser=async_agent_browser, visible_only=False),\n", + "]\n", + "playwright_toolkit" + ] + }, + { + "cell_type": "markdown", + "id": "5c_KiBCHqTjv", + "metadata": { + "id": "5c_KiBCHqTjv" + }, + "source": [ + "### Use with a ReAct Agent\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "HaAPyYr7quau", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "HaAPyYr7quau", + "outputId": "2de2496c-06eb-47db-91d3-f2171caf0640" + }, + "outputs": [], + "source": [ + "%pip install --quiet -U langgraph" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "ShcjQ352qRS_", + "metadata": { + "id": "ShcjQ352qRS_" + }, + "outputs": [], + "source": [ + "from langgraph.prebuilt import create_react_agent\n", + "\n", + "# You need to set up an llm, please refer to the chaining section\n", + "agent_executor = create_react_agent(\n", + " llm, agentql_toolkit.get_tools() + playwright_toolkit\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "YEdeprRCq_7E", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "YEdeprRCq_7E", + "outputId": "f834b9f3-802d-49d6-c5c3-06c86d6d82e8" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "================================\u001b[1m Human Message \u001b[0m=================================\n", + "\n", + "\n", + "Navigate to https://news.ycombinator.com/,\n", + "extract the news titles on the current page,\n", + "show the current page url,\n", + "find the button on the webpage that direct to the next page,\n", + "click on the button,\n", + "show the current page url,\n", + "extract the news title on the current page\n", + "extract the news titles that mention \"AI\" from the two pages.\n", + "\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " navigate_browser (call_3eY5a0BRwyYj7kaNpAxkquTD)\n", + " Call ID: call_3eY5a0BRwyYj7kaNpAxkquTD\n", + " Args:\n", + " url: https://news.ycombinator.com/\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: navigate_browser\n", + "\n", + "Navigating to https://news.ycombinator.com/ returned status code 200\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " extract_web_data_from_browser (call_WvRrZKGGo8mq3JewRlaIS5xx)\n", + " Call ID: call_WvRrZKGGo8mq3JewRlaIS5xx\n", + " Args:\n", + " prompt: Extract all the news titles from this page.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.11/dist-packages/agentql/_core/_utils.py:167: UserWarning: \u001b[31m🚨 The function get_data_by_prompt_experimental is experimental and may not work as expected 🚨\u001b[0m\n", + " warnings.warn(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: extract_web_data_from_browser\n", + "\n", + "{\"news_item\": [{\"title\": \"I Went to SQL Injection Court\"}, {\"title\": \"Framework's first desktop is a strange–but unique–mini ITX gaming PC\"}, {\"title\": \"Hyperspace\"}, {\"title\": \"The XB-70 (2019)\"}, {\"title\": \"How core Git developers configure Git\"}, {\"title\": \"Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs [pdf]\"}, {\"title\": \"Hard problems that reduce to document ranking\"}, {\"title\": \"Ggwave: Tiny Data-over-Sound Library\"}, {\"title\": \"Bald eagles are thriving again after near extinction\"}, {\"title\": \"Forum with 2.6M posts being deleted due to UK Online Safety Act\"}, {\"title\": \"Launch HN: Browser Use (YC W25) – open-source web agents\"}, {\"title\": \"Part two of Grant Sanderson's video with Terry Tao on the cosmic distance ladder\"}, {\"title\": \"New maps of the chaotic space-time inside black holes\"}, {\"title\": \"Knitting Your Parachute\"}, {\"title\": \"Chicory: A JVM native WebAssembly runtime\"}, {\"title\": \"Low Overhead Allocation Sampling with VMProf in PyPy's GC\"}, {\"title\": \"Sigma BF Camera\"}, {\"title\": \"DeepSearcher: A local open-source Deep Research\"}, {\"title\": \"Xonsh – A Python-powered shell\"}, {\"title\": \"A possible future of Python in the browser\"}, {\"title\": \"Show HN: GoatDB – A lightweight, offline-first, realtime NoDB for Deno and React\"}, {\"title\": \"Embedding Python in Elixir, it's fine\"}, {\"title\": \"The Deep Research problem\"}, {\"title\": \"Why are QR Codes with capital letters smaller than QR codes with lower case?\"}, {\"title\": \"Show HN: My new wiki for Silicon Graphics stuff\"}, {\"title\": \"AI is blurring the line between PMs and engineers?\"}, {\"title\": \"I recreated Shazam's algorithm with Go [video]\"}, {\"title\": \"Dogs may have domesticated themselves because they liked snacks, model suggests\"}, {\"title\": \"Show HN: Txtl – Fast static website of text utilities\"}, {\"title\": \"Have we been wrong about why Mars is red?\"}]}\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " get_web_element_from_browser (call_B6jn5ItasceNW7eeb640UhQQ)\n", + " Call ID: call_B6jn5ItasceNW7eeb640UhQQ\n", + " Args:\n", + " prompt: button or link to go to the next page\n", + " extract_web_data_from_browser (call_Wyh2VH76bzrlDozp7gpkVBl7)\n", + " Call ID: call_Wyh2VH76bzrlDozp7gpkVBl7\n", + " Args:\n", + " prompt: Extract the current page URL\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.11/dist-packages/agentql/_core/_utils.py:167: UserWarning: \u001b[31m🚨 The function get_data_by_prompt_experimental is experimental and may not work as expected 🚨\u001b[0m\n", + " warnings.warn(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: extract_web_data_from_browser\n", + "\n", + "{\"current_page_url\": \"https://news.ycombinator.com/news\"}\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " click_element (call_NLGIW1lLutkZ6k0vqkfGbOD7)\n", + " Call ID: call_NLGIW1lLutkZ6k0vqkfGbOD7\n", + " Args:\n", + " selector: [tf623_id='944']\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: click_element\n", + "\n", + "Clicked element '[tf623_id='944']'\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " extract_web_data_from_browser (call_QPt8R2hqiSgytUvLcWUUORKF)\n", + " Call ID: call_QPt8R2hqiSgytUvLcWUUORKF\n", + " Args:\n", + " prompt: Extract the current page URL\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.11/dist-packages/agentql/_core/_utils.py:167: UserWarning: \u001b[31m🚨 The function get_data_by_prompt_experimental is experimental and may not work as expected 🚨\u001b[0m\n", + " warnings.warn(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: extract_web_data_from_browser\n", + "\n", + "{\"current_page_url\": \"https://news.ycombinator.com/news?p=2\"}\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " extract_web_data_from_browser (call_ZZOPrIfVaVQ1A26j8EGE913W)\n", + " Call ID: call_ZZOPrIfVaVQ1A26j8EGE913W\n", + " Args:\n", + " prompt: Extract all the news titles from this page.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.11/dist-packages/agentql/_core/_utils.py:167: UserWarning: \u001b[31m🚨 The function get_data_by_prompt_experimental is experimental and may not work as expected 🚨\u001b[0m\n", + " warnings.warn(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: extract_web_data_from_browser\n", + "\n", + "{\"news_item\": [{\"title\": \"'Hey Number 17 '\"}, {\"title\": \"Building and operating a pretty big storage system called S3 (2023)\"}, {\"title\": \"Ghost House – software for automatic inbetweens\"}, {\"title\": \"Ask HN: Former devs who can't get a job, what did you end up doing for work?\"}, {\"title\": \"DeepSeek open source DeepEP – library for MoE training and Inference\"}, {\"title\": \"SETI's hard steps and how to resolve them\"}, {\"title\": \"A Defense of Weird Research\"}, {\"title\": \"DigiCert: Threat of legal action to stifle Bugzilla discourse\"}, {\"title\": \"Show HN: Tach – Visualize and untangle your Python codebase\"}, {\"title\": \"Ask HN: A retrofitted C dialect?\"}, {\"title\": \"“The closer to the train station, the worse the kebab” – a “study”\"}, {\"title\": \"Brewing Clean Water: The metal-remediating benefits of tea preparation\"}, {\"title\": \"Invoker Commands (Explainer)\"}, {\"title\": \"Freelancing: How I found clients, part 1\"}, {\"title\": \"Claude 3.7 Sonnet and Claude Code\"}, {\"title\": \"Clean Code vs. A Philosophy Of Software Design\"}, {\"title\": \"Show HN: While the world builds AI Agents, I'm just building calculators\"}, {\"title\": \"History of CAD\"}, {\"title\": \"Fans are better than tech at organizing information online (2019)\"}, {\"title\": \"Some Programming Language Ideas\"}, {\"title\": \"The independent researcher (2018)\"}, {\"title\": \"The best way to use text embeddings portably is with Parquet and Polars\"}, {\"title\": \"Show HN: Prioritize Anything with Stacks\"}, {\"title\": \"Ashby (YC W19) Is Hiring Principal Product Engineers\"}, {\"title\": \"GibberLink [AI-AI Communication]\"}, {\"title\": \"Show HN: I made a site to tell the time in corporate\"}, {\"title\": \"It’s still worth blogging in the age of AI\"}, {\"title\": \"What would happen if we didn't use TCP or UDP?\"}, {\"title\": \"Closing the “green gap”: energy savings from the math of the landscape function\"}, {\"title\": \"Larry Ellison's half-billion-dollar quest to change farming\"}]}\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "Here's a summary of the actions and results:\n", + "\n", + "### Page 1\n", + "- **URL:** [https://news.ycombinator.com/news](https://news.ycombinator.com/news)\n", + "- **News Titles:**\n", + " 1. I Went to SQL Injection Court\n", + " 2. Framework's first desktop is a strange–but unique–mini ITX gaming PC\n", + " 3. Hyperspace\n", + " 4. The XB-70 (2019)\n", + " 5. How core Git developers configure Git\n", + " 6. Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs [pdf]\n", + " 7. Hard problems that reduce to document ranking\n", + " 8. Ggwave: Tiny Data-over-Sound Library\n", + " 9. Bald eagles are thriving again after near extinction\n", + " 10. Forum with 2.6M posts being deleted due to UK Online Safety Act\n", + " 11. Launch HN: Browser Use (YC W25) – open-source web agents\n", + " 12. Part two of Grant Sanderson's video with Terry Tao on the cosmic distance ladder\n", + " 13. New maps of the chaotic space-time inside black holes\n", + " 14. Knitting Your Parachute\n", + " 15. Chicory: A JVM native WebAssembly runtime\n", + " 16. Low Overhead Allocation Sampling with VMProf in PyPy's GC\n", + " 17. Sigma BF Camera\n", + " 18. DeepSearcher: A local open-source Deep Research\n", + " 19. Xonsh – A Python-powered shell\n", + " 20. A possible future of Python in the browser\n", + " 21. Show HN: GoatDB – A lightweight, offline-first, realtime NoDB for Deno and React\n", + " 22. Embedding Python in Elixir, it's fine\n", + " 23. The Deep Research problem\n", + " 24. Why are QR Codes with capital letters smaller than QR codes with lower case?\n", + " 25. Show HN: My new wiki for Silicon Graphics stuff\n", + " 26. **AI is blurring the line between PMs and engineers?**\n", + " 27. I recreated Shazam's algorithm with Go [video]\n", + " 28. Dogs may have domesticated themselves because they liked snacks, model suggests\n", + " 29. Show HN: Txtl – Fast static website of text utilities\n", + " 30. Have we been wrong about why Mars is red?\n", + "\n", + "### Page 2\n", + "- **URL:** [https://news.ycombinator.com/news?p=2](https://news.ycombinator.com/news?p=2)\n", + "- **News Titles:**\n", + " 1. 'Hey Number 17'\n", + " 2. Building and operating a pretty big storage system called S3 (2023)\n", + " 3. Ghost House – software for automatic inbetweens\n", + " 4. Ask HN: Former devs who can't get a job, what did you end up doing for work?\n", + " 5. DeepSeek open source DeepEP – library for MoE training and Inference\n", + " 6. SETI's hard steps and how to resolve them\n", + " 7. A Defense of Weird Research\n", + " 8. DigiCert: Threat of legal action to stifle Bugzilla discourse\n", + " 9. Show HN: Tach – Visualize and untangle your Python codebase\n", + " 10. Ask HN: A retrofitted C dialect?\n", + " 11. “The closer to the train station, the worse the kebab” – a “study”\n", + " 12. Brewing Clean Water: The metal-remediating benefits of tea preparation\n", + " 13. Invoker Commands (Explainer)\n", + " 14. Freelancing: How I found clients, part 1\n", + " 15. Claude 3.7 Sonnet and Claude Code\n", + " 16. Clean Code vs. A Philosophy Of Software Design\n", + " 17. **Show HN: While the world builds AI Agents, I'm just building calculators**\n", + " 18. History of CAD\n", + " 19. Fans are better than tech at organizing information online (2019)\n", + " 20. Some Programming Language Ideas\n", + " 21. The independent researcher (2018)\n", + " 22. The best way to use text embeddings portably is with Parquet and Polars\n", + " 23. Show HN: Prioritize Anything with Stacks\n", + " 24. Ashby (YC W19) Is Hiring Principal Product Engineers\n", + " 25. **GibberLink [AI-AI Communication]**\n", + " 26. Show HN: I made a site to tell the time in corporate\n", + " 27. **It’s still worth blogging in the age of AI**\n", + " 28. What would happen if we didn't use TCP or UDP?\n", + " 29. Closing the “green gap”: energy savings from the math of the landscape function\n", + " 30. Larry Ellison's half-billion-dollar quest to change farming\n", + "\n", + "### News Titles Mentioning \"AI\":\n", + "1. Page 1: **AI is blurring the line between PMs and engineers?**\n", + "2. Page 2:\n", + " - **Show HN: While the world builds AI Agents, I'm just building calculators**\n", + " - **GibberLink [AI-AI Communication]**\n", + " - **It’s still worth blogging in the age of AI**\n" + ] + } + ], + "source": [ + "prompt = \"\"\"\n", + "Navigate to https://news.ycombinator.com/,\n", + "extract the news titles on the current page,\n", + "show the current page url,\n", + "find the button on the webpage that direct to the next page,\n", + "click on the button,\n", + "show the current page url,\n", + "extract the news title on the current page\n", + "extract the news titles that mention \"AI\" from the two pages.\n", + "\"\"\"\n", + "\n", + "events = agent_executor.astream(\n", + " {\"messages\": [(\"user\", prompt)]},\n", + " stream_mode=\"values\",\n", + ")\n", + "async for event in events:\n", + " event[\"messages\"][-1].pretty_print()" + ] + }, + { + "cell_type": "markdown", + "id": "9b3660a4", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For more information on how to use this integration, please refer to the [git repo](https://github.com/tinyfish-io/agentql-integrations/tree/main/langchain) or the [langchain integration documentation](https://docs.agentql.com/integrations/langchain)" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.1" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/scripts/tool_feat_table.py b/docs/scripts/tool_feat_table.py index c92d2344a93..5e5a3498413 100644 --- a/docs/scripts/tool_feat_table.py +++ b/docs/scripts/tool_feat_table.py @@ -147,6 +147,11 @@ WEBBROWSING_TOOL_FEAT_TABLE = { "interactions": True, "pricing": "40 free requests/day", }, + "AgentQL Toolkit": { + "link": "/docs/integrations/tools/agentql", + "interactions": True, + "pricing": "Free trial, with pay-as-you-go and flat rate plans after", + }, } DATABASE_TOOL_FEAT_TABLE = { diff --git a/docs/src/theme/FeatureTables.js b/docs/src/theme/FeatureTables.js index 73e9bfaa090..2703760561c 100644 --- a/docs/src/theme/FeatureTables.js +++ b/docs/src/theme/FeatureTables.js @@ -819,6 +819,13 @@ const FEATURE_TABLES = { source: "Platform for running and scaling headless browsers, can be used to scrape/crawl any site", api: "API", apiLink: "https://python.langchain.com/docs/integrations/document_loaders/hyperbrowser/" + }, + { + name: "AgentQL", + link: "agentql", + source: "Web interaction and structured data extraction from any web page using an AgentQL query or a Natural Language prompt", + api: "API", + apiLink: "https://python.langchain.com/docs/integrations/document_loaders/agentql/" } ] }, diff --git a/libs/packages.yml b/libs/packages.yml index a8eea203610..8b807f75a6a 100644 --- a/libs/packages.yml +++ b/libs/packages.yml @@ -513,3 +513,6 @@ packages: - name: langchain-opengradient path: . repo: OpenGradient/og-langchain +- name: langchain-agentql + path: langchain + repo: tinyfish-io/agentql-integrations From cd1ea8e94d52d1f054877aa930fc859c47248a7d Mon Sep 17 00:00:00 2001 From: ccurme Date: Wed, 12 Mar 2025 12:25:46 -0400 Subject: [PATCH 13/14] openai[patch]: support Responses API (#30231) Co-authored-by: Bagatur --- .github/workflows/_release.yml | 38 +- docs/docs/integrations/chat/openai.ipynb | 401 +++++++++- libs/core/langchain_core/messages/ai.py | 7 +- .../langchain_core/utils/function_calling.py | 14 +- libs/core/pyproject.toml | 2 +- .../tests/unit_tests/chat_models/test_base.py | 1 + .../langchain_openai/chat_models/base.py | 582 +++++++++++++- libs/partners/openai/pyproject.toml | 6 +- .../chat_models/test_responses_api.py | 168 ++++ .../tests/unit_tests/chat_models/test_base.py | 751 +++++++++++++++++- libs/partners/openai/uv.lock | 12 +- uv.lock | 25 +- 12 files changed, 1933 insertions(+), 74 deletions(-) create mode 100644 libs/partners/openai/tests/integration_tests/chat_models/test_responses_api.py diff --git a/.github/workflows/_release.yml b/.github/workflows/_release.yml index 982d9c30be1..d02248d8006 100644 --- a/.github/workflows/_release.yml +++ b/.github/workflows/_release.yml @@ -100,15 +100,32 @@ jobs: PKG_NAME: ${{ needs.build.outputs.pkg-name }} VERSION: ${{ needs.build.outputs.version }} run: | - PREV_TAG="$PKG_NAME==${VERSION%.*}.$(( ${VERSION##*.} - 1 ))"; [[ "${VERSION##*.}" -eq 0 ]] && PREV_TAG="" + # Handle regular versions and pre-release versions differently + if [[ "$VERSION" == *"-"* ]]; then + # This is a pre-release version (contains a hyphen) + # Extract the base version without the pre-release suffix + BASE_VERSION=${VERSION%%-*} + # Look for the latest release of the same base version + REGEX="^$PKG_NAME==$BASE_VERSION\$" + PREV_TAG=$(git tag --sort=-creatordate | (grep -P "$REGEX" || true) | head -1) + + # If no exact base version match, look for the latest release of any kind + if [ -z "$PREV_TAG" ]; then + REGEX="^$PKG_NAME==\\d+\\.\\d+\\.\\d+\$" + PREV_TAG=$(git tag --sort=-creatordate | (grep -P "$REGEX" || true) | head -1) + fi + else + # Regular version handling + PREV_TAG="$PKG_NAME==${VERSION%.*}.$(( ${VERSION##*.} - 1 ))"; [[ "${VERSION##*.}" -eq 0 ]] && PREV_TAG="" - # backup case if releasing e.g. 0.3.0, looks up last release - # note if last release (chronologically) was e.g. 0.1.47 it will get - # that instead of the last 0.2 release - if [ -z "$PREV_TAG" ]; then - REGEX="^$PKG_NAME==\\d+\\.\\d+\\.\\d+\$" - echo $REGEX - PREV_TAG=$(git tag --sort=-creatordate | (grep -P $REGEX || true) | head -1) + # backup case if releasing e.g. 0.3.0, looks up last release + # note if last release (chronologically) was e.g. 0.1.47 it will get + # that instead of the last 0.2 release + if [ -z "$PREV_TAG" ]; then + REGEX="^$PKG_NAME==\\d+\\.\\d+\\.\\d+\$" + echo $REGEX + PREV_TAG=$(git tag --sort=-creatordate | (grep -P $REGEX || true) | head -1) + fi fi # if PREV_TAG is empty, let it be empty @@ -363,10 +380,9 @@ jobs: # Shallow-fetch just that single tag git fetch --depth=1 origin tag "$LATEST_PACKAGE_TAG" - # Navigate to the partner directory - cd $GITHUB_WORKSPACE/libs/partners/${{ matrix.partner }} - # Checkout the latest package files + rm -rf $GITHUB_WORKSPACE/libs/partners/${{ matrix.partner }}/* + cd $GITHUB_WORKSPACE/libs/partners/${{ matrix.partner }} git checkout "$LATEST_PACKAGE_TAG" -- . # Print as a sanity check diff --git a/docs/docs/integrations/chat/openai.ipynb b/docs/docs/integrations/chat/openai.ipynb index 35d8bf295e9..f8300a34f2b 100644 --- a/docs/docs/integrations/chat/openai.ipynb +++ b/docs/docs/integrations/chat/openai.ipynb @@ -322,7 +322,7 @@ "source": [ "### ``strict=True``\n", "\n", - ":::info Requires ``langchain-openai>=0.1.21rc1``\n", + ":::info Requires ``langchain-openai>=0.1.21``\n", "\n", ":::\n", "\n", @@ -397,6 +397,405 @@ "For more on binding tools and tool call outputs, head to the [tool calling](/docs/how_to/function_calling) docs." ] }, + { + "cell_type": "markdown", + "id": "84833dd0-17e9-4269-82ed-550639d65751", + "metadata": {}, + "source": [ + "## Responses API\n", + "\n", + ":::info Requires ``langchain-openai>=0.3.9-rc.1``\n", + "\n", + ":::\n", + "\n", + "OpenAI supports a [Responses](https://platform.openai.com/docs/guides/responses-vs-chat-completions) API that is oriented toward building [agentic](/docs/concepts/agents/) applications. It includes a suite of [built-in tools](https://platform.openai.com/docs/guides/tools?api-mode=responses), including web and file search. It also supports management of [conversation state](https://platform.openai.com/docs/guides/conversation-state?api-mode=responses), allowing you to continue a conversational thread without explicitly passing in previous messages.\n", + "\n", + "`ChatOpenAI` will route to the Responses API if one of these features is used. You can also specify `use_responses_api=True` when instantiating `ChatOpenAI`.\n", + "\n", + "### Built-in tools\n", + "\n", + "Equipping `ChatOpenAI` with built-in tools will ground its responses with outside information, such as via context in files or the web. The [AIMessage](/docs/concepts/messages/#aimessage) generated from the model will include information about the built-in tool invocation.\n", + "\n", + "#### Web search\n", + "\n", + "To trigger a web search, pass `{\"type\": \"web_search_preview\"}` to the model as you would another tool.\n", + "\n", + ":::tip\n", + "\n", + "You can also pass built-in tools as invocation params:\n", + "```python\n", + "llm.invoke(\"...\", tools=[{\"type\": \"web_search_preview\"}])\n", + "```\n", + "\n", + ":::" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "0d8bfe89-948b-42d4-beac-85ef2a72491d", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_openai import ChatOpenAI\n", + "\n", + "llm = ChatOpenAI(model=\"gpt-4o-mini\")\n", + "\n", + "tool = {\"type\": \"web_search_preview\"}\n", + "llm_with_tools = llm.bind_tools([tool])\n", + "\n", + "response = llm_with_tools.invoke(\"What was a positive news story from today?\")" + ] + }, + { + "cell_type": "markdown", + "id": "c9fe67c6-38ff-40a5-93b3-a4b7fca76372", + "metadata": {}, + "source": [ + "Note that the response includes structured [content blocks](/docs/concepts/messages/#content-1) that include both the text of the response and OpenAI [annotations](https://platform.openai.com/docs/guides/tools-web-search?api-mode=responses#output-and-citations) citing its sources:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "3ea5a4b1-f57a-4c8a-97f4-60ab8330a804", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'type': 'text',\n", + " 'text': 'Today, a heartwarming story emerged from Minnesota, where a group of high school robotics students built a custom motorized wheelchair for a 2-year-old boy named Cillian Jackson. Born with a genetic condition that limited his mobility, Cillian\\'s family couldn\\'t afford the $20,000 wheelchair he needed. The students at Farmington High School\\'s Rogue Robotics team took it upon themselves to modify a Power Wheels toy car into a functional motorized wheelchair for Cillian, complete with a joystick, safety bumpers, and a harness. One team member remarked, \"I think we won here more than we do in our competitions. Instead of completing a task, we\\'re helping change someone\\'s life.\" ([boredpanda.com](https://www.boredpanda.com/wholesome-global-positive-news/?utm_source=openai))\\n\\nThis act of kindness highlights the profound impact that community support and innovation can have on individuals facing challenges. ',\n", + " 'annotations': [{'end_index': 778,\n", + " 'start_index': 682,\n", + " 'title': '“Global Positive News”: 40 Posts To Remind Us There’s Good In The World',\n", + " 'type': 'url_citation',\n", + " 'url': 'https://www.boredpanda.com/wholesome-global-positive-news/?utm_source=openai'}]}]" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response.content" + ] + }, + { + "cell_type": "markdown", + "id": "95fbc34c-2f12-4d51-92c5-bf62a2f8900c", + "metadata": {}, + "source": [ + ":::tip\n", + "\n", + "You can recover just the text content of the response as a string by using `response.text()`. For example, to stream response text:\n", + "\n", + "```python\n", + "for token in llm_with_tools.stream(\"...\"):\n", + " print(token.text(), end=\"|\")\n", + "```\n", + "\n", + "See the [streaming guide](/docs/how_to/chat_streaming/) for more detail.\n", + "\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "id": "2a332940-d409-41ee-ac36-2e9bee900e83", + "metadata": {}, + "source": [ + "The output message will also contain information from any tool invocations:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "a8011049-6c90-4fcb-82d4-850c72b46941", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'tool_outputs': [{'id': 'ws_67d192aeb6cc81918e736ad4a57937570d6f8507990d9d71',\n", + " 'status': 'completed',\n", + " 'type': 'web_search_call'}]}" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response.additional_kwargs" + ] + }, + { + "cell_type": "markdown", + "id": "288d47bb-3ccb-412f-a3d3-9f6cee0e6214", + "metadata": {}, + "source": [ + "#### File search\n", + "\n", + "To trigger a file search, pass a [file search tool](https://platform.openai.com/docs/guides/tools-file-search) to the model as you would another tool. You will need to populate an OpenAI-managed vector store and include the vector store ID in the tool definition. See [OpenAI documentation](https://platform.openai.com/docs/guides/tools-file-search) for more detail." + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "1f758726-33ef-4c04-8a54-49adb783bbb3", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Deep Research by OpenAI is a new capability integrated into ChatGPT that allows for the execution of multi-step research tasks independently. It can synthesize extensive amounts of online information and produce comprehensive reports similar to what a research analyst would do, significantly speeding up processes that would typically take hours for a human.\n", + "\n", + "### Key Features:\n", + "- **Independent Research**: Users simply provide a prompt, and the model can find, analyze, and synthesize information from hundreds of online sources.\n", + "- **Multi-Modal Capabilities**: The model is also able to browse user-uploaded files, plot graphs using Python, and embed visualizations in its outputs.\n", + "- **Training**: Deep Research has been trained using reinforcement learning on real-world tasks that require extensive browsing and reasoning.\n", + "\n", + "### Applications:\n", + "- Useful for professionals in sectors like finance, science, policy, and engineering, enabling them to obtain accurate and thorough research quickly.\n", + "- It can also be beneficial for consumers seeking personalized recommendations on complex purchases.\n", + "\n", + "### Limitations:\n", + "Although Deep Research presents significant advancements, it has some limitations, such as the potential to hallucinate facts or struggle with authoritative information. \n", + "\n", + "Deep Research aims to facilitate access to thorough and documented information, marking a significant step toward the broader goal of developing artificial general intelligence (AGI).\n" + ] + } + ], + "source": [ + "llm = ChatOpenAI(model=\"gpt-4o-mini\")\n", + "\n", + "openai_vector_store_ids = [\n", + " \"vs_...\", # your IDs here\n", + "]\n", + "\n", + "tool = {\n", + " \"type\": \"file_search\",\n", + " \"vector_store_ids\": openai_vector_store_ids,\n", + "}\n", + "llm_with_tools = llm.bind_tools([tool])\n", + "\n", + "response = llm_with_tools.invoke(\"What is deep research by OpenAI?\")\n", + "print(response.text())" + ] + }, + { + "cell_type": "markdown", + "id": "f88bbd71-83b0-45a6-9141-46ec9da93df6", + "metadata": {}, + "source": [ + "As with [web search](#web-search), the response will include content blocks with citations:" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "865bc14e-1599-438e-be44-857891004979", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'file_id': 'file-3UzgX7jcC8Dt9ZAFzywg5k',\n", + " 'index': 346,\n", + " 'type': 'file_citation',\n", + " 'filename': 'deep_research_blog.pdf'},\n", + " {'file_id': 'file-3UzgX7jcC8Dt9ZAFzywg5k',\n", + " 'index': 575,\n", + " 'type': 'file_citation',\n", + " 'filename': 'deep_research_blog.pdf'}]" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response.content[0][\"annotations\"][:2]" + ] + }, + { + "cell_type": "markdown", + "id": "dd00f6be-2862-4634-a0c3-14ee39915c90", + "metadata": {}, + "source": [ + "It will also include information from the built-in tool invocations:" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "e16a7110-d2d8-45fa-b372-5109f330540b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'tool_outputs': [{'id': 'fs_67d196fbb83c8191ba20586175331687089228ce932eceb1',\n", + " 'queries': ['What is deep research by OpenAI?'],\n", + " 'status': 'completed',\n", + " 'type': 'file_search_call'}]}" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response.additional_kwargs" + ] + }, + { + "cell_type": "markdown", + "id": "6fda05f0-4b81-4709-9407-f316d760ad50", + "metadata": {}, + "source": [ + "### Managing conversation state\n", + "\n", + "The Responses API supports management of [conversation state](https://platform.openai.com/docs/guides/conversation-state?api-mode=responses).\n", + "\n", + "#### Manually manage state\n", + "\n", + "You can manage the state manually or using [LangGraph](/docs/tutorials/chatbot/), as with other chat models:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "51d3e4d3-ea78-426c-9205-aecb0937fca7", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "As of March 12, 2025, here are some positive news stories that highlight recent uplifting events:\n", + "\n", + "*... exemplify positive developments in health, environmental sustainability, and community well-being. \n" + ] + } + ], + "source": [ + "from langchain_openai import ChatOpenAI\n", + "\n", + "llm = ChatOpenAI(model=\"gpt-4o-mini\")\n", + "\n", + "tool = {\"type\": \"web_search_preview\"}\n", + "llm_with_tools = llm.bind_tools([tool])\n", + "\n", + "first_query = \"What was a positive news story from today?\"\n", + "messages = [{\"role\": \"user\", \"content\": first_query}]\n", + "\n", + "response = llm_with_tools.invoke(messages)\n", + "response_text = response.text()\n", + "print(f\"{response_text[:100]}... {response_text[-100:]}\")" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "5da9d20f-9712-46f4-a395-5be5a7c1bc62", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Your question was: \"What was a positive news story from today?\"\n", + "\n", + "The last sentence of my answer was: \"These stories exemplify positive developments in health, environmental sustainability, and community well-being.\"\n" + ] + } + ], + "source": [ + "second_query = (\n", + " \"Repeat my question back to me, as well as the last sentence of your answer.\"\n", + ")\n", + "\n", + "messages.extend(\n", + " [\n", + " response,\n", + " {\"role\": \"user\", \"content\": second_query},\n", + " ]\n", + ")\n", + "second_response = llm_with_tools.invoke(messages)\n", + "print(second_response.text())" + ] + }, + { + "cell_type": "markdown", + "id": "5fd8ca21-8a5e-4294-af32-11f26a040171", + "metadata": {}, + "source": [ + ":::tip\n", + "\n", + "You can use [LangGraph](https://langchain-ai.github.io/langgraph/) to manage conversational threads for you in a variety of backends, including in-memory and Postgres. See [this tutorial](/docs/tutorials/chatbot/) to get started.\n", + "\n", + ":::\n", + "\n", + "\n", + "#### Passing `previous_response_id`\n", + "\n", + "When using the Responses API, LangChain messages will include an `\"id\"` field in its metadata. Passing this ID to subsequent invocations will continue the conversation. Note that this is [equivalent](https://platform.openai.com/docs/guides/conversation-state?api-mode=responses#openai-apis-for-conversation-state) to manually passing in messages from a billing perspective." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "009e541a-b372-410e-b9dd-608a8052ce09", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Hi Bob! How can I assist you today?\n" + ] + } + ], + "source": [ + "from langchain_openai import ChatOpenAI\n", + "\n", + "llm = ChatOpenAI(\n", + " model=\"gpt-4o-mini\",\n", + " use_responses_api=True,\n", + ")\n", + "response = llm.invoke(\"Hi, I'm Bob.\")\n", + "print(response.text())" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "393a443a-4c5f-4a07-bc0e-c76e529b35e3", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Your name is Bob. How can I help you today, Bob?\n" + ] + } + ], + "source": [ + "second_response = llm.invoke(\n", + " \"What is my name?\",\n", + " previous_response_id=response.response_metadata[\"id\"],\n", + ")\n", + "print(second_response.text())" + ] + }, { "cell_type": "markdown", "id": "57e27714", diff --git a/libs/core/langchain_core/messages/ai.py b/libs/core/langchain_core/messages/ai.py index c317bf099c7..19267060472 100644 --- a/libs/core/langchain_core/messages/ai.py +++ b/libs/core/langchain_core/messages/ai.py @@ -443,6 +443,11 @@ def add_ai_message_chunks( else: usage_metadata = None + id = None + for id_ in [left.id] + [o.id for o in others]: + if id_: + id = id_ + break return left.__class__( example=left.example, content=content, @@ -450,7 +455,7 @@ def add_ai_message_chunks( tool_call_chunks=tool_call_chunks, response_metadata=response_metadata, usage_metadata=usage_metadata, - id=left.id, + id=id, ) diff --git a/libs/core/langchain_core/utils/function_calling.py b/libs/core/langchain_core/utils/function_calling.py index fc03f36b7e2..72d261ec4c5 100644 --- a/libs/core/langchain_core/utils/function_calling.py +++ b/libs/core/langchain_core/utils/function_calling.py @@ -531,9 +531,19 @@ def convert_to_openai_tool( 'description' and 'parameters' keys are now optional. Only 'name' is required and guaranteed to be part of the output. + + .. versionchanged:: 0.3.44 + + Return OpenAI Responses API-style tools unchanged. This includes + any dict with "type" in "file_search", "function", "computer_use_preview", + "web_search_preview". """ - if isinstance(tool, dict) and tool.get("type") == "function" and "function" in tool: - return tool + if isinstance(tool, dict): + if tool.get("type") in ("function", "file_search", "computer_use_preview"): + return tool + # As of 03.12.25 can be "web_search_preview" or "web_search_preview_2025_03_11" + if (tool.get("type") or "").startswith("web_search_preview"): + return tool oai_function = convert_to_openai_function(tool, strict=strict) return {"type": "function", "function": oai_function} diff --git a/libs/core/pyproject.toml b/libs/core/pyproject.toml index 484bfaa165c..620a0ea23ab 100644 --- a/libs/core/pyproject.toml +++ b/libs/core/pyproject.toml @@ -17,7 +17,7 @@ dependencies = [ "pydantic<3.0.0,>=2.7.4; python_full_version >= \"3.12.4\"", ] name = "langchain-core" -version = "0.3.44" +version = "0.3.45-rc.1" description = "Building applications with LLMs through composability" readme = "README.md" diff --git a/libs/langchain/tests/unit_tests/chat_models/test_base.py b/libs/langchain/tests/unit_tests/chat_models/test_base.py index 46055b092cb..2e6fc4f4521 100644 --- a/libs/langchain/tests/unit_tests/chat_models/test_base.py +++ b/libs/langchain/tests/unit_tests/chat_models/test_base.py @@ -133,6 +133,7 @@ def test_configurable() -> None: "extra_body": None, "include_response_headers": False, "stream_usage": False, + "use_responses_api": None, }, "kwargs": { "tools": [ diff --git a/libs/partners/openai/langchain_openai/chat_models/base.py b/libs/partners/openai/langchain_openai/chat_models/base.py index e8fb104c077..a2720764a86 100644 --- a/libs/partners/openai/langchain_openai/chat_models/base.py +++ b/libs/partners/openai/langchain_openai/chat_models/base.py @@ -12,9 +12,11 @@ import sys import warnings from functools import partial from io import BytesIO +from json import JSONDecodeError from math import ceil from operator import itemgetter from typing import ( + TYPE_CHECKING, Any, AsyncIterator, Callable, @@ -89,6 +91,7 @@ from langchain_core.runnables import ( ) from langchain_core.runnables.config import run_in_executor from langchain_core.tools import BaseTool +from langchain_core.tools.base import _stringify from langchain_core.utils import get_pydantic_field_names from langchain_core.utils.function_calling import ( convert_to_openai_function, @@ -104,12 +107,17 @@ from pydantic import BaseModel, ConfigDict, Field, SecretStr, model_validator from pydantic.v1 import BaseModel as BaseModelV1 from typing_extensions import Self +if TYPE_CHECKING: + from openai.types.responses import Response + logger = logging.getLogger(__name__) # This SSL context is equivelent to the default `verify=True`. # https://www.python-httpx.org/advanced/ssl/#configuring-client-instances global_ssl_context = ssl.create_default_context(cafile=certifi.where()) +_FUNCTION_CALL_IDS_MAP_KEY = "__openai_function_call_ids__" + def _convert_dict_to_message(_dict: Mapping[str, Any]) -> BaseMessage: """Convert a dictionary to a LangChain message. @@ -528,6 +536,14 @@ class BaseChatOpenAI(BaseChatModel): invocation. """ + use_responses_api: Optional[bool] = None + """Whether to use the Responses API instead of the Chat API. + + If not specified then will be inferred based on invocation params. + + .. versionadded:: 0.3.9 + """ + model_config = ConfigDict(populate_by_name=True) @model_validator(mode="before") @@ -654,7 +670,7 @@ class BaseChatOpenAI(BaseChatModel): if output is None: # Happens in streaming continue - token_usage = output["token_usage"] + token_usage = output.get("token_usage") if token_usage is not None: for k, v in token_usage.items(): if v is None: @@ -725,6 +741,50 @@ class BaseChatOpenAI(BaseChatModel): ) return generation_chunk + def _stream_responses( + self, + messages: List[BaseMessage], + stop: Optional[List[str]] = None, + run_manager: Optional[CallbackManagerForLLMRun] = None, + **kwargs: Any, + ) -> Iterator[ChatGenerationChunk]: + kwargs["stream"] = True + payload = self._get_request_payload(messages, stop=stop, **kwargs) + context_manager = self.root_client.responses.create(**payload) + + with context_manager as response: + for chunk in response: + if generation_chunk := _convert_responses_chunk_to_generation_chunk( + chunk + ): + if run_manager: + run_manager.on_llm_new_token( + generation_chunk.text, chunk=generation_chunk + ) + yield generation_chunk + + async def _astream_responses( + self, + messages: List[BaseMessage], + stop: Optional[List[str]] = None, + run_manager: Optional[AsyncCallbackManagerForLLMRun] = None, + **kwargs: Any, + ) -> AsyncIterator[ChatGenerationChunk]: + kwargs["stream"] = True + payload = self._get_request_payload(messages, stop=stop, **kwargs) + context_manager = await self.root_async_client.responses.create(**payload) + + async with context_manager as response: + async for chunk in response: + if generation_chunk := _convert_responses_chunk_to_generation_chunk( + chunk + ): + if run_manager: + await run_manager.on_llm_new_token( + generation_chunk.text, chunk=generation_chunk + ) + yield generation_chunk + def _stream( self, messages: List[BaseMessage], @@ -819,10 +879,19 @@ class BaseChatOpenAI(BaseChatModel): raw_response = self.client.with_raw_response.create(**payload) response = raw_response.parse() generation_info = {"headers": dict(raw_response.headers)} + elif self._use_responses_api(payload): + response = self.root_client.responses.create(**payload) + return _construct_lc_result_from_responses_api(response) else: response = self.client.create(**payload) return self._create_chat_result(response, generation_info) + def _use_responses_api(self, payload: dict) -> bool: + if isinstance(self.use_responses_api, bool): + return self.use_responses_api + else: + return _use_responses_api(payload) + def _get_request_payload( self, input_: LanguageModelInput, @@ -834,11 +903,12 @@ class BaseChatOpenAI(BaseChatModel): if stop is not None: kwargs["stop"] = stop - return { - "messages": [_convert_message_to_dict(m) for m in messages], - **self._default_params, - **kwargs, - } + payload = {**self._default_params, **kwargs} + if self._use_responses_api(payload): + payload = _construct_responses_api_payload(messages, payload) + else: + payload["messages"] = [_convert_message_to_dict(m) for m in messages] + return payload def _create_chat_result( self, @@ -877,6 +947,8 @@ class BaseChatOpenAI(BaseChatModel): "model_name": response_dict.get("model", self.model_name), "system_fingerprint": response_dict.get("system_fingerprint", ""), } + if "id" in response_dict: + llm_output["id"] = response_dict["id"] if isinstance(response, openai.BaseModel) and getattr( response, "choices", None @@ -989,6 +1061,9 @@ class BaseChatOpenAI(BaseChatModel): raw_response = await self.async_client.with_raw_response.create(**payload) response = raw_response.parse() generation_info = {"headers": dict(raw_response.headers)} + elif self._use_responses_api(payload): + response = await self.root_async_client.responses.create(**payload) + return _construct_lc_result_from_responses_api(response) else: response = await self.async_client.create(**payload) return await run_in_executor( @@ -1258,33 +1333,38 @@ class BaseChatOpenAI(BaseChatModel): formatted_tools = [ convert_to_openai_tool(tool, strict=strict) for tool in tools ] + tool_names = [] + for tool in formatted_tools: + if "function" in tool: + tool_names.append(tool["function"]["name"]) + elif "name" in tool: + tool_names.append(tool["name"]) + else: + pass if tool_choice: if isinstance(tool_choice, str): # tool_choice is a tool/function name - if tool_choice not in ("auto", "none", "any", "required"): + if tool_choice in tool_names: tool_choice = { "type": "function", "function": {"name": tool_choice}, } + elif tool_choice in ( + "file_search", + "web_search_preview", + "computer_use_preview", + ): + tool_choice = {"type": tool_choice} # 'any' is not natively supported by OpenAI API. # We support 'any' since other models use this instead of 'required'. - if tool_choice == "any": + elif tool_choice == "any": tool_choice = "required" + else: + pass elif isinstance(tool_choice, bool): tool_choice = "required" elif isinstance(tool_choice, dict): - tool_names = [ - formatted_tool["function"]["name"] - for formatted_tool in formatted_tools - ] - if not any( - tool_name == tool_choice["function"]["name"] - for tool_name in tool_names - ): - raise ValueError( - f"Tool choice {tool_choice} was specified, but the only " - f"provided tools were {tool_names}." - ) + pass else: raise ValueError( f"Unrecognized tool_choice type. Expected str, bool or dict. " @@ -1562,6 +1642,8 @@ class ChatOpenAI(BaseChatOpenAI): # type: ignore[override] stream_options: Dict Configure streaming outputs, like whether to return token usage when streaming (``{"include_usage": True}``). + use_responses_api: Optional[bool] + Whether to use the responses API. See full list of supported init args and their descriptions in the params section. @@ -1805,6 +1887,79 @@ class ChatOpenAI(BaseChatOpenAI): # type: ignore[override] See ``ChatOpenAI.bind_tools()`` method for more. + .. dropdown:: Built-in tools + + .. versionadded:: 0.3.9 + + You can access `built-in tools `_ + supported by the OpenAI Responses API. See LangChain + `docs `_ for more + detail. + + .. code-block:: python + + from langchain_openai import ChatOpenAI + + llm = ChatOpenAI(model="gpt-4o-mini") + + tool = {"type": "web_search_preview"} + llm_with_tools = llm.bind_tools([tool]) + + response = llm_with_tools.invoke("What was a positive news story from today?") + response.content + + .. code-block:: python + + [ + { + "type": "text", + "text": "Today, a heartwarming story emerged from ...", + "annotations": [ + { + "end_index": 778, + "start_index": 682, + "title": "Title of story", + "type": "url_citation", + "url": "", + } + ], + } + ] + + .. dropdown:: Managing conversation state + + .. versionadded:: 0.3.9 + + OpenAI's Responses API supports management of + `conversation state `_. + Passing in response IDs from previous messages will continue a conversational + thread. See LangChain + `docs `_ for more + detail. + + .. code-block:: python + + from langchain_openai import ChatOpenAI + + llm = ChatOpenAI(model="gpt-4o-mini", use_responses_api=True) + response = llm.invoke("Hi, I'm Bob.") + response.text() + + .. code-block:: python + + "Hi Bob! How can I assist you today?" + + .. code-block:: python + + second_response = llm.invoke( + "What is my name?", previous_response_id=response.response_metadata["id"] + ) + second_response.text() + + .. code-block:: python + + "Your name is Bob. How can I help you today, Bob?" + .. dropdown:: Structured output .. code-block:: python @@ -2082,27 +2237,34 @@ class ChatOpenAI(BaseChatOpenAI): # type: ignore[override] self, *args: Any, stream_usage: Optional[bool] = None, **kwargs: Any ) -> Iterator[ChatGenerationChunk]: """Set default stream_options.""" - stream_usage = self._should_stream_usage(stream_usage, **kwargs) - # Note: stream_options is not a valid parameter for Azure OpenAI. - # To support users proxying Azure through ChatOpenAI, here we only specify - # stream_options if include_usage is set to True. - # See https://learn.microsoft.com/en-us/azure/ai-services/openai/whats-new - # for release notes. - if stream_usage: - kwargs["stream_options"] = {"include_usage": stream_usage} + if self._use_responses_api(kwargs): + return super()._stream_responses(*args, **kwargs) + else: + stream_usage = self._should_stream_usage(stream_usage, **kwargs) + # Note: stream_options is not a valid parameter for Azure OpenAI. + # To support users proxying Azure through ChatOpenAI, here we only specify + # stream_options if include_usage is set to True. + # See https://learn.microsoft.com/en-us/azure/ai-services/openai/whats-new + # for release notes. + if stream_usage: + kwargs["stream_options"] = {"include_usage": stream_usage} - return super()._stream(*args, **kwargs) + return super()._stream(*args, **kwargs) async def _astream( self, *args: Any, stream_usage: Optional[bool] = None, **kwargs: Any ) -> AsyncIterator[ChatGenerationChunk]: """Set default stream_options.""" - stream_usage = self._should_stream_usage(stream_usage, **kwargs) - if stream_usage: - kwargs["stream_options"] = {"include_usage": stream_usage} + if self._use_responses_api(kwargs): + async for chunk in super()._astream_responses(*args, **kwargs): + yield chunk + else: + stream_usage = self._should_stream_usage(stream_usage, **kwargs) + if stream_usage: + kwargs["stream_options"] = {"include_usage": stream_usage} - async for chunk in super()._astream(*args, **kwargs): - yield chunk + async for chunk in super()._astream(*args, **kwargs): + yield chunk def with_structured_output( self, @@ -2617,3 +2779,355 @@ def _create_usage_metadata(oai_token_usage: dict) -> UsageMetadata: **{k: v for k, v in output_token_details.items() if v is not None} ), ) + + +def _create_usage_metadata_responses(oai_token_usage: dict) -> UsageMetadata: + input_tokens = oai_token_usage.get("input_tokens", 0) + output_tokens = oai_token_usage.get("output_tokens", 0) + total_tokens = oai_token_usage.get("total_tokens", input_tokens + output_tokens) + + output_token_details: dict = { + "audio": (oai_token_usage.get("completion_tokens_details") or {}).get( + "audio_tokens" + ), + "reasoning": (oai_token_usage.get("output_token_details") or {}).get( + "reasoning_tokens" + ), + } + return UsageMetadata( + input_tokens=input_tokens, + output_tokens=output_tokens, + total_tokens=total_tokens, + output_token_details=OutputTokenDetails( + **{k: v for k, v in output_token_details.items() if v is not None} + ), + ) + + +def _is_builtin_tool(tool: dict) -> bool: + return "type" in tool and tool["type"] != "function" + + +def _use_responses_api(payload: dict) -> bool: + uses_builtin_tools = "tools" in payload and any( + _is_builtin_tool(tool) for tool in payload["tools"] + ) + responses_only_args = {"previous_response_id", "text", "truncation", "include"} + return bool(uses_builtin_tools or responses_only_args.intersection(payload)) + + +def _construct_responses_api_payload( + messages: Sequence[BaseMessage], payload: dict +) -> dict: + payload["input"] = _construct_responses_api_input(messages) + if tools := payload.pop("tools", None): + new_tools: list = [] + for tool in tools: + # chat api: {"type": "function", "function": {"name": "...", "description": "...", "parameters": {...}, "strict": ...}} # noqa: E501 + # responses api: {"type": "function", "name": "...", "description": "...", "parameters": {...}, "strict": ...} # noqa: E501 + if tool["type"] == "function" and "function" in tool: + new_tools.append({"type": "function", **tool["function"]}) + else: + new_tools.append(tool) + payload["tools"] = new_tools + if tool_choice := payload.pop("tool_choice", None): + # chat api: {"type": "function", "function": {"name": "..."}} + # responses api: {"type": "function", "name": "..."} + if tool_choice["type"] == "function" and "function" in tool_choice: + payload["tool_choice"] = {"type": "function", **tool_choice["function"]} + else: + payload["tool_choice"] = tool_choice + if response_format := payload.pop("response_format", None): + if payload.get("text"): + text = payload["text"] + raise ValueError( + "Can specify at most one of 'response_format' or 'text', received both:" + f"\n{response_format=}\n{text=}" + ) + # chat api: {"type": "json_schema, "json_schema": {"schema": {...}, "name": "...", "description": "...", "strict": ...}} # noqa: E501 + # responses api: {"type": "json_schema, "schema": {...}, "name": "...", "description": "...", "strict": ...} # noqa: E501 + if response_format["type"] == "json_schema": + payload["text"] = {"type": "json_schema", **response_format["json_schema"]} + else: + payload["text"] = response_format + return payload + + +def _construct_responses_api_input(messages: Sequence[BaseMessage]) -> list: + input_ = [] + for lc_msg in messages: + msg = _convert_message_to_dict(lc_msg) + if msg["role"] == "tool": + tool_output = msg["content"] + if not isinstance(tool_output, str): + tool_output = _stringify(tool_output) + function_call_output = { + "type": "function_call_output", + "output": tool_output, + "call_id": msg["tool_call_id"], + } + input_.append(function_call_output) + elif msg["role"] == "assistant": + function_calls = [] + if tool_calls := msg.pop("tool_calls", None): + # TODO: should you be able to preserve the function call object id on + # the langchain tool calls themselves? + if not lc_msg.additional_kwargs.get(_FUNCTION_CALL_IDS_MAP_KEY): + raise ValueError("") + function_call_ids = lc_msg.additional_kwargs[_FUNCTION_CALL_IDS_MAP_KEY] + for tool_call in tool_calls: + function_call = { + "type": "function_call", + "name": tool_call["function"]["name"], + "arguments": tool_call["function"]["arguments"], + "call_id": tool_call["id"], + "id": function_call_ids[tool_call["id"]], + } + function_calls.append(function_call) + + msg["content"] = msg.get("content") or [] + if lc_msg.additional_kwargs.get("refusal"): + if isinstance(msg["content"], str): + msg["content"] = [ + { + "type": "output_text", + "text": msg["content"], + "annotations": [], + } + ] + msg["content"] = msg["content"] + [ + {"type": "refusal", "refusal": lc_msg.additional_kwargs["refusal"]} + ] + if isinstance(msg["content"], list): + new_blocks = [] + for block in msg["content"]: + # chat api: {"type": "text", "text": "..."} + # responses api: {"type": "output_text", "text": "...", "annotations": [...]} # noqa: E501 + if block["type"] == "text": + new_blocks.append( + { + "type": "output_text", + "text": block["text"], + "annotations": block.get("annotations") or [], + } + ) + elif block["type"] in ("output_text", "refusal"): + new_blocks.append(block) + else: + pass + msg["content"] = new_blocks + if msg["content"]: + input_.append(msg) + input_.extend(function_calls) + elif msg["role"] == "user": + if isinstance(msg["content"], list): + new_blocks = [] + for block in msg["content"]: + # chat api: {"type": "text", "text": "..."} + # responses api: {"type": "input_text", "text": "..."} + if block["type"] == "text": + new_blocks.append({"type": "input_text", "text": block["text"]}) + # chat api: {"type": "image_url", "image_url": {"url": "...", "detail": "..."}} # noqa: E501 + # responses api: {"type": "image_url", "image_url": "...", "detail": "...", "file_id": "..."} # noqa: E501 + elif block["type"] == "image_url": + new_block = { + "type": "input_image", + "image_url": block["image_url"]["url"], + } + if block["image_url"].get("detail"): + new_block["detail"] = block["image_url"]["detail"] + new_blocks.append(new_block) + elif block["type"] in ("input_text", "input_image", "input_file"): + new_blocks.append(block) + else: + pass + msg["content"] = new_blocks + input_.append(msg) + else: + input_.append(msg) + + return input_ + + +def _construct_lc_result_from_responses_api(response: Response) -> ChatResult: + """Construct ChatResponse from OpenAI Response API response.""" + if response.error: + raise ValueError(response.error) + + response_metadata = { + k: v + for k, v in response.model_dump(exclude_none=True, mode="json").items() + if k + in ( + "created_at", + "id", + "incomplete_details", + "metadata", + "object", + "status", + "user", + "model", + ) + } + # for compatibility with chat completion calls. + response_metadata["model_name"] = response_metadata.get("model") + if response.usage: + usage_metadata = _create_usage_metadata_responses(response.usage.model_dump()) + else: + usage_metadata = None + + content_blocks: list = [] + tool_calls = [] + invalid_tool_calls = [] + additional_kwargs: dict = {} + msg_id = None + for output in response.output: + if output.type == "message": + for content in output.content: + if content.type == "output_text": + block = { + "type": "text", + "text": content.text, + "annotations": [ + annotation.model_dump() + for annotation in content.annotations + ], + } + content_blocks.append(block) + if content.type == "refusal": + additional_kwargs["refusal"] = content.refusal + msg_id = output.id + elif output.type == "function_call": + try: + args = json.loads(output.arguments, strict=False) + error = None + except JSONDecodeError as e: + args = output.arguments + error = str(e) + if error is None: + tool_call = { + "type": "tool_call", + "name": output.name, + "args": args, + "id": output.call_id, + } + tool_calls.append(tool_call) + else: + tool_call = { + "type": "invalid_tool_call", + "name": output.name, + "args": args, + "id": output.call_id, + "error": error, + } + invalid_tool_calls.append(tool_call) + if _FUNCTION_CALL_IDS_MAP_KEY not in additional_kwargs: + additional_kwargs[_FUNCTION_CALL_IDS_MAP_KEY] = {} + additional_kwargs[_FUNCTION_CALL_IDS_MAP_KEY][output.call_id] = output.id + elif output.type == "reasoning": + additional_kwargs["reasoning"] = output.model_dump( + exclude_none=True, mode="json" + ) + else: + tool_output = output.model_dump(exclude_none=True, mode="json") + if "tool_outputs" in additional_kwargs: + additional_kwargs["tool_outputs"].append(tool_output) + else: + additional_kwargs["tool_outputs"] = [tool_output] + message = AIMessage( + content=content_blocks, + id=msg_id, + usage_metadata=usage_metadata, + response_metadata=response_metadata, + additional_kwargs=additional_kwargs, + tool_calls=tool_calls, + invalid_tool_calls=invalid_tool_calls, + ) + return ChatResult(generations=[ChatGeneration(message=message)]) + + +def _convert_responses_chunk_to_generation_chunk( + chunk: Any, +) -> Optional[ChatGenerationChunk]: + content = [] + tool_call_chunks: list = [] + additional_kwargs: dict = {} + response_metadata = {} + usage_metadata = None + id = None + if chunk.type == "response.output_text.delta": + content.append( + {"type": "text", "text": chunk.delta, "index": chunk.content_index} + ) + elif chunk.type == "response.output_text.annotation.added": + content.append( + { + "annotations": [ + chunk.annotation.model_dump(exclude_none=True, mode="json") + ], + "index": chunk.content_index, + } + ) + elif chunk.type == "response.created": + response_metadata["id"] = chunk.response.id + elif chunk.type == "response.completed": + msg = cast( + AIMessage, + ( + _construct_lc_result_from_responses_api(chunk.response) + .generations[0] + .message + ), + ) + usage_metadata = msg.usage_metadata + response_metadata = { + k: v for k, v in msg.response_metadata.items() if k != "id" + } + elif chunk.type == "response.output_item.added" and chunk.item.type == "message": + id = chunk.item.id + elif ( + chunk.type == "response.output_item.added" + and chunk.item.type == "function_call" + ): + tool_call_chunks.append( + { + "type": "tool_call_chunk", + "name": chunk.item.name, + "args": chunk.item.arguments, + "id": chunk.item.call_id, + "index": chunk.output_index, + } + ) + additional_kwargs[_FUNCTION_CALL_IDS_MAP_KEY] = { + chunk.item.call_id: chunk.item.id + } + elif chunk.type == "response.output_item.done" and chunk.item.type in ( + "web_search_call", + "file_search_call", + ): + additional_kwargs["tool_outputs"] = [ + chunk.item.model_dump(exclude_none=True, mode="json") + ] + elif chunk.type == "response.function_call_arguments.delta": + tool_call_chunks.append( + { + "type": "tool_call_chunk", + "args": chunk.delta, + "index": chunk.output_index, + } + ) + elif chunk.type == "response.refusal.done": + additional_kwargs["refusal"] = chunk.refusal + else: + return None + + return ChatGenerationChunk( + message=AIMessageChunk( + content=content, # type: ignore[arg-type] + tool_call_chunks=tool_call_chunks, + usage_metadata=usage_metadata, + response_metadata=response_metadata, + additional_kwargs=additional_kwargs, + id=id, + ) + ) diff --git a/libs/partners/openai/pyproject.toml b/libs/partners/openai/pyproject.toml index 72d2eb4be97..4b4939d1032 100644 --- a/libs/partners/openai/pyproject.toml +++ b/libs/partners/openai/pyproject.toml @@ -7,12 +7,12 @@ authors = [] license = { text = "MIT" } requires-python = "<4.0,>=3.9" dependencies = [ - "langchain-core<1.0.0,>=0.3.43", - "openai<2.0.0,>=1.58.1", + "langchain-core<1.0.0,>=0.3.45-rc.1", + "openai<2.0.0,>=1.66.0", "tiktoken<1,>=0.7", ] name = "langchain-openai" -version = "0.3.8" +version = "0.3.9-rc.1" description = "An integration package connecting OpenAI and LangChain" readme = "README.md" diff --git a/libs/partners/openai/tests/integration_tests/chat_models/test_responses_api.py b/libs/partners/openai/tests/integration_tests/chat_models/test_responses_api.py new file mode 100644 index 00000000000..c320083e6ef --- /dev/null +++ b/libs/partners/openai/tests/integration_tests/chat_models/test_responses_api.py @@ -0,0 +1,168 @@ +"""Test Responses API usage.""" + +import os +from typing import Any, Optional, cast + +import pytest +from langchain_core.messages import ( + AIMessage, + AIMessageChunk, + BaseMessage, + BaseMessageChunk, +) + +from langchain_openai import ChatOpenAI + + +def _check_response(response: Optional[BaseMessage]) -> None: + assert isinstance(response, AIMessage) + assert isinstance(response.content, list) + for block in response.content: + assert isinstance(block, dict) + if block["type"] == "text": + assert isinstance(block["text"], str) + for annotation in block["annotations"]: + if annotation["type"] == "file_citation": + assert all( + key in annotation + for key in ["file_id", "filename", "index", "type"] + ) + elif annotation["type"] == "web_search": + assert all( + key in annotation + for key in ["end_index", "start_index", "title", "type", "url"] + ) + + text_content = response.text() + assert isinstance(text_content, str) + assert text_content + assert response.usage_metadata + assert response.usage_metadata["input_tokens"] > 0 + assert response.usage_metadata["output_tokens"] > 0 + assert response.usage_metadata["total_tokens"] > 0 + assert response.response_metadata["model_name"] + for tool_output in response.additional_kwargs["tool_outputs"]: + assert tool_output["id"] + assert tool_output["status"] + assert tool_output["type"] + + +def test_web_search() -> None: + llm = ChatOpenAI(model="gpt-4o-mini") + first_response = llm.invoke( + "What was a positive news story from today?", + tools=[{"type": "web_search_preview"}], + ) + _check_response(first_response) + + # Test streaming + full: Optional[BaseMessageChunk] = None + for chunk in llm.stream( + "What was a positive news story from today?", + tools=[{"type": "web_search_preview"}], + ): + assert isinstance(chunk, AIMessageChunk) + full = chunk if full is None else full + chunk + _check_response(full) + + # Use OpenAI's stateful API + response = llm.invoke( + "what about a negative one", + tools=[{"type": "web_search_preview"}], + previous_response_id=first_response.response_metadata["id"], + ) + _check_response(response) + + # Manually pass in chat history + response = llm.invoke( + [ + first_response, + { + "role": "user", + "content": [{"type": "text", "text": "what about a negative one"}], + }, + ], + tools=[{"type": "web_search_preview"}], + ) + _check_response(response) + + # Bind tool + response = llm.bind_tools([{"type": "web_search_preview"}]).invoke( + "What was a positive news story from today?" + ) + _check_response(response) + + +async def test_web_search_async() -> None: + llm = ChatOpenAI(model="gpt-4o-mini") + response = await llm.ainvoke( + "What was a positive news story from today?", + tools=[{"type": "web_search_preview"}], + ) + _check_response(response) + assert response.response_metadata["status"] + + # Test streaming + full: Optional[BaseMessageChunk] = None + async for chunk in llm.astream( + "What was a positive news story from today?", + tools=[{"type": "web_search_preview"}], + ): + assert isinstance(chunk, AIMessageChunk) + full = chunk if full is None else full + chunk + assert isinstance(full, AIMessageChunk) + _check_response(full) + + +def test_function_calling() -> None: + def multiply(x: int, y: int) -> int: + """return x * y""" + return x * y + + llm = ChatOpenAI(model="gpt-4o-mini") + bound_llm = llm.bind_tools([multiply, {"type": "web_search_preview"}]) + ai_msg = cast(AIMessage, bound_llm.invoke("whats 5 * 4")) + assert len(ai_msg.tool_calls) == 1 + assert ai_msg.tool_calls[0]["name"] == "multiply" + assert set(ai_msg.tool_calls[0]["args"]) == {"x", "y"} + + full: Any = None + for chunk in bound_llm.stream("whats 5 * 4"): + assert isinstance(chunk, AIMessageChunk) + full = chunk if full is None else full + chunk + assert len(full.tool_calls) == 1 + assert full.tool_calls[0]["name"] == "multiply" + assert set(full.tool_calls[0]["args"]) == {"x", "y"} + + response = bound_llm.invoke("whats some good news from today") + _check_response(response) + + +def test_stateful_api() -> None: + llm = ChatOpenAI(model="gpt-4o-mini", use_responses_api=True) + response = llm.invoke("how are you, my name is Bobo") + assert "id" in response.response_metadata + + second_response = llm.invoke( + "what's my name", previous_response_id=response.response_metadata["id"] + ) + assert isinstance(second_response.content, list) + assert "bobo" in second_response.content[0]["text"].lower() # type: ignore + + +def test_file_search() -> None: + pytest.skip() # TODO: set up infra + llm = ChatOpenAI(model="gpt-4o-mini") + tool = { + "type": "file_search", + "vector_store_ids": [os.environ["OPENAI_VECTOR_STORE_ID"]], + } + response = llm.invoke("What is deep research by OpenAI?", tools=[tool]) + _check_response(response) + + full: Optional[BaseMessageChunk] = None + for chunk in llm.stream("What is deep research by OpenAI?", tools=[tool]): + assert isinstance(chunk, AIMessageChunk) + full = chunk if full is None else full + chunk + assert isinstance(full, AIMessageChunk) + _check_response(full) diff --git a/libs/partners/openai/tests/unit_tests/chat_models/test_base.py b/libs/partners/openai/tests/unit_tests/chat_models/test_base.py index 8f8c6fa0361..e5e89990b78 100644 --- a/libs/partners/openai/tests/unit_tests/chat_models/test_base.py +++ b/libs/partners/openai/tests/unit_tests/chat_models/test_base.py @@ -3,7 +3,7 @@ import json from functools import partial from types import TracebackType -from typing import Any, Dict, List, Literal, Optional, Type, Union +from typing import Any, Dict, List, Literal, Optional, Type, Union, cast from unittest.mock import AsyncMock, MagicMock, patch import pytest @@ -19,13 +19,30 @@ from langchain_core.messages import ( ToolMessage, ) from langchain_core.messages.ai import UsageMetadata -from langchain_core.outputs import ChatGeneration +from langchain_core.outputs import ChatGeneration, ChatResult from langchain_core.runnables import RunnableLambda +from openai.types.responses import ResponseOutputMessage +from openai.types.responses.response import IncompleteDetails, Response, ResponseUsage +from openai.types.responses.response_error import ResponseError +from openai.types.responses.response_file_search_tool_call import ( + ResponseFileSearchToolCall, + Result, +) +from openai.types.responses.response_function_tool_call import ResponseFunctionToolCall +from openai.types.responses.response_function_web_search import ( + ResponseFunctionWebSearch, +) +from openai.types.responses.response_output_refusal import ResponseOutputRefusal +from openai.types.responses.response_output_text import ResponseOutputText +from openai.types.responses.response_usage import OutputTokensDetails from pydantic import BaseModel, Field from typing_extensions import TypedDict from langchain_openai import ChatOpenAI from langchain_openai.chat_models.base import ( + _FUNCTION_CALL_IDS_MAP_KEY, + _construct_lc_result_from_responses_api, + _construct_responses_api_input, _convert_dict_to_message, _convert_message_to_dict, _convert_to_openai_response_format, @@ -862,7 +879,7 @@ def test_nested_structured_output_strict() -> None: setup: str punchline: str - self_evaluation: SelfEvaluation + _evaluation: SelfEvaluation llm.with_structured_output(JokeWithEvaluation, method="json_schema") @@ -936,3 +953,731 @@ def test_structured_outputs_parser() -> None: assert isinstance(deserialized, ChatGeneration) result = output_parser.invoke(deserialized.message) assert result == parsed_response + + +def test__construct_lc_result_from_responses_api_error_handling() -> None: + """Test that errors in the response are properly raised.""" + response = Response( + id="resp_123", + created_at=1234567890, + model="gpt-4o", + object="response", + error=ResponseError(message="Test error", code="server_error"), + parallel_tool_calls=True, + tools=[], + tool_choice="auto", + output=[], + ) + + with pytest.raises(ValueError) as excinfo: + _construct_lc_result_from_responses_api(response) + + assert "Test error" in str(excinfo.value) + + +def test__construct_lc_result_from_responses_api_basic_text_response() -> None: + """Test a basic text response with no tools or special features.""" + response = Response( + id="resp_123", + created_at=1234567890, + model="gpt-4o", + object="response", + parallel_tool_calls=True, + tools=[], + tool_choice="auto", + output=[ + ResponseOutputMessage( + type="message", + id="msg_123", + content=[ + ResponseOutputText( + type="output_text", text="Hello, world!", annotations=[] + ) + ], + role="assistant", + status="completed", + ) + ], + usage=ResponseUsage( + input_tokens=10, + output_tokens=3, + total_tokens=13, + output_tokens_details=OutputTokensDetails(reasoning_tokens=0), + ), + ) + + result = _construct_lc_result_from_responses_api(response) + + assert isinstance(result, ChatResult) + assert len(result.generations) == 1 + assert isinstance(result.generations[0], ChatGeneration) + assert isinstance(result.generations[0].message, AIMessage) + assert result.generations[0].message.content == [ + {"type": "text", "text": "Hello, world!", "annotations": []} + ] + assert result.generations[0].message.id == "msg_123" + assert result.generations[0].message.usage_metadata + assert result.generations[0].message.usage_metadata["input_tokens"] == 10 + assert result.generations[0].message.usage_metadata["output_tokens"] == 3 + assert result.generations[0].message.usage_metadata["total_tokens"] == 13 + assert result.generations[0].message.response_metadata["id"] == "resp_123" + assert result.generations[0].message.response_metadata["model_name"] == "gpt-4o" + + +def test__construct_lc_result_from_responses_api_multiple_text_blocks() -> None: + """Test a response with multiple text blocks.""" + response = Response( + id="resp_123", + created_at=1234567890, + model="gpt-4o", + object="response", + parallel_tool_calls=True, + tools=[], + tool_choice="auto", + output=[ + ResponseOutputMessage( + type="message", + id="msg_123", + content=[ + ResponseOutputText( + type="output_text", text="First part", annotations=[] + ), + ResponseOutputText( + type="output_text", text="Second part", annotations=[] + ), + ], + role="assistant", + status="completed", + ) + ], + ) + + result = _construct_lc_result_from_responses_api(response) + + assert len(result.generations[0].message.content) == 2 + assert result.generations[0].message.content[0]["text"] == "First part" # type: ignore + assert result.generations[0].message.content[1]["text"] == "Second part" # type: ignore + + +def test__construct_lc_result_from_responses_api_refusal_response() -> None: + """Test a response with a refusal.""" + response = Response( + id="resp_123", + created_at=1234567890, + model="gpt-4o", + object="response", + parallel_tool_calls=True, + tools=[], + tool_choice="auto", + output=[ + ResponseOutputMessage( + type="message", + id="msg_123", + content=[ + ResponseOutputRefusal( + type="refusal", refusal="I cannot assist with that request." + ) + ], + role="assistant", + status="completed", + ) + ], + ) + + result = _construct_lc_result_from_responses_api(response) + + assert result.generations[0].message.content == [] + assert ( + result.generations[0].message.additional_kwargs["refusal"] + == "I cannot assist with that request." + ) + + +def test__construct_lc_result_from_responses_api_function_call_valid_json() -> None: + """Test a response with a valid function call.""" + response = Response( + id="resp_123", + created_at=1234567890, + model="gpt-4o", + object="response", + parallel_tool_calls=True, + tools=[], + tool_choice="auto", + output=[ + ResponseFunctionToolCall( + type="function_call", + id="func_123", + call_id="call_123", + name="get_weather", + arguments='{"location": "New York", "unit": "celsius"}', + ) + ], + ) + + result = _construct_lc_result_from_responses_api(response) + + msg: AIMessage = cast(AIMessage, result.generations[0].message) + assert len(msg.tool_calls) == 1 + assert msg.tool_calls[0]["type"] == "tool_call" + assert msg.tool_calls[0]["name"] == "get_weather" + assert msg.tool_calls[0]["id"] == "call_123" + assert msg.tool_calls[0]["args"] == {"location": "New York", "unit": "celsius"} + assert _FUNCTION_CALL_IDS_MAP_KEY in result.generations[0].message.additional_kwargs + assert ( + result.generations[0].message.additional_kwargs[_FUNCTION_CALL_IDS_MAP_KEY][ + "call_123" + ] + == "func_123" + ) + + +def test__construct_lc_result_from_responses_api_function_call_invalid_json() -> None: + """Test a response with an invalid JSON function call.""" + response = Response( + id="resp_123", + created_at=1234567890, + model="gpt-4o", + object="response", + parallel_tool_calls=True, + tools=[], + tool_choice="auto", + output=[ + ResponseFunctionToolCall( + type="function_call", + id="func_123", + call_id="call_123", + name="get_weather", + arguments='{"location": "New York", "unit": "celsius"', + # Missing closing brace + ) + ], + ) + + result = _construct_lc_result_from_responses_api(response) + + msg: AIMessage = cast(AIMessage, result.generations[0].message) + assert len(msg.invalid_tool_calls) == 1 + assert msg.invalid_tool_calls[0]["type"] == "invalid_tool_call" + assert msg.invalid_tool_calls[0]["name"] == "get_weather" + assert msg.invalid_tool_calls[0]["id"] == "call_123" + assert ( + msg.invalid_tool_calls[0]["args"] + == '{"location": "New York", "unit": "celsius"' + ) + assert "error" in msg.invalid_tool_calls[0] + assert _FUNCTION_CALL_IDS_MAP_KEY in result.generations[0].message.additional_kwargs + + +def test__construct_lc_result_from_responses_api_complex_response() -> None: + """Test a complex response with multiple output types.""" + response = Response( + id="resp_123", + created_at=1234567890, + model="gpt-4o", + object="response", + parallel_tool_calls=True, + tools=[], + tool_choice="auto", + output=[ + ResponseOutputMessage( + type="message", + id="msg_123", + content=[ + ResponseOutputText( + type="output_text", + text="Here's the information you requested:", + annotations=[], + ) + ], + role="assistant", + status="completed", + ), + ResponseFunctionToolCall( + type="function_call", + id="func_123", + call_id="call_123", + name="get_weather", + arguments='{"location": "New York"}', + ), + ], + metadata=dict(key1="value1", key2="value2"), + incomplete_details=IncompleteDetails(reason="max_output_tokens"), + status="completed", + user="user_123", + ) + + result = _construct_lc_result_from_responses_api(response) + + # Check message content + assert result.generations[0].message.content == [ + { + "type": "text", + "text": "Here's the information you requested:", + "annotations": [], + } + ] + + # Check tool calls + msg: AIMessage = cast(AIMessage, result.generations[0].message) + assert len(msg.tool_calls) == 1 + assert msg.tool_calls[0]["name"] == "get_weather" + + # Check metadata + assert result.generations[0].message.response_metadata["id"] == "resp_123" + assert result.generations[0].message.response_metadata["metadata"] == { + "key1": "value1", + "key2": "value2", + } + assert result.generations[0].message.response_metadata["incomplete_details"] == { + "reason": "max_output_tokens" + } + assert result.generations[0].message.response_metadata["status"] == "completed" + assert result.generations[0].message.response_metadata["user"] == "user_123" + + +def test__construct_lc_result_from_responses_api_no_usage_metadata() -> None: + """Test a response without usage metadata.""" + response = Response( + id="resp_123", + created_at=1234567890, + model="gpt-4o", + object="response", + parallel_tool_calls=True, + tools=[], + tool_choice="auto", + output=[ + ResponseOutputMessage( + type="message", + id="msg_123", + content=[ + ResponseOutputText( + type="output_text", text="Hello, world!", annotations=[] + ) + ], + role="assistant", + status="completed", + ) + ], + # No usage field + ) + + result = _construct_lc_result_from_responses_api(response) + + assert cast(AIMessage, result.generations[0].message).usage_metadata is None + + +def test__construct_lc_result_from_responses_api_web_search_response() -> None: + """Test a response with web search output.""" + from openai.types.responses.response_function_web_search import ( + ResponseFunctionWebSearch, + ) + + response = Response( + id="resp_123", + created_at=1234567890, + model="gpt-4o", + object="response", + parallel_tool_calls=True, + tools=[], + tool_choice="auto", + output=[ + ResponseFunctionWebSearch( + id="websearch_123", type="web_search_call", status="completed" + ) + ], + ) + + result = _construct_lc_result_from_responses_api(response) + + assert "tool_outputs" in result.generations[0].message.additional_kwargs + assert len(result.generations[0].message.additional_kwargs["tool_outputs"]) == 1 + assert ( + result.generations[0].message.additional_kwargs["tool_outputs"][0]["type"] + == "web_search_call" + ) + assert ( + result.generations[0].message.additional_kwargs["tool_outputs"][0]["id"] + == "websearch_123" + ) + assert ( + result.generations[0].message.additional_kwargs["tool_outputs"][0]["status"] + == "completed" + ) + + +def test__construct_lc_result_from_responses_api_file_search_response() -> None: + """Test a response with file search output.""" + response = Response( + id="resp_123", + created_at=1234567890, + model="gpt-4o", + object="response", + parallel_tool_calls=True, + tools=[], + tool_choice="auto", + output=[ + ResponseFileSearchToolCall( + id="filesearch_123", + type="file_search_call", + status="completed", + queries=["python code", "langchain"], + results=[ + Result( + file_id="file_123", + filename="example.py", + score=0.95, + text="def hello_world() -> None:\n print('Hello, world!')", + attributes={"language": "python", "size": 42}, + ) + ], + ) + ], + ) + + result = _construct_lc_result_from_responses_api(response) + + assert "tool_outputs" in result.generations[0].message.additional_kwargs + assert len(result.generations[0].message.additional_kwargs["tool_outputs"]) == 1 + assert ( + result.generations[0].message.additional_kwargs["tool_outputs"][0]["type"] + == "file_search_call" + ) + assert ( + result.generations[0].message.additional_kwargs["tool_outputs"][0]["id"] + == "filesearch_123" + ) + assert ( + result.generations[0].message.additional_kwargs["tool_outputs"][0]["status"] + == "completed" + ) + assert result.generations[0].message.additional_kwargs["tool_outputs"][0][ + "queries" + ] == ["python code", "langchain"] + assert ( + len( + result.generations[0].message.additional_kwargs["tool_outputs"][0][ + "results" + ] + ) + == 1 + ) + assert ( + result.generations[0].message.additional_kwargs["tool_outputs"][0]["results"][ + 0 + ]["file_id"] + == "file_123" + ) + assert ( + result.generations[0].message.additional_kwargs["tool_outputs"][0]["results"][ + 0 + ]["score"] + == 0.95 + ) + + +def test__construct_lc_result_from_responses_api_mixed_search_responses() -> None: + """Test a response with both web search and file search outputs.""" + + response = Response( + id="resp_123", + created_at=1234567890, + model="gpt-4o", + object="response", + parallel_tool_calls=True, + tools=[], + tool_choice="auto", + output=[ + ResponseOutputMessage( + type="message", + id="msg_123", + content=[ + ResponseOutputText( + type="output_text", text="Here's what I found:", annotations=[] + ) + ], + role="assistant", + status="completed", + ), + ResponseFunctionWebSearch( + id="websearch_123", type="web_search_call", status="completed" + ), + ResponseFileSearchToolCall( + id="filesearch_123", + type="file_search_call", + status="completed", + queries=["python code"], + results=[ + Result( + file_id="file_123", + filename="example.py", + score=0.95, + text="def hello_world() -> None:\n print('Hello, world!')", + ) + ], + ), + ], + ) + + result = _construct_lc_result_from_responses_api(response) + + # Check message content + assert result.generations[0].message.content == [ + {"type": "text", "text": "Here's what I found:", "annotations": []} + ] + + # Check tool outputs + assert "tool_outputs" in result.generations[0].message.additional_kwargs + assert len(result.generations[0].message.additional_kwargs["tool_outputs"]) == 2 + + # Check web search output + web_search = next( + output + for output in result.generations[0].message.additional_kwargs["tool_outputs"] + if output["type"] == "web_search_call" + ) + assert web_search["id"] == "websearch_123" + assert web_search["status"] == "completed" + + # Check file search output + file_search = next( + output + for output in result.generations[0].message.additional_kwargs["tool_outputs"] + if output["type"] == "file_search_call" + ) + assert file_search["id"] == "filesearch_123" + assert file_search["queries"] == ["python code"] + assert file_search["results"][0]["filename"] == "example.py" + + +def test__construct_responses_api_input_human_message_with_text_blocks_conversion() -> ( + None +): + """Test that human messages with text blocks are properly converted.""" + messages: list = [ + HumanMessage(content=[{"type": "text", "text": "What's in this image?"}]) + ] + result = _construct_responses_api_input(messages) + + assert len(result) == 1 + assert result[0]["role"] == "user" + assert isinstance(result[0]["content"], list) + assert len(result[0]["content"]) == 1 + assert result[0]["content"][0]["type"] == "input_text" + assert result[0]["content"][0]["text"] == "What's in this image?" + + +def test__construct_responses_api_input_human_message_with_image_url_conversion() -> ( + None +): + """Test that human messages with image_url blocks are properly converted.""" + messages: list = [ + HumanMessage( + content=[ + {"type": "text", "text": "What's in this image?"}, + { + "type": "image_url", + "image_url": { + "url": "https://example.com/image.jpg", + "detail": "high", + }, + }, + ] + ) + ] + result = _construct_responses_api_input(messages) + + assert len(result) == 1 + assert result[0]["role"] == "user" + assert isinstance(result[0]["content"], list) + assert len(result[0]["content"]) == 2 + + # Check text block conversion + assert result[0]["content"][0]["type"] == "input_text" + assert result[0]["content"][0]["text"] == "What's in this image?" + + # Check image block conversion + assert result[0]["content"][1]["type"] == "input_image" + assert result[0]["content"][1]["image_url"] == "https://example.com/image.jpg" + assert result[0]["content"][1]["detail"] == "high" + + +def test__construct_responses_api_input_ai_message_with_tool_calls() -> None: + """Test that AI messages with tool calls are properly converted.""" + tool_calls = [ + { + "id": "call_123", + "name": "get_weather", + "args": {"location": "San Francisco"}, + "type": "tool_call", + } + ] + + # Create a mapping from tool call IDs to function call IDs + function_call_ids = {"call_123": "func_456"} + + ai_message = AIMessage( + content="", + tool_calls=tool_calls, + additional_kwargs={_FUNCTION_CALL_IDS_MAP_KEY: function_call_ids}, + ) + + result = _construct_responses_api_input([ai_message]) + + assert len(result) == 1 + assert result[0]["type"] == "function_call" + assert result[0]["name"] == "get_weather" + assert result[0]["arguments"] == '{"location": "San Francisco"}' + assert result[0]["call_id"] == "call_123" + assert result[0]["id"] == "func_456" + + +def test__construct_responses_api_input_ai_message_with_tool_calls_and_content() -> ( + None +): + """Test that AI messages with both tool calls and content are properly converted.""" + tool_calls = [ + { + "id": "call_123", + "name": "get_weather", + "args": {"location": "San Francisco"}, + "type": "tool_call", + } + ] + + # Create a mapping from tool call IDs to function call IDs + function_call_ids = {"call_123": "func_456"} + + ai_message = AIMessage( + content="I'll check the weather for you.", + tool_calls=tool_calls, + additional_kwargs={_FUNCTION_CALL_IDS_MAP_KEY: function_call_ids}, + ) + + result = _construct_responses_api_input([ai_message]) + + assert len(result) == 2 + + # Check content + assert result[0]["role"] == "assistant" + assert result[0]["content"] == "I'll check the weather for you." + + # Check function call + assert result[1]["type"] == "function_call" + assert result[1]["name"] == "get_weather" + assert result[1]["arguments"] == '{"location": "San Francisco"}' + assert result[1]["call_id"] == "call_123" + assert result[1]["id"] == "func_456" + + +def test__construct_responses_api_input_missing_function_call_ids() -> None: + """Test AI messages with tool calls but missing function call IDs raise an error.""" + tool_calls = [ + { + "id": "call_123", + "name": "get_weather", + "args": {"location": "San Francisco"}, + "type": "tool_call", + } + ] + + ai_message = AIMessage(content="", tool_calls=tool_calls) + + with pytest.raises(ValueError): + _construct_responses_api_input([ai_message]) + + +def test__construct_responses_api_input_tool_message_conversion() -> None: + """Test that tool messages are properly converted to function_call_output.""" + messages = [ + ToolMessage( + content='{"temperature": 72, "conditions": "sunny"}', + tool_call_id="call_123", + ) + ] + + result = _construct_responses_api_input(messages) + + assert len(result) == 1 + assert result[0]["type"] == "function_call_output" + assert result[0]["output"] == '{"temperature": 72, "conditions": "sunny"}' + assert result[0]["call_id"] == "call_123" + + +def test__construct_responses_api_input_multiple_message_types() -> None: + """Test conversion of a conversation with multiple message types.""" + messages = [ + SystemMessage(content="You are a helpful assistant."), + HumanMessage(content="What's the weather in San Francisco?"), + HumanMessage( + content=[{"type": "text", "text": "What's the weather in San Francisco?"}] + ), + AIMessage( + content="", + tool_calls=[ + { + "type": "tool_call", + "id": "call_123", + "name": "get_weather", + "args": {"location": "San Francisco"}, + } + ], + additional_kwargs={_FUNCTION_CALL_IDS_MAP_KEY: {"call_123": "func_456"}}, + ), + ToolMessage( + content='{"temperature": 72, "conditions": "sunny"}', + tool_call_id="call_123", + ), + AIMessage(content="The weather in San Francisco is 72°F and sunny."), + AIMessage( + content=[ + { + "type": "text", + "text": "The weather in San Francisco is 72°F and sunny.", + } + ] + ), + ] + messages_copy = [m.copy(deep=True) for m in messages] + + result = _construct_responses_api_input(messages) + + assert len(result) == len(messages) + + # Check system message + assert result[0]["role"] == "system" + assert result[0]["content"] == "You are a helpful assistant." + + # Check human message + assert result[1]["role"] == "user" + assert result[1]["content"] == "What's the weather in San Francisco?" + assert result[2]["role"] == "user" + assert result[2]["content"] == [ + {"type": "input_text", "text": "What's the weather in San Francisco?"} + ] + + # Check function call + assert result[3]["type"] == "function_call" + assert result[3]["name"] == "get_weather" + assert result[3]["arguments"] == '{"location": "San Francisco"}' + assert result[3]["call_id"] == "call_123" + assert result[3]["id"] == "func_456" + + # Check function call output + assert result[4]["type"] == "function_call_output" + assert result[4]["output"] == '{"temperature": 72, "conditions": "sunny"}' + assert result[4]["call_id"] == "call_123" + + assert result[5]["role"] == "assistant" + assert result[5]["content"] == "The weather in San Francisco is 72°F and sunny." + + assert result[6]["role"] == "assistant" + assert result[6]["content"] == [ + { + "type": "output_text", + "text": "The weather in San Francisco is 72°F and sunny.", + "annotations": [], + } + ] + + # assert no mutation has occurred + assert messages_copy == messages diff --git a/libs/partners/openai/uv.lock b/libs/partners/openai/uv.lock index 3ab88aaa485..b79bdca9765 100644 --- a/libs/partners/openai/uv.lock +++ b/libs/partners/openai/uv.lock @@ -462,7 +462,7 @@ wheels = [ [[package]] name = "langchain-core" -version = "0.3.43" +version = "0.3.45rc1" source = { editable = "../../core" } dependencies = [ { name = "jsonpatch" }, @@ -520,7 +520,7 @@ typing = [ [[package]] name = "langchain-openai" -version = "0.3.8" +version = "0.3.9rc1" source = { editable = "." } dependencies = [ { name = "langchain-core" }, @@ -566,7 +566,7 @@ typing = [ [package.metadata] requires-dist = [ { name = "langchain-core", editable = "../../core" }, - { name = "openai", specifier = ">=1.58.1,<2.0.0" }, + { name = "openai", specifier = ">=1.66.0,<2.0.0" }, { name = "tiktoken", specifier = ">=0.7,<1" }, ] @@ -751,7 +751,7 @@ wheels = [ [[package]] name = "openai" -version = "1.61.1" +version = "1.66.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "anyio" }, @@ -763,9 +763,9 @@ dependencies = [ { name = "tqdm" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/d9/cf/61e71ce64cf0a38f029da0f9a5f10c9fa0e69a7a977b537126dac50adfea/openai-1.61.1.tar.gz", hash = "sha256:ce1851507218209961f89f3520e06726c0aa7d0512386f0f977e3ac3e4f2472e", size = 350784 } +sdist = { url = "https://files.pythonhosted.org/packages/84/c5/3c422ca3ccc81c063955e7c20739d7f8f37fea0af865c4a60c81e6225e14/openai-1.66.0.tar.gz", hash = "sha256:8a9e672bc6eadec60a962f0b40d7d1c09050010179c919ed65322e433e2d1025", size = 396819 } wheels = [ - { url = "https://files.pythonhosted.org/packages/9a/b6/2e2a011b2dc27a6711376808b4cd8c922c476ea0f1420b39892117fa8563/openai-1.61.1-py3-none-any.whl", hash = "sha256:72b0826240ce26026ac2cd17951691f046e5be82ad122d20a8e1b30ca18bd11e", size = 463126 }, + { url = "https://files.pythonhosted.org/packages/d7/f1/d52960dac9519c9de64593460826a0fe2e19159389ec97ecf3e931d2e6a3/openai-1.66.0-py3-none-any.whl", hash = "sha256:43e4a3c0c066cc5809be4e6aac456a3ebc4ec1848226ef9d1340859ac130d45a", size = 566389 }, ] [[package]] diff --git a/uv.lock b/uv.lock index bb434502fbe..a140495983b 100644 --- a/uv.lock +++ b/uv.lock @@ -1,4 +1,5 @@ version = 1 +revision = 1 requires-python = ">=3.9, <4.0" resolution-markers = [ "python_full_version >= '3.13' and platform_python_implementation == 'PyPy'", @@ -2152,7 +2153,7 @@ wheels = [ [[package]] name = "langchain" -version = "0.3.19" +version = "0.3.20" source = { editable = "libs/langchain" } dependencies = [ { name = "async-timeout", marker = "python_full_version < '3.11'" }, @@ -2191,6 +2192,7 @@ requires-dist = [ { name = "requests", specifier = ">=2,<3" }, { name = "sqlalchemy", specifier = ">=1.4,<3" }, ] +provides-extras = ["community", "anthropic", "openai", "cohere", "google-vertexai", "google-genai", "fireworks", "ollama", "together", "mistralai", "huggingface", "groq", "aws", "deepseek", "xai"] [package.metadata.requires-dev] codespell = [{ name = "codespell", specifier = ">=2.2.0,<3.0.0" }] @@ -2259,7 +2261,7 @@ typing = [ [[package]] name = "langchain-anthropic" -version = "0.3.8" +version = "0.3.9" source = { editable = "libs/partners/anthropic" } dependencies = [ { name = "anthropic" }, @@ -2360,7 +2362,7 @@ typing = [ [[package]] name = "langchain-community" -version = "0.3.18" +version = "0.3.19" source = { editable = "libs/community" } dependencies = [ { name = "aiohttp" }, @@ -2385,8 +2387,7 @@ requires-dist = [ { name = "langchain", editable = "libs/langchain" }, { name = "langchain-core", editable = "libs/core" }, { name = "langsmith", specifier = ">=0.1.125,<0.4" }, - { name = "numpy", marker = "python_full_version < '3.12'", specifier = ">=1.26.4,<2" }, - { name = "numpy", marker = "python_full_version >= '3.12'", specifier = ">=1.26.2,<3" }, + { name = "numpy", specifier = ">=1.26.2,<3" }, { name = "pydantic-settings", specifier = ">=2.4.0,<3.0.0" }, { name = "pyyaml", specifier = ">=5.3" }, { name = "requests", specifier = ">=2,<3" }, @@ -2450,7 +2451,7 @@ typing = [ [[package]] name = "langchain-core" -version = "0.3.40" +version = "0.3.43" source = { editable = "libs/core" } dependencies = [ { name = "jsonpatch" }, @@ -2573,7 +2574,7 @@ dependencies = [ [[package]] name = "langchain-groq" -version = "0.2.4" +version = "0.2.5" source = { editable = "libs/partners/groq" } dependencies = [ { name = "groq" }, @@ -2732,7 +2733,7 @@ typing = [] [[package]] name = "langchain-openai" -version = "0.3.7" +version = "0.3.8" source = { editable = "libs/partners/openai" } dependencies = [ { name = "langchain-core" }, @@ -2743,7 +2744,7 @@ dependencies = [ [package.metadata] requires-dist = [ { name = "langchain-core", editable = "libs/core" }, - { name = "openai", specifier = ">=1.58.1,<2.0.0" }, + { name = "openai", specifier = ">=1.66.0,<2.0.0" }, { name = "tiktoken", specifier = ">=0.7,<1" }, ] @@ -3630,7 +3631,7 @@ wheels = [ [[package]] name = "openai" -version = "1.61.1" +version = "1.66.2" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "anyio" }, @@ -3642,9 +3643,9 @@ dependencies = [ { name = "tqdm" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/d9/cf/61e71ce64cf0a38f029da0f9a5f10c9fa0e69a7a977b537126dac50adfea/openai-1.61.1.tar.gz", hash = "sha256:ce1851507218209961f89f3520e06726c0aa7d0512386f0f977e3ac3e4f2472e", size = 350784 } +sdist = { url = "https://files.pythonhosted.org/packages/d8/e1/b3e1fda1aa32d4f40d4de744e91de4de65c854c3e53c63342e4b5f9c5995/openai-1.66.2.tar.gz", hash = "sha256:9b3a843c25f81ee09b6469d483d9fba779d5c6ea41861180772f043481b0598d", size = 397041 } wheels = [ - { url = "https://files.pythonhosted.org/packages/9a/b6/2e2a011b2dc27a6711376808b4cd8c922c476ea0f1420b39892117fa8563/openai-1.61.1-py3-none-any.whl", hash = "sha256:72b0826240ce26026ac2cd17951691f046e5be82ad122d20a8e1b30ca18bd11e", size = 463126 }, + { url = "https://files.pythonhosted.org/packages/2c/6f/3315b3583ffe3e31c55b446cb22d2a7c235e65ca191674fffae62deb3c11/openai-1.66.2-py3-none-any.whl", hash = "sha256:75194057ee6bb8b732526387b6041327a05656d976fc21c064e21c8ac6b07999", size = 567268 }, ] [[package]] From 5237987643578a9b7f2f014b538671db1ce4e0a0 Mon Sep 17 00:00:00 2001 From: ccurme Date: Wed, 12 Mar 2025 13:45:13 -0400 Subject: [PATCH 14/14] docs: update readme (#30239) Co-authored-by: Vadym Barda --- README.md | 176 +++++++++++---------------------- docs/static/img/logo-dark.svg | 25 +++++ docs/static/img/logo-light.svg | 25 +++++ 3 files changed, 108 insertions(+), 118 deletions(-) create mode 100644 docs/static/img/logo-dark.svg create mode 100644 docs/static/img/logo-light.svg diff --git a/README.md b/README.md index 26c58cae21f..801cd6532c2 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,12 @@ -# 🦜️🔗 LangChain + + + + LangChain Logo + -⚡ Build context-aware reasoning applications ⚡ +
+
+
[![Release Notes](https://img.shields.io/github/release/langchain-ai/langchain?style=flat-square)](https://github.com/langchain-ai/langchain/releases) [![CI](https://github.com/langchain-ai/langchain/actions/workflows/check_diffs.yml/badge.svg)](https://github.com/langchain-ai/langchain/actions/workflows/check_diffs.yml) @@ -12,131 +18,65 @@ [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/langchain-ai/langchain) [![Twitter](https://img.shields.io/twitter/url/https/twitter.com/langchainai.svg?style=social&label=Follow%20%40LangChainAI)](https://twitter.com/langchainai) -Looking for the JS/TS library? Check out [LangChain.js](https://github.com/langchain-ai/langchainjs). +> [!NOTE] +> Looking for the JS/TS library? Check out [LangChain.js](https://github.com/langchain-ai/langchainjs). -To help you ship LangChain apps to production faster, check out [LangSmith](https://smith.langchain.com). -[LangSmith](https://smith.langchain.com) is a unified developer platform for building, testing, and monitoring LLM applications. -Fill out [this form](https://www.langchain.com/contact-sales) to speak with our sales team. - -## Quick Install - -With pip: +LangChain is a framework for building LLM-powered applications. It helps you chain +together interoperable components and third-party integrations to simplify AI +application development — all while future-proofing decisions as the underlying +technology evolves. ```bash -pip install langchain +pip install -U langchain ``` -With conda: +To learn more about LangChain, check out +[the docs](https://python.langchain.com/docs/introduction/). If you’re looking for more +advanced customization or agent orchestration, check out +[LangGraph](https://langchain-ai.github.io/langgraph/), our framework for building +controllable agent workflows. -```bash -conda install langchain -c conda-forge -``` +## Why use LangChain? -## 🤔 What is LangChain? +LangChain helps developers build applications powered by LLMs through a standard +interface for models, embeddings, vector stores, and more. -**LangChain** is a framework for developing applications powered by large language models (LLMs). +Use LangChain for: +- **Real-time data augmentation**. Easily connect LLMs to diverse data sources and +external / internal systems, drawing from LangChain’s vast library of integrations with +model providers, tools, vector stores, retrievers, and more. +- **Model interoperability**. Swap models in and out as your engineering team +experiments to find the best choice for your application’s needs. As the industry +frontier evolves, adapt quickly — LangChain’s abstractions keep you moving without +losing momentum. -For these applications, LangChain simplifies the entire application lifecycle: +## LangChain’s ecosystem +While the LangChain framework can be used standalone, it also integrates seamlessly +with any LangChain product, giving developers a full suite of tools when building LLM +applications. +To improve your LLM application development, pair LangChain with: -- **Open-source libraries**: Build your applications using LangChain's open-source -[components](https://python.langchain.com/docs/concepts/) and -[third-party integrations](https://python.langchain.com/docs/integrations/providers/). - Use [LangGraph](https://langchain-ai.github.io/langgraph/) to build stateful agents with first-class streaming and human-in-the-loop support. -- **Productionization**: Inspect, monitor, and evaluate your apps with [LangSmith](https://docs.smith.langchain.com/) so that you can constantly optimize and deploy with confidence. -- **Deployment**: Turn your LangGraph applications into production-ready APIs and Assistants with [LangGraph Platform](https://langchain-ai.github.io/langgraph/cloud/). +- [LangSmith](http://www.langchain.com/langsmith) - Helpful for agent evals and +observability. Debug poor-performing LLM app runs, evaluate agent trajectories, gain +visibility in production, and improve performance over time. +- [LangGraph](https://langchain-ai.github.io/langgraph/) - Build agents that can +reliably handle complex tasks with LangGraph, our low-level agent orchestration +framework. LangGraph offers customizable architecture, long-term memory, and +human-in-the-loop workflows — and is trusted in production by companies like LinkedIn, +Uber, Klarna, and GitLab. +- [LangGraph Platform](https://langchain-ai.github.io/langgraph/concepts/#langgraph-platform) - Deploy +and scale agents effortlessly with a purpose-built deployment platform for long +running, stateful workflows. Discover, reuse, configure, and share agents across +teams — and iterate quickly with visual prototyping in +[LangGraph Studio](https://langchain-ai.github.io/langgraph/concepts/langgraph_studio/). -### Open-source libraries - -- **`langchain-core`**: Base abstractions. -- **Integration packages** (e.g. **`langchain-openai`**, **`langchain-anthropic`**, etc.): Important integrations have been split into lightweight packages that are co-maintained by the LangChain team and the integration developers. -- **`langchain`**: Chains, agents, and retrieval strategies that make up an application's cognitive architecture. -- **`langchain-community`**: Third-party integrations that are community maintained. -- **[LangGraph](https://langchain-ai.github.io/langgraph)**: LangGraph powers production-grade agents, trusted by Linkedin, Uber, Klarna, GitLab, and many more. Build robust and stateful multi-actor applications with LLMs by modeling steps as edges and nodes in a graph. Integrates smoothly with LangChain, but can be used without it. To learn more about LangGraph, check out our first LangChain Academy course, *Introduction to LangGraph*, available [here](https://academy.langchain.com/courses/intro-to-langgraph). - -### Productionization: - -- **[LangSmith](https://docs.smith.langchain.com/)**: A developer platform that lets you debug, test, evaluate, and monitor chains built on any LLM framework and seamlessly integrates with LangChain. - -### Deployment: - -- **[LangGraph Platform](https://langchain-ai.github.io/langgraph/cloud/)**: Turn your LangGraph applications into production-ready APIs and Assistants. - -![Diagram outlining the hierarchical organization of the LangChain framework, displaying the interconnected parts across multiple layers.](docs/static/svg/langchain_stack_112024.svg#gh-light-mode-only "LangChain Architecture Overview") -![Diagram outlining the hierarchical organization of the LangChain framework, displaying the interconnected parts across multiple layers.](docs/static/svg/langchain_stack_112024_dark.svg#gh-dark-mode-only "LangChain Architecture Overview") - -## 🧱 What can you build with LangChain? - -**❓ Question answering with RAG** - -- [Documentation](https://python.langchain.com/docs/tutorials/rag/) -- End-to-end Example: [Chat LangChain](https://chat.langchain.com) and [repo](https://github.com/langchain-ai/chat-langchain) - -**🧱 Extracting structured output** - -- [Documentation](https://python.langchain.com/docs/tutorials/extraction/) -- End-to-end Example: [LangChain Extract](https://github.com/langchain-ai/langchain-extract/) - -**🤖 Chatbots** - -- [Documentation](https://python.langchain.com/docs/tutorials/chatbot/) -- End-to-end Example: [Web LangChain (web researcher chatbot)](https://weblangchain.vercel.app) and [repo](https://github.com/langchain-ai/weblangchain) - -And much more! Head to the [Tutorials](https://python.langchain.com/docs/tutorials/) section of the docs for more. - -## 🚀 How does LangChain help? - -The main value props of the LangChain libraries are: - -1. **Components**: composable building blocks, tools and integrations for working with language models. Components are modular and easy-to-use, whether you are using the rest of the LangChain framework or not. -2. **Easy orchestration with LangGraph**: [LangGraph](https://langchain-ai.github.io/langgraph/), -built on top of `langchain-core`, has built-in support for [messages](https://python.langchain.com/docs/concepts/messages/), [tools](https://python.langchain.com/docs/concepts/tools/), -and other LangChain abstractions. This makes it easy to combine components into -production-ready applications with persistence, streaming, and other key features. -Check out the LangChain [tutorials page](https://python.langchain.com/docs/tutorials/#orchestration) for examples. - -## Components - -Components fall into the following **modules**: - -**📃 Model I/O** - -This includes [prompt management](https://python.langchain.com/docs/concepts/prompt_templates/) -and a generic interface for [chat models](https://python.langchain.com/docs/concepts/chat_models/), including a consistent interface for [tool-calling](https://python.langchain.com/docs/concepts/tool_calling/) and [structured output](https://python.langchain.com/docs/concepts/structured_outputs/) across model providers. - -**📚 Retrieval** - -Retrieval Augmented Generation involves [loading data](https://python.langchain.com/docs/concepts/document_loaders/) from a variety of sources, [preparing it](https://python.langchain.com/docs/concepts/text_splitters/), then [searching over (a.k.a. retrieving from)](https://python.langchain.com/docs/concepts/retrievers/) it for use in the generation step. - -**🤖 Agents** - -Agents allow an LLM autonomy over how a task is accomplished. Agents make decisions about which Actions to take, then take that Action, observe the result, and repeat until the task is complete. [LangGraph](https://langchain-ai.github.io/langgraph/) makes it easy to use -LangChain components to build both [custom](https://langchain-ai.github.io/langgraph/tutorials/) -and [built-in](https://langchain-ai.github.io/langgraph/how-tos/create-react-agent/) -LLM agents. - -## 📖 Documentation - -Please see [here](https://python.langchain.com) for full documentation, which includes: - -- [Introduction](https://python.langchain.com/docs/introduction/): Overview of the framework and the structure of the docs. -- [Tutorials](https://python.langchain.com/docs/tutorials/): If you're looking to build something specific or are more of a hands-on learner, check out our tutorials. This is the best place to get started. -- [How-to guides](https://python.langchain.com/docs/how_to/): Answers to “How do I….?” type questions. These guides are goal-oriented and concrete; they're meant to help you complete a specific task. -- [Conceptual guide](https://python.langchain.com/docs/concepts/): Conceptual explanations of the key parts of the framework. -- [API Reference](https://python.langchain.com/api_reference/): Thorough documentation of every class and method. - -## 🌐 Ecosystem - -- [🦜🛠️ LangSmith](https://docs.smith.langchain.com/): Trace and evaluate your language model applications and intelligent agents to help you move from prototype to production. -- [🦜🕸️ LangGraph](https://langchain-ai.github.io/langgraph/): Create stateful, multi-actor applications with LLMs. Integrates smoothly with LangChain, but can be used without it. -- [🦜🕸️ LangGraph Platform](https://langchain-ai.github.io/langgraph/concepts/#langgraph-platform): Deploy LLM applications built with LangGraph into production. - -## 💁 Contributing - -As an open-source project in a rapidly developing field, we are extremely open to contributions, whether it be in the form of a new feature, improved infrastructure, or better documentation. - -For detailed information on how to contribute, see [here](https://python.langchain.com/docs/contributing/). - -## 🌟 Contributors - -[![langchain contributors](https://contrib.rocks/image?repo=langchain-ai/langchain&max=2000)](https://github.com/langchain-ai/langchain/graphs/contributors) +## Additional resources +- [Tutorials](https://python.langchain.com/docs/tutorials/): Simple walkthroughs with +guided examples on getting started with LangChain. +- [How-to Guides](https://python.langchain.com/docs/how_to/): Quick, actionable code +snippets for topics such as tool calling, RAG use cases, and more. +- [Conceptual Guides](https://python.langchain.com/docs/concepts/): Explanations of key +concepts behind the LangChain framework. +- [API Reference](https://python.langchain.com/api_reference/): Detailed reference on +navigating base packages and integrations for LangChain. diff --git a/docs/static/img/logo-dark.svg b/docs/static/img/logo-dark.svg new file mode 100644 index 00000000000..81fad4638e6 --- /dev/null +++ b/docs/static/img/logo-dark.svg @@ -0,0 +1,25 @@ + + + + + + + \ No newline at end of file diff --git a/docs/static/img/logo-light.svg b/docs/static/img/logo-light.svg new file mode 100644 index 00000000000..6ee90746271 --- /dev/null +++ b/docs/static/img/logo-light.svg @@ -0,0 +1,25 @@ + + + + + + + \ No newline at end of file