github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-09-06 13:33:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

community[minor]: add support for Guardrails for Amazon Bedrock (#15099)

Browse Source 
Added support for optionally supplying 'Guardrails for Amazon Bedrock'
on both types of model invocations (batch/regular and streaming) and for
all models supported by the Amazon Bedrock service.

@baskaryan  @hwchase17

```python 
llm = Bedrock(model_id="<model_id>", client=bedrock,
                  model_kwargs={},
                  guardrails={"id": " <guardrail_id>",
                              "version": "<guardrail_version>",
                               "trace": True}, callbacks=[BedrockAsyncCallbackHandler()])

class BedrockAsyncCallbackHandler(AsyncCallbackHandler):
    """Async callback handler that can be used to handle callbacks from langchain."""

    async def on_llm_error(
            self,
            error: BaseException,
            **kwargs: Any,
    ) -> Any:
        reason = kwargs.get("reason")
        if reason == "GUARDRAIL_INTERVENED":
           # kwargs contains additional trace information sent by 'Guardrails for Bedrock' service.
            print(f"""Guardrails: {kwargs}""")


# streaming 
llm = Bedrock(model_id="<model_id>", client=bedrock,
                  model_kwargs={},
                  streaming=True,
                  guardrails={"id": "<guardrail_id>",
                              "version": "<guardrail_version>"})
```

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
This commit is contained in:
Harel Gal
2024-01-25 00:44:19 +02:00
committed by GitHub
parent 04651f0248
commit a91181fe6d
3 changed files with 375 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
docs/docs/integrations/llms/bedrock.ipynb
View File

@@ -106,6 +106,45 @@
"\n",
"conversation.predict(input=\"Hi there!\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Guardrails for Amazon Bedrock example \n",
"\n",
"In this section, we are going to set up a Bedrock language model with specific guardrails that include tracing capabilities. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from typing import Any\n",
"\n",
"from langchain_core.callbacks import AsyncCallbackHandler\n",
"\n",
"\n",
"class BedrockAsyncCallbackHandler(AsyncCallbackHandler):\n",
" # Async callback handler that can be used to handle callbacks from langchain.\n",
"\n",
" async def on_llm_error(self, error: BaseException, **kwargs: Any) -> Any:\n",
" reason = kwargs.get(\"reason\")\n",
" if reason == \"GUARDRAIL_INTERVENED\":\n",
" print(f\"Guardrails: {kwargs}\")\n",
"\n",
"\n",
"# guardrails for Amazon Bedrock with trace\n",
"llm = Bedrock(\n",
" credentials_profile_name=\"bedrock-admin\",\n",
" model_id=\"<Model_ID>\",\n",
" model_kwargs={},\n",
" guardrails={\"id\": \"<Guardrail_ID>\", \"version\": \"<Version>\", \"trace\": True},\n",
" callbacks=[BedrockAsyncCallbackHandler()],\n",
")"
]
}
],
"metadata": {