mirror of
https://github.com/hwchase17/langchain.git
synced 2025-10-02 02:38:36 +00:00
0606aabfa3
Thank you for contributing to LangChain! Changes: - docs: Added `WatsonxRerank` documentation - docs Updated `WatsonxEmbeddings` with docs template - docs: Updated `ChatWatsonx` with docs template - docs: Updated `WatsonxLLM` with docs template - docs: Added `ChatWatsonx` to list with Chat models providers. Added [test_chat_models_standard](https://github.com/langchain-ai/langchain-ibm/blob/main/libs/ibm/tests/integration_tests/test_chat_models_standard.py) to `langchain_ibm` tests suite. - docs: Added `IBM` to list with Embedding models providers. Added [test_embeddings_standard](https://github.com/langchain-ai/langchain-ibm/blob/main/libs/ibm/tests/integration_tests/test_embeddings_standard.py) to `langchain_ibm` tests suite. - docs: Updated `langcahin_ibm` recommended versions compatible with `LangChain v0.3` --------- Co-authored-by: Erick Friis <erick@langchain.dev>
443 lines
13 KiB
Plaintext
443 lines
13 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "70996d8a",
|
||
"metadata": {},
|
||
"source": [
|
||
"# IBM watsonx.ai\n",
|
||
"\n",
|
||
">[WatsonxLLM](https://ibm.github.io/watsonx-ai-python-sdk/fm_extensions.html#langchain) is a wrapper for IBM [watsonx.ai](https://www.ibm.com/products/watsonx-ai) foundation models.\n",
|
||
"\n",
|
||
"This example shows how to communicate with `watsonx.ai` models using `LangChain`."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5b8d9390",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Overview\n",
|
||
"\n",
|
||
"### Integration details\n",
|
||
"| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/llms/ibm/) | Package downloads | Package latest |\n",
|
||
"| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n",
|
||
"| [WatsonxLLM](https://python.langchain.com/api_reference/ibm/llms/langchain_ibm.llms.WatsonxLLM.html) | [langchain-ibm](https://python.langchain.com/api_reference/ibm/index.html) | ❌ | ❌ | ✅ |  |  |"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ea35b2b7",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Setup\n",
|
||
"\n",
|
||
"To access IBM watsonx.ai models you'll need to create an IBM watsonx.ai account, get an API key, and install the `langchain-ibm` integration package.\n",
|
||
"\n",
|
||
"### Credentials\n",
|
||
"\n",
|
||
"The cell below defines the credentials required to work with watsonx Foundation Model inferencing.\n",
|
||
"\n",
|
||
"**Action:** Provide the IBM Cloud user API key. For details, see\n",
|
||
"[Managing user API keys](https://cloud.ibm.com/docs/account?topic=account-userapikey&interface=ui)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "11d572a1",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import os\n",
|
||
"from getpass import getpass\n",
|
||
"\n",
|
||
"watsonx_api_key = getpass()\n",
|
||
"os.environ[\"WATSONX_APIKEY\"] = watsonx_api_key"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c59782a7",
|
||
"metadata": {},
|
||
"source": [
|
||
"Additionaly you are able to pass additional secrets as an environment variable. "
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "f98c573c",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import os\n",
|
||
"\n",
|
||
"os.environ[\"WATSONX_URL\"] = \"your service instance url\"\n",
|
||
"os.environ[\"WATSONX_TOKEN\"] = \"your token for accessing the CPD cluster\"\n",
|
||
"os.environ[\"WATSONX_PASSWORD\"] = \"your password for accessing the CPD cluster\"\n",
|
||
"os.environ[\"WATSONX_USERNAME\"] = \"your username for accessing the CPD cluster\"\n",
|
||
"os.environ[\"WATSONX_INSTANCE_ID\"] = \"your instance_id for accessing the CPD cluster\""
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f918d229",
|
||
"metadata": {},
|
||
"source": [
|
||
"### Installation\n",
|
||
"\n",
|
||
"The LangChain IBM integration lives in the `langchain-ibm` package:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "f925c9aa",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"!pip install -qU langchain-ibm"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e36acbef",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Instantiation\n",
|
||
"\n",
|
||
"You might need to adjust model `parameters` for different models or tasks. For details, refer to [documentation](https://ibm.github.io/watsonx-ai-python-sdk/fm_model.html#metanames.GenTextParamsMetaNames)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "407cd500",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"parameters = {\n",
|
||
" \"decoding_method\": \"sample\",\n",
|
||
" \"max_new_tokens\": 100,\n",
|
||
" \"min_new_tokens\": 1,\n",
|
||
" \"temperature\": 0.5,\n",
|
||
" \"top_k\": 50,\n",
|
||
" \"top_p\": 1,\n",
|
||
"}"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2b586538",
|
||
"metadata": {},
|
||
"source": [
|
||
"Initialize the `WatsonxLLM` class with previously set parameters.\n",
|
||
"\n",
|
||
"\n",
|
||
"**Note**: \n",
|
||
"\n",
|
||
"- To provide context for the API call, you must add `project_id` or `space_id`. For more information see [documentation](https://www.ibm.com/docs/en/watsonx-as-a-service?topic=projects).\n",
|
||
"- Depending on the region of your provisioned service instance, use one of the urls described [here](https://ibm.github.io/watsonx-ai-python-sdk/setup_cloud.html#authentication).\n",
|
||
"\n",
|
||
"In this example, we’ll use the `project_id` and Dallas url.\n",
|
||
"\n",
|
||
"\n",
|
||
"You need to specify `model_id` that will be used for inferencing. All available models you can find in [documentation](https://ibm.github.io/watsonx-ai-python-sdk/fm_model.html#TextModels)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "359898de",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from langchain_ibm import WatsonxLLM\n",
|
||
"\n",
|
||
"watsonx_llm = WatsonxLLM(\n",
|
||
" model_id=\"ibm/granite-13b-instruct-v2\",\n",
|
||
" url=\"https://us-south.ml.cloud.ibm.com\",\n",
|
||
" project_id=\"PASTE YOUR PROJECT_ID HERE\",\n",
|
||
" params=parameters,\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2202f4e0",
|
||
"metadata": {},
|
||
"source": [
|
||
"Alternatively you can use Cloud Pak for Data credentials. For details, see [documentation](https://ibm.github.io/watsonx-ai-python-sdk/setup_cpd.html). "
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "243ecccb",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"watsonx_llm = WatsonxLLM(\n",
|
||
" model_id=\"ibm/granite-13b-instruct-v2\",\n",
|
||
" url=\"PASTE YOUR URL HERE\",\n",
|
||
" username=\"PASTE YOUR USERNAME HERE\",\n",
|
||
" password=\"PASTE YOUR PASSWORD HERE\",\n",
|
||
" instance_id=\"openshift\",\n",
|
||
" version=\"4.8\",\n",
|
||
" project_id=\"PASTE YOUR PROJECT_ID HERE\",\n",
|
||
" params=parameters,\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "96ed13d4",
|
||
"metadata": {},
|
||
"source": [
|
||
"Instead of `model_id`, you can also pass the `deployment_id` of the previously tuned model. The entire model tuning workflow is described [here](https://ibm.github.io/watsonx-ai-python-sdk/pt_working_with_class_and_prompt_tuner.html)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "08e66c88",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"watsonx_llm = WatsonxLLM(\n",
|
||
" deployment_id=\"PASTE YOUR DEPLOYMENT_ID HERE\",\n",
|
||
" url=\"https://us-south.ml.cloud.ibm.com\",\n",
|
||
" project_id=\"PASTE YOUR PROJECT_ID HERE\",\n",
|
||
" params=parameters,\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e4a1e0f1",
|
||
"metadata": {},
|
||
"source": [
|
||
"For certain requirements, there is an option to pass the IBM's [`APIClient`](https://ibm.github.io/watsonx-ai-python-sdk/base.html#apiclient) object into the `WatsonxLLM` class."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "4b28afc1",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from ibm_watsonx_ai import APIClient\n",
|
||
"\n",
|
||
"api_client = APIClient(...)\n",
|
||
"\n",
|
||
"watsonx_llm = WatsonxLLM(\n",
|
||
" model_id=\"ibm/granite-13b-instruct-v2\",\n",
|
||
" watsonx_client=api_client,\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "7c4a632b",
|
||
"metadata": {},
|
||
"source": [
|
||
"You can also pass the IBM's [`ModelInference`](https://ibm.github.io/watsonx-ai-python-sdk/fm_model_inference.html) object into the `WatsonxLLM` class."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "5335b148",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from ibm_watsonx_ai.foundation_models import ModelInference\n",
|
||
"\n",
|
||
"model = ModelInference(...)\n",
|
||
"\n",
|
||
"watsonx_llm = WatsonxLLM(watsonx_model=model)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f571001d",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Invocation\n",
|
||
"To obtain completions, you can call the model directly using a string prompt."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "beea2b5b",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"\"Man's best friend is his dog. Dogs are man's best friend because they are always there for you, they never judge you, and they love you unconditionally. Dogs are also great companions and can help reduce stress levels. \""
|
||
]
|
||
},
|
||
"execution_count": 3,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# Calling a single prompt\n",
|
||
"\n",
|
||
"watsonx_llm.invoke(\"Who is man's best friend?\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "8ab1a25a",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"LLMResult(generations=[[Generation(text='The fastest dog in the world is the greyhound. Greyhounds can run up to 45 mph, which is about the same speed as a Usain Bolt.', generation_info={'finish_reason': 'eos_token'})], [Generation(text='The Labrador Retriever is a breed of retriever that was bred for hunting. They are a very smart breed and are very easy to train. They are also very loyal and will make great companions. ', generation_info={'finish_reason': 'eos_token'})]], llm_output={'token_usage': {'generated_token_count': 82, 'input_token_count': 13}, 'model_id': 'ibm/granite-13b-instruct-v2', 'deployment_id': None}, run=[RunInfo(run_id=UUID('750b8a0f-8846-456d-93d0-e039e95b1276')), RunInfo(run_id=UUID('aa4c2a1c-5b08-4fcf-87aa-50228de46db5'))], type='LLMResult')"
|
||
]
|
||
},
|
||
"execution_count": 4,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# Calling multiple prompts\n",
|
||
"\n",
|
||
"watsonx_llm.generate(\n",
|
||
" [\n",
|
||
" \"The fastest dog in the world?\",\n",
|
||
" \"Describe your chosen dog breed\",\n",
|
||
" ]\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d2c9da33",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Streaming the Model output \n",
|
||
"\n",
|
||
"You can stream the model output."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"id": "3f63166a",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"My favorite breed of dog is a Labrador Retriever. They are my favorite breed because they are my favorite color, yellow. They are also very smart and easy to train. "
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"for chunk in watsonx_llm.stream(\n",
|
||
" \"Describe your favorite breed of dog and why it is your favorite.\"\n",
|
||
"):\n",
|
||
" print(chunk, end=\"\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "9fc88fdd",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Chaining\n",
|
||
"Create `PromptTemplate` objects which will be responsible for creating a random question."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "ad63fa27",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from langchain_core.prompts import PromptTemplate\n",
|
||
"\n",
|
||
"template = \"Generate a random question about {topic}: Question: \"\n",
|
||
"\n",
|
||
"prompt = PromptTemplate.from_template(template)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "677699db",
|
||
"metadata": {},
|
||
"source": [
|
||
"Provide a topic and run the chain."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"id": "868af75c",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"'What is the origin of the name \"Pomeranian\"?'"
|
||
]
|
||
},
|
||
"execution_count": 7,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"llm_chain = prompt | watsonx_llm\n",
|
||
"\n",
|
||
"topic = \"dog\"\n",
|
||
"\n",
|
||
"llm_chain.invoke(topic)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "59480270",
|
||
"metadata": {},
|
||
"source": [
|
||
"## API reference\n",
|
||
"\n",
|
||
"For detailed documentation of all `WatsonxLLM` features and configurations head to the [API reference](https://python.langchain.com/api_reference/ibm/llms/langchain_ibm.llms.WatsonxLLM.html)."
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.10.14"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|