github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-07-04 04:07:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: update NVIDIA Riva tool to use NVIDIA NIM for LLM (#22873)

Browse Source 
**Description:**
Update the NVIDIA Riva tool documentation to use NVIDIA NIM for the LLM.
Show how to use NVIDIA NIMs and link to documentation for LangChain with
NIM.

---------

Co-authored-by: Hayden Wolff <hwolff@nvidia.com>
Co-authored-by: Isaac Francisco <78627776+isahers1@users.noreply.github.com>
This commit is contained in:
Hayden Wolff 2024-06-13 19:26:05 -07:00 committed by GitHub
parent ada1e5cc64
commit d1cdde267a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 48 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
docs/docs/integrations/tools/nvidia_riva.ipynb
View File

@ -527,9 +527,54 @@
"## 6. Create Additional Chain Components\n",
"As usual, declare the other parts of the chain. In this case, it's just a prompt template and an LLM.\n",
"\n",
"You can use any [LangChain compatible LLM](https://python.langchain.com/v0.1/docs/integrations/llms/) in the chain. In this example, we use a [Mixtral8x7b NIM from NVIDIA](https://python.langchain.com/v0.2/docs/integrations/chat/nvidia_ai_endpoints/). NVIDIA NIMs are supported in LangChain via the `langchain-nvidia-ai-endpoints` package, so you can easily build applications with best in class throughput and latency. \n",
"\n",
"LangChain compatible NVIDIA LLMs from [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) can also be used by following these [instructions](https://python.langchain.com/docs/integrations/chat/nvidia_ai_endpoints). "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7fb27b941602401d91542211134fc71a",
"metadata": {},
"outputs": [],
"source": [
"%pip install --upgrade --quiet langchain-nvidia-ai-endpoints"
]
},
{
"cell_type": "markdown",
"id": "1744eec9",
"metadata": {},
"source": [
"Follow the [instructions for LangChain](https://python.langchain.com/v0.2/docs/integrations/chat/nvidia_ai_endpoints/) to use NVIDIA NIM in your speech-enabled LangChain application. \n",
"\n",
"Set your key for NVIDIA API catalog, where NIMs are hosted for you to try."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0e37bdab",
"metadata": {},
"outputs": [],
"source": [
"import getpass\n",
"import os\n",
"\n",
"nvapi_key = getpass.getpass(\"NVAPI Key (starts with nvapi-): \")\n",
"assert nvapi_key.startswith(\"nvapi-\"), f\"{nvapi_key[:5]}... is not a valid key\"\n",
"os.environ[\"NVIDIA_API_KEY\"] = nvapi_key"
]
},
{
"cell_type": "markdown",
"id": "c754acb0",
"metadata": {},
"source": [
"Instantiate LLM."
]
},
{
"cell_type": "code",
"execution_count": 7,
@ -538,10 +583,11 @@
"outputs": [],
"source": [
"from langchain_core.prompts import PromptTemplate\n",
"from langchain_openai import OpenAI\n",
"from langchain_nvidia_ai_endpoints import ChatNVIDIA\n",
"\n",
"prompt = PromptTemplate.from_template(\"{user_input}\")\n",
"llm = OpenAI(openai_api_key=\"sk-xxx\")"
"\n",
"llm = ChatNVIDIA(model=\"mistralai/mixtral-8x7b-instruct-v0.1\")"
]
},
{