diff --git a/docs/docs/integrations/tools/nvidia_riva.ipynb b/docs/docs/integrations/tools/nvidia_riva.ipynb index a4cf2f299ae..1cc5b9c964b 100644 --- a/docs/docs/integrations/tools/nvidia_riva.ipynb +++ b/docs/docs/integrations/tools/nvidia_riva.ipynb @@ -527,9 +527,54 @@ "## 6. Create Additional Chain Components\n", "As usual, declare the other parts of the chain. In this case, it's just a prompt template and an LLM.\n", "\n", + "You can use any [LangChain compatible LLM](https://python.langchain.com/v0.1/docs/integrations/llms/) in the chain. In this example, we use a [Mixtral8x7b NIM from NVIDIA](https://python.langchain.com/v0.2/docs/integrations/chat/nvidia_ai_endpoints/). NVIDIA NIMs are supported in LangChain via the `langchain-nvidia-ai-endpoints` package, so you can easily build applications with best in class throughput and latency. \n", + "\n", "LangChain compatible NVIDIA LLMs from [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) can also be used by following these [instructions](https://python.langchain.com/docs/integrations/chat/nvidia_ai_endpoints). " ] }, + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install --upgrade --quiet langchain-nvidia-ai-endpoints" + ] + }, + { + "cell_type": "markdown", + "id": "1744eec9", + "metadata": {}, + "source": [ + "Follow the [instructions for LangChain](https://python.langchain.com/v0.2/docs/integrations/chat/nvidia_ai_endpoints/) to use NVIDIA NIM in your speech-enabled LangChain application. \n", + "\n", + "Set your key for NVIDIA API catalog, where NIMs are hosted for you to try." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0e37bdab", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "nvapi_key = getpass.getpass(\"NVAPI Key (starts with nvapi-): \")\n", + "assert nvapi_key.startswith(\"nvapi-\"), f\"{nvapi_key[:5]}... is not a valid key\"\n", + "os.environ[\"NVIDIA_API_KEY\"] = nvapi_key" + ] + }, + { + "cell_type": "markdown", + "id": "c754acb0", + "metadata": {}, + "source": [ + "Instantiate LLM." + ] + }, { "cell_type": "code", "execution_count": 7, @@ -538,10 +583,11 @@ "outputs": [], "source": [ "from langchain_core.prompts import PromptTemplate\n", - "from langchain_openai import OpenAI\n", + "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n", "\n", "prompt = PromptTemplate.from_template(\"{user_input}\")\n", - "llm = OpenAI(openai_api_key=\"sk-xxx\")" + "\n", + "llm = ChatNVIDIA(model=\"mistralai/mixtral-8x7b-instruct-v0.1\")" ] }, {