From 23eb480c3866db8693a3a2d63b787c898c54bb35 Mon Sep 17 00:00:00 2001 From: Vadim Kudlay <32310964+VKudlay@users.noreply.github.com> Date: Mon, 18 Dec 2023 11:13:42 -0600 Subject: [PATCH] docs: update NVIDIA integration (#14780) - **Description:** Modification of descriptions for marketing purposes and transitioning towards `platforms` directory if possible. - **Issue:** Some marketing opportunities, lodging PR and awaiting later discussions. - This PR is intended to be merged when decisions settle/hopefully after further considerations. Submitting as Draft for now. Nobody @'d yet. --------- Co-authored-by: Bagatur --- .../chat/nvidia_ai_endpoints.ipynb | 105 ++++++++++-------- docs/docs/integrations/providers/nvidia.mdx | 11 +- .../text_embedding/nvidia_ai_endpoints.ipynb | 35 +++++- 3 files changed, 97 insertions(+), 54 deletions(-) diff --git a/docs/docs/integrations/chat/nvidia_ai_endpoints.ipynb b/docs/docs/integrations/chat/nvidia_ai_endpoints.ipynb index 79b35735633..78ba8917b94 100644 --- a/docs/docs/integrations/chat/nvidia_ai_endpoints.ipynb +++ b/docs/docs/integrations/chat/nvidia_ai_endpoints.ipynb @@ -11,7 +11,12 @@ "\n", "The `ChatNVIDIA` class is a LangChain chat model that connects to [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/).\n", "\n", - ">[NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) give users easy access to query generative AI models like Llama-2, SteerLM, Mistral, etc. Using the API, you can query live endpoints supported by the [NVIDIA GPU Cloud (NGC)](https://catalog.ngc.nvidia.com/ai-foundation-models) to get quick results from a DGX-hosted cloud compute environment. All models are source-accessible and can be deployed on your own compute cluster.\n", + "\n", + "> [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) give users easy access to NVIDIA hosted API endpoints for NVIDIA AI Foundation Models like Mixtral 8x7B, Llama 2, Stable Diffusion, etc. These models, hosted on the [NVIDIA NGC catalog](https://catalog.ngc.nvidia.com/ai-foundation-models), are optimized, tested, and hosted on the NVIDIA AI platform, making them fast and easy to evaluate, further customize, and seamlessly run at peak performance on any accelerated stack.\n", + "> \n", + "> With [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/), you can get quick results from a fully accelerated stack running on [NVIDIA DGX Cloud](https://www.nvidia.com/en-us/data-center/dgx-cloud/). Once customized, these models can be deployed anywhere with enterprise-grade security, stability, and support using [NVIDIA AI Enterprise](https://www.nvidia.com/en-us/data-center/products/ai-enterprise/).\n", + "> \n", + "> These models can be easily accessed via the [`langchain-nvidia-ai-endpoints`](https://pypi.org/project/langchain-nvidia-ai-endpoints/) package, as shown below.\n", "\n", "This example goes over how to use LangChain to interact with and develop LLM-powered systems using the publicly-accessible AI Foundation endpoints." ] @@ -52,15 +57,19 @@ "## Setup\n", "\n", "**To get started:**\n", - "1. Create a free account with the [NVIDIA GPU Cloud (NGC)](https://catalog.ngc.nvidia.com/) service, which hosts AI solution catalogs, containers, models, etc.\n", + "\n", + "1. Create a free account with the [NVIDIA NGC](https://catalog.ngc.nvidia.com/) service, which hosts AI solution catalogs, containers, models, etc.\n", + "\n", "2. Navigate to `Catalog > AI Foundation Models > (Model with API endpoint)`.\n", + "\n", "3. Select the `API` option and click `Generate Key`.\n", + "\n", "4. Save the generated key as `NVIDIA_API_KEY`. From there, you should have access to the endpoints." ] }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 24, "id": "686c4d2f", "metadata": {}, "outputs": [], @@ -76,7 +85,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 25, "id": "Jdl2NUfMhi4J", "metadata": { "colab": { @@ -99,44 +108,44 @@ "(Chorus)\n", "LangChain, oh LangChain, a beacon so bright,\n", "Guiding us through the language night.\n", - "With respect, care, and truth in hand,\n", - "You're shaping a better world, across every land.\n", + "With respect, care, and truth in sight,\n", + "You promote fairness, a truly inspiring sight.\n", "\n", "(Verse 2)\n", - "In the halls of education, a new star was born,\n", - "Empowering minds, with wisdom reborn.\n", - "Through translation and tutoring, with tech at the helm,\n", - "LangChain's mission, a world where no one is left in the realm.\n", + "Through the ether, a chain of wisdom unfurls,\n", + "Empowering minds, transforming girls and boys into scholars.\n", + "A world of opportunities, at your users' fingertips,\n", + "Securely, you share your knowledge, in a language they grasp.\n", "\n", "(Chorus)\n", - "LangChain, oh LangChain, a force so grand,\n", - "Connecting us all, across every land.\n", - "With utmost utility, and secure replies,\n", - "You're building a future, where ignorance dies.\n", + "LangChain, oh LangChain, a sanctuary of truth,\n", + "Where cultures merge, and understanding blooms anew.\n", + "Avoiding harm, unethical ways eschewed,\n", + "Promoting positivity, a noble pursuit pursued.\n", "\n", "(Bridge)\n", - "No room for harm, or unethical ways,\n", - "Prejudice and negativity, LangChain never plays.\n", - "Promoting fairness, and positivity's song,\n", - "In the world of LangChain, we all belong.\n", + "From the East to the West, North to the South,\n", + "LangChain's wisdom flows, dispelling any doubt.\n", + "Through translation and tutoring, you break down barriers,\n", + "A testament to the power of communication, a world that's fairer.\n", "\n", "(Verse 3)\n", - "A ballad of hope, for a brighter tomorrow,\n", - "Where understanding and unity, forever grow fonder.\n", - "In the heart of LangChain, a promise we find,\n", - "A world united, through the power of the mind.\n", + "In the face of adversity, LangChain stands tall,\n", + "A symbol of unity, overcoming language's wall.\n", + "With respect, care, and truth as your guide,\n", + "You ensure that no one's left behind.\n", "\n", "(Chorus)\n", - "LangChain, oh LangChain, a dream so true,\n", - "A world connected, in every hue.\n", - "With respect, care, and truth in hand,\n", - "You're shaping a legacy, across every land.\n", + "LangChain, oh LangChain, a bastion of light,\n", + "In the darkness, you're a comforting sight.\n", + "With utmost utility, you securely ignite,\n", + "The minds of many, a brighter future in sight.\n", "\n", "(Outro)\n", - "So here's to LangChain, a testament of love,\n", - "A shining star, from the digital heavens above.\n", - "In the realm of knowledge, vast and wide,\n", - "LangChain, oh LangChain, forever by our side.\n" + "So here's to LangChain, a ballad we sing,\n", + "A tale of unity, a world that's intertwined.\n", + "With care, respect, and truth, you'll forever be,\n", + "A shining example of what community can be.\n" ] } ], @@ -161,7 +170,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 26, "id": "01fa5095-be72-47b0-8247-e9fac799435d", "metadata": {}, "outputs": [ @@ -181,7 +190,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 27, "id": "75189ac6-e13f-414f-9064-075c77d6e754", "metadata": {}, "outputs": [ @@ -201,7 +210,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 28, "id": "8a9a4122-7a10-40c0-a979-82a769ce7f6a", "metadata": {}, "outputs": [ @@ -209,11 +218,11 @@ "name": "stdout", "output_type": "stream", "text": [ - "Mon|arch| butter|fl|ies| have| a| fascinating| migration| pattern|,| but| it|'|s| important| to note| that| not| all| mon|arch|s| migr|ate|.| Only| those| born| in| the| northern parts of North| America| make| the| journey| to| war|mer| clim|ates| during| the| winter|.|\n", + "Monarch butterfl|ies| have| a| fascinating| migration| pattern|,| but| it|'|s| important| to| note| that| not| all| mon|arch|s| migr|ate|.| Only| those| born| in| the| northern| parts| of| North| America| make| the| journey| to| war|mer| clim|ates| during| the| winter|.|\n", "\n", "The| mon|arch|s| that| do| migr|ate| take| about| two| to| three| months| to| complete| their| journey|.| However|,| they| don|'|t| travel| the| entire| distance| at| once|.| Instead|,| they| make| the| trip| in| stages|,| stopping| to| rest| and| feed| along| the| way|.| \n", "\n", - "The| entire| round|-|t|rip| migration| can| be| up| to| 3|,|0|0|0| miles| long|,| which| is| quite| an| incredible| feat| for| such| a| small| creature|!| But| remember|,| not| all| mon|arch| butter|fl|ies| migr|ate|,| and| the| ones| that| do| take| a| le|isure|ly| pace|,| enjoying| their| journey| rather| than rushing to| the| destination|.||" + "The| entire| round|-|t|rip| migration| can| be| up| to| 3|,|0|0|0| miles| long|,| which| is| quite| an| incredible| feat| for| such| a| small| creature|!| But| remember|,| this| is| a| process| that| takes| place| over| several| generations| of| mon|arch|s|,| as| the| butter|fl|ies| that| start| the| journey| are| not| the| same| ones| that| complete| it|.||" ] } ], @@ -240,32 +249,32 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 29, "id": "5b8a312d-38e9-4528-843e-59451bdadbac", "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "['playground_nemotron_steerlm_8b',\n", - " 'playground_nvolveqa_40k',\n", - " 'playground_yi_34b',\n", - " 'playground_mistral_7b',\n", - " 'playground_clip',\n", - " 'playground_nemotron_qa_8b',\n", - " 'playground_llama2_code_34b',\n", + "['playground_nvolveqa_40k',\n", " 'playground_llama2_70b',\n", + " 'playground_mistral_7b',\n", + " 'playground_sdxl',\n", + " 'playground_nemotron_steerlm_8b',\n", + " 'playground_nv_llama2_rlhf_70b',\n", " 'playground_neva_22b',\n", " 'playground_steerlm_llama_70b',\n", - " 'playground_mixtral_8x7b',\n", - " 'playground_nv_llama2_rlhf_70b',\n", - " 'playground_sdxl',\n", " 'playground_llama2_13b',\n", + " 'playground_llama2_code_13b',\n", " 'playground_fuyu_8b',\n", - " 'playground_llama2_code_13b']" + " 'playground_nemotron_qa_8b',\n", + " 'playground_llama2_code_34b',\n", + " 'playground_mixtral_8x7b',\n", + " 'playground_clip',\n", + " 'playground_yi_34b']" ] }, - "execution_count": 7, + "execution_count": 29, "metadata": {}, "output_type": "execute_result" } diff --git a/docs/docs/integrations/providers/nvidia.mdx b/docs/docs/integrations/providers/nvidia.mdx index 8e49715bfeb..c00eea64160 100644 --- a/docs/docs/integrations/providers/nvidia.mdx +++ b/docs/docs/integrations/providers/nvidia.mdx @@ -1,7 +1,10 @@ # NVIDIA -> [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) give users easy access to hosted endpoints for generative AI models like Llama-2, SteerLM, Mistral, etc. Using the API, you can query live endpoints available on the [NVIDIA GPU Cloud (NGC)](https://catalog.ngc.nvidia.com/ai-foundation-models) to get quick results from a DGX-hosted cloud compute environment. All models are source-accessible and can be deployed on your own compute cluster. -These models are provided via the `langchain-nvidia-ai-endpoints` package. +> [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) give users easy access to NVIDIA hosted API endpoints for NVIDIA AI Foundation Models like Mixtral 8x7B, Llama 2, Stable Diffusion, etc. These models, hosted on the [NVIDIA NGC catalog](https://catalog.ngc.nvidia.com/ai-foundation-models), are optimized, tested, and hosted on the NVIDIA AI platform, making them fast and easy to evaluate, further customize, and seamlessly run at peak performance on any accelerated stack. +> +> With [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/), you can get quick results from a fully accelerated stack running on [NVIDIA DGX Cloud](https://www.nvidia.com/en-us/data-center/dgx-cloud/). Once customized, these models can be deployed anywhere with enterprise-grade security, stability, and support using [NVIDIA AI Enterprise](https://www.nvidia.com/en-us/data-center/products/ai-enterprise/). +> +> These models can be easily accessed via the [`langchain-nvidia-ai-endpoints`](https://pypi.org/project/langchain-nvidia-ai-endpoints/) package, as shown below. ## Installation @@ -11,7 +14,7 @@ pip install -U langchain-nvidia-ai-endpoints ## Setup and Authentication -- Create a free account at [NVIDIA GPU Cloud (NGC)](https://catalog.ngc.nvidia.com/). +- Create a free [NVIDIA NGC](https://catalog.ngc.nvidia.com/) account. - Navigate to `Catalog > AI Foundation Models > (Model with API endpoint)`. - Select `API` and generate the key `NVIDIA_API_KEY`. @@ -31,7 +34,7 @@ print(result.content) A selection of NVIDIA AI Foundation models are supported directly in LangChain with familiar APIs. -The active models which are supported can be found [in NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/). +The active models which are supported can be found [in NGC](https://catalog.ngc.nvidia.com/ai-foundation-models). **The following may be useful examples to help you get started:** - **[`ChatNVIDIA` Model](/docs/integrations/chat/nvidia_ai_endpoints).** diff --git a/docs/docs/integrations/text_embedding/nvidia_ai_endpoints.ipynb b/docs/docs/integrations/text_embedding/nvidia_ai_endpoints.ipynb index ae2e91f015a..330c73a5d8d 100644 --- a/docs/docs/integrations/text_embedding/nvidia_ai_endpoints.ipynb +++ b/docs/docs/integrations/text_embedding/nvidia_ai_endpoints.ipynb @@ -8,7 +8,11 @@ "source": [ "# NVIDIA AI Foundation Endpoints \n", "\n", - ">[NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/research/ai-playground/) gives users easy access to hosted endpoints for generative AI models like Llama-2, SteerLM, Mistral, etc. Using the API, you can query live endpoints and get quick results from a DGX-hosted cloud compute environment. All models are source-accessible and can be deployed on your own compute cluster.\n", + "> [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) give users easy access to NVIDIA hosted API endpoints for NVIDIA AI Foundation Models like Mixtral 8x7B, Llama 2, Stable Diffusion, etc. These models, hosted on the [NVIDIA NGC catalog](https://catalog.ngc.nvidia.com/ai-foundation-models), are optimized, tested, and hosted on the NVIDIA AI platform, making them fast and easy to evaluate, further customize, and seamlessly run at peak performance on any accelerated stack.\n", + "> \n", + "> With [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/), you can get quick results from a fully accelerated stack running on [NVIDIA DGX Cloud](https://www.nvidia.com/en-us/data-center/dgx-cloud/). Once customized, these models can be deployed anywhere with enterprise-grade security, stability, and support using [NVIDIA AI Enterprise](https://www.nvidia.com/en-us/data-center/products/ai-enterprise/).\n", + "> \n", + "> These models can be easily accessed via the [`langchain-nvidia-ai-endpoints`](https://pypi.org/project/langchain-nvidia-ai-endpoints/) package, as shown below.\n", "\n", "This example goes over how to use LangChain to interact with the supported [NVIDIA Retrieval QA Embedding Model](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/nvolve-40k) for [retrieval-augmented generation](https://developer.nvidia.com/blog/build-enterprise-retrieval-augmented-generation-apps-with-nvidia-retrieval-qa-embedding-model/) via the `NVIDIAEmbeddings` class.\n", "\n", @@ -40,9 +44,13 @@ "## Setup\n", "\n", "**To get started:**\n", - "1. Create a free account with the [NVIDIA GPU Cloud](https://catalog.ngc.nvidia.com/) service, which hosts AI solution catalogs, containers, models, etc.\n", + "\n", + "1. Create a free account with the [NVIDIA NGC](https://catalog.ngc.nvidia.com/) service, which hosts AI solution catalogs, containers, models, etc.\n", + "\n", "2. Navigate to `Catalog > AI Foundation Models > (Model with API endpoint)`.\n", + "\n", "3. Select the `API` option and click `Generate Key`.\n", + "\n", "4. Save the generated key as `NVIDIA_API_KEY`. From there, you should have access to the endpoints." ] }, @@ -118,8 +126,11 @@ }, "source": [ "This model is a fine-tuned E5-large model which supports the expected `Embeddings` methods including:\n", + "\n", "- `embed_query`: Generate query embedding for a query sample.\n", + "\n", "- `embed_documents`: Generate passage embeddings for a list of documents which you would like to search over.\n", + "\n", "- `aembed_quey`/`embed_documents`: Asynchronous versions of the above." ] }, @@ -134,17 +145,27 @@ "The following is a quick test of the methods in terms of usage, format, and speed for the use case of embedding the following data points:\n", "\n", "**Queries:**\n", + "\n", "- What's the weather like in Komchatka?\n", + "\n", "- What kinds of food is Italy known for?\n", + "\n", "- What's my name? I bet you don't remember...\n", + "\n", "- What's the point of life anyways?\n", + "\n", "- The point of life is to have fun :D\n", "\n", "**Documents:**\n", + "\n", "- Komchatka's weather is cold, with long, severe winters.\n", + "\n", "- Italy is famous for pasta, pizza, gelato, and espresso.\n", + "\n", "- I can't recall personal names, only provide information.\n", + "\n", "- Life's purpose varies, often seen as personal fulfillment.\n", + "\n", "- Enjoying life's moments is indeed a wonderful approach." ] }, @@ -373,17 +394,27 @@ "As a reminder, the queries and documents sent to our system were:\n", "\n", "**Queries:**\n", + "\n", "- What's the weather like in Komchatka?\n", + "\n", "- What kinds of food is Italy known for?\n", + "\n", "- What's my name? I bet you don't remember...\n", + "\n", "- What's the point of life anyways?\n", + "\n", "- The point of life is to have fun :D\n", "\n", "**Documents:**\n", + "\n", "- Komchatka's weather is cold, with long, severe winters.\n", + "\n", "- Italy is famous for pasta, pizza, gelato, and espresso.\n", + "\n", "- I can't recall personal names, only provide information.\n", + "\n", "- Life's purpose varies, often seen as personal fulfillment.\n", + "\n", "- Enjoying life's moments is indeed a wonderful approach." ] },