langchain/docs/versioned_docs/version-0.2.x/integrations/text_embedding/nemo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "abede47c-6a58-40c3-b7ef-10966a4fc085",
   "metadata": {},
   "source": [
    "# NVIDIA NeMo embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38f3d4ce-b36a-48c6-88b0-5970c26bb146",
   "metadata": {},
   "source": [
    "Connect to NVIDIA's embedding service using the `NeMoEmbeddings` class.\n",
    "\n",
    "The NeMo Retriever Embedding Microservice (NREM) brings the power of state-of-the-art text embedding to your applications, providing unmatched natural language processing and understanding capabilities. Whether you're developing semantic search, Retrieval Augmented Generation (RAG) pipelines—or any application that needs to use text embeddings—NREM has you covered. Built on the NVIDIA software platform incorporating CUDA, TensorRT, and Triton, NREM brings state of the art GPU accelerated Text Embedding model serving.\n",
    "\n",
    "NREM uses NVIDIA's TensorRT built on top of the Triton Inference Server for optimized inference of text embedding models."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5ab6ea1-d074-4f36-ae45-50312a6a82b9",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "32deab16-530d-455c-b40c-914db048cb05",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.embeddings import NeMoEmbeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de40023c-3391-474d-96cf-fbfb2311e9d7",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "37177018-47f4-48be-8575-83ce5c9a5447",
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size = 16\n",
    "model = \"NV-Embed-QA-003\"\n",
    "api_endpoint_url = \"http://localhost:8080/v1/embeddings\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "08161ed2-8ba3-4226-a387-15c348f8c343",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Checking if endpoint is live: http://localhost:8080/v1/embeddings\n"
     ]
    }
   ],
   "source": [
    "embedding_model = NeMoEmbeddings(\n",
    "    batch_size=batch_size, model=model, api_endpoint_url=api_endpoint_url\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c69070c3-fe2d-4ff7-be4a-73304e2c4f3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_model.embed_query(\"This is a test.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5d1d8852-5298-40b5-89c4-5a91ccfc95e5",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}