Harrison/self hosted runhouse (#1154)

Co-authored-by: Donny Greenberg <dongreenberg2@gmail.com>
Co-authored-by: John Dagdelen <jdagdelen@users.noreply.github.com>
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
Co-authored-by: Andrew White <white.d.andrew@gmail.com>
Co-authored-by: Peng Qu <82029664+pengqu123@users.noreply.github.com>
Co-authored-by: Matt Robinson <mthw.wm.robinson@gmail.com>
Co-authored-by: jeff <tangj1122@gmail.com>
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MacBook-Pro.local>
Co-authored-by: zanderchase <zander@unfold.ag>
Co-authored-by: Charles Frye <cfrye59@gmail.com>
Co-authored-by: zanderchase <zanderchase@gmail.com>
Co-authored-by: Shahriar Tajbakhsh <sh.tajbakhsh@gmail.com>
Co-authored-by: Stefan Keselj <skeselj@princeton.edu>
Co-authored-by: Francisco Ingham <fpingham@gmail.com>
Co-authored-by: Dhruv Anand <105786647+dhruv-anand-aintech@users.noreply.github.com>
Co-authored-by: cragwolfe <cragcw@gmail.com>
Co-authored-by: Anton Troynikov <atroyn@users.noreply.github.com>
Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>
Co-authored-by: Oliver Klingefjord <oliver@klingefjord.com>
Co-authored-by: blob42 <contact@blob42.xyz>
Co-authored-by: blob42 <spike@w530>
Co-authored-by: Enrico Shippole <henryshippole@gmail.com>
Co-authored-by: Ibis Prevedello <ibiscp@gmail.com>
Co-authored-by: jped <jonathanped@gmail.com>
Co-authored-by: Justin Torre <justintorre75@gmail.com>
Co-authored-by: Ivan Vendrov <ivan@anthropic.com>
Co-authored-by: Sasmitha Manathunga <70096033+mmz-001@users.noreply.github.com>
Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>
Co-authored-by: Matt Robinson <mrobinson@unstructuredai.io>
Co-authored-by: Jeff Huber <jeffchuber@gmail.com>
Co-authored-by: Akshay <64036106+akshayvkt@users.noreply.github.com>
Co-authored-by: Andrew Huang <jhuang16888@gmail.com>
Co-authored-by: rogerserper <124558887+rogerserper@users.noreply.github.com>
Co-authored-by: seanaedmiston <seane999@gmail.com>
Co-authored-by: Hasegawa Yuya <52068175+Hase-U@users.noreply.github.com>
Co-authored-by: Ivan Vendrov <ivendrov@gmail.com>
Co-authored-by: Chen Wu (吴尘) <henrychenwu@cmu.edu>
Co-authored-by: Dennis Antela Martinez <dennis.antela@gmail.com>
Co-authored-by: Maxime Vidal <max.vidal@hotmail.fr>
Co-authored-by: Rishabh Raizada <110235735+rishabh-ti@users.noreply.github.com>

docs/modules/utils/combine_docs_examples/embeddings.ipynb

@@ -313,13 +313,156 @@
     "query_result = embeddings.embed_query(text)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "eec4efda",
+   "metadata": {},
+   "source": [
+    "## Self Hosted Embeddings\n",
+    "Let's load the SelfHostedEmbeddings, SelfHostedHuggingFaceEmbeddings, and SelfHostedHuggingFaceInstructEmbeddings classes."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "a961cdb5",
+   "id": "d338722a",
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "from langchain.embeddings import (\n",
+    "    SelfHostedEmbeddings, \n",
+    "    SelfHostedHuggingFaceEmbeddings, \n",
+    "    SelfHostedHuggingFaceInstructEmbeddings\n",
+    ")\n",
+    "import runhouse as rh"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "146559e8",
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "# For an on-demand A100 with GCP, Azure, or Lambda\n",
+    "gpu = rh.cluster(name=\"rh-a10x\", instance_type=\"A100:1\", use_spot=False)\n",
+    "\n",
+    "# For an on-demand A10G with AWS (no single A100s on AWS)\n",
+    "# gpu = rh.cluster(name='rh-a10x', instance_type='g5.2xlarge', provider='aws')\n",
+    "\n",
+    "# For an existing cluster\n",
+    "# gpu = rh.cluster(ips=['<ip of the cluster>'], \n",
+    "#                  ssh_creds={'ssh_user': '...', 'ssh_private_key':'<path_to_key>'},\n",
+    "#                  name='my-cluster')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1230f7df",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "embeddings = SelfHostedHuggingFaceEmbeddings(hardware=gpu)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "2684e928",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "text = \"This is a test document.\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1dc5e606",
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "query_result = embeddings.embed_query(text)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cef9cc54",
+   "metadata": {},
+   "source": [
+    "And similarly for SelfHostedHuggingFaceInstructEmbeddings:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "81a17ca3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "embeddings = SelfHostedHuggingFaceInstructEmbeddings(hardware=gpu)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5a33d1c8",
+   "metadata": {},
+   "source": [
+    "Now let's load an embedding model with a custom load function:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "c4af5679",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def get_pipeline():\n",
+    "    from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline  # Must be inside the function in notebooks\n",
+    "    model_id = \"facebook/bart-base\"\n",
+    "    tokenizer = AutoTokenizer.from_pretrained(model_id)\n",
+    "    model = AutoModelForCausalLM.from_pretrained(model_id)\n",
+    "    return pipeline(\"feature-extraction\", model=model, tokenizer=tokenizer)\n",
+    "\n",
+    "def inference_fn(pipeline, prompt):\n",
+    "    # Return last hidden state of the model\n",
+    "    if isinstance(prompt, list):\n",
+    "        return [emb[0][-1] for emb in pipeline(prompt)] \n",
+    "    return pipeline(prompt)[0][-1]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8654334b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "embeddings = SelfHostedEmbeddings(\n",
+    "    model_load_fn=get_pipeline, \n",
+    "    hardware=gpu,\n",
+    "    model_reqs=[\"./\", \"torch\", \"transformers\"],\n",
+    "    inference_fn=inference_fn\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fc1bfd0f",
+   "metadata": {
+    "scrolled": false
+   },
+   "outputs": [],
+   "source": [
+    "query_result = embeddings.embed_query(text)"
+   ]
   }
  ],
  "metadata": {
@@ -338,7 +481,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.9"
   },
   "vscode": {
    "interpreter": {