mirror of
https://github.com/hwchase17/langchain.git
synced 2025-05-13 02:57:22 +00:00
135 lines
3.2 KiB
Plaintext
135 lines
3.2 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ed47bb62",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Sentence Transformers on Hugging Face\n",
|
|
"\n",
|
|
">[Hugging Face sentence-transformers](https://huggingface.co/sentence-transformers) is a Python framework for state-of-the-art sentence, text and image embeddings.\n",
|
|
">You can use these embedding models from the `HuggingFaceEmbeddings` class.\n",
|
|
"\n",
|
|
":::caution\n",
|
|
"\n",
|
|
"Running sentence-transformers locally can be affected by your operating system and other global factors. It is recommended for experienced users only.\n",
|
|
"\n",
|
|
":::\n",
|
|
"\n",
|
|
"## Setup\n",
|
|
"\n",
|
|
"You'll need to install the `langchain_huggingface` package as a dependency:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "06c9f47d",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install -qU langchain-huggingface"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8fb16f74",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Usage"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"id": "ff9be586",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[-0.038338568061590195, 0.12346471101045609, -0.028642969205975533, 0.05365273356437683, 0.008845377...\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain_huggingface import HuggingFaceEmbeddings\n",
|
|
"\n",
|
|
"embeddings = HuggingFaceEmbeddings(model_name=\"all-MiniLM-L6-v2\")\n",
|
|
"\n",
|
|
"text = \"This is a test document.\"\n",
|
|
"query_result = embeddings.embed_query(text)\n",
|
|
"\n",
|
|
"# show only the first 100 characters of the stringified vector\n",
|
|
"print(str(query_result)[:100] + \"...\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"id": "bb5e74c0",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[[-0.038338497281074524, 0.12346471846103668, -0.028642890974879265, 0.05365274101495743, 0.00884535...\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"doc_result = embeddings.embed_documents([text, \"This is not a test document.\"])\n",
|
|
"print(str(doc_result)[:100] + \"...\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1e6525cb",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Troubleshooting\n",
|
|
"\n",
|
|
"If you are having issues with the `accelerate` package not being found or failing to import, installing/upgrading it may help:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "bbae70f7",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install -qU accelerate"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.5"
|
|
},
|
|
"vscode": {
|
|
"interpreter": {
|
|
"hash": "7377c2ccc78bc62c2683122d48c8cd1fb85a53850a1b1fc29736ed39852c9885"
|
|
}
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|