community: Added functions to make async calls to HuggingFaceHub's embedding endpoint in HuggingFaceHubEmbeddings class (#15737)

**Description:**
Added aembed_documents() and aembed_query() async functions in
HuggingFaceHubEmbeddings class in
langchain_community\embeddings\huggingface_hub.py file. It will support
to make async calls to HuggingFaceHub's
embedding endpoint and generate embeddings asynchronously.

Test Cases: Added test_huggingfacehub_embedding_async_documents() and
test_huggingfacehub_embedding_async_query()
functions in test_huggingface_hub.py file to test the two async
functions created in HuggingFaceHubEmbeddings class.

Documentation: Updated huggingfacehub.ipynb with steps to install
huggingface_hub package and use
HuggingFaceHubEmbeddings.

**Dependencies:** None,
**Twitter handle:** I do not have a Twitter account

---------

Co-authored-by: H161961 <Raunak.Raunak@Honeywell.com>
This commit is contained in:
Raunak
2024-01-12 11:22:55 +05:30
committed by GitHub
parent eb9b334a6b
commit e26e1f8b37
3 changed files with 126 additions and 2 deletions

View File

@@ -106,7 +106,7 @@
"metadata": {},
"outputs": [
{
"name": "stdin",
"name": "stdout",
"output_type": "stream",
"text": [
"Enter your HF Inference API Key:\n",
@@ -148,6 +148,75 @@
"query_result = embeddings.embed_query(text)\n",
"query_result[:3]"
]
},
{
"cell_type": "markdown",
"id": "19ef2d31",
"metadata": {},
"source": [
"## Hugging Face Hub\n",
"We can also generate embeddings locally via the Hugging Face Hub package, which requires us to install ``huggingface_hub ``"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "39e85945",
"metadata": {},
"outputs": [],
"source": [
"!pip install huggingface_hub"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c78a2779",
"metadata": {},
"outputs": [],
"source": [
"from langchain_community.embeddings import HuggingFaceHubEmbeddings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "116f3ce7",
"metadata": {},
"outputs": [],
"source": [
"embeddings = HuggingFaceHubEmbeddings()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d6f97ee9",
"metadata": {},
"outputs": [],
"source": [
"text = \"This is a test document.\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fb6adc67",
"metadata": {},
"outputs": [],
"source": [
"query_result = embeddings.embed_query(text)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1f42c311",
"metadata": {},
"outputs": [],
"source": [
"query_result[:3]"
]
}
],
"metadata": {