community[minor]: Add QuantizedEmbedders (#17391)

**Description:** 
* adding Quantized embedders using optimum-intel and
intel-extension-for-pytorch.
* added mdx documentation and example notebooks 
* added embedding import testing.

**Dependencies:** 
optimum = {extras = ["neural-compressor"], version = "^1.14.0", optional
= true}
intel_extension_for_pytorch = {version = "^2.2.0", optional = true}

Dependencies have been added to pyproject.toml for the community lib.  

**Twitter handle:** @peter_izsak

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
This commit is contained in:
Moshe Berchansky
2024-02-15 21:01:24 +02:00
committed by GitHub
parent bccc9241ea
commit 20a56fe0a2
6 changed files with 1029 additions and 0 deletions

View File

@@ -71,6 +71,7 @@ from langchain_community.embeddings.oci_generative_ai import OCIGenAIEmbeddings
from langchain_community.embeddings.octoai_embeddings import OctoAIEmbeddings
from langchain_community.embeddings.ollama import OllamaEmbeddings
from langchain_community.embeddings.openai import OpenAIEmbeddings
from langchain_community.embeddings.optimum_intel import QuantizedBiEncoderEmbeddings
from langchain_community.embeddings.sagemaker_endpoint import (
SagemakerEndpointEmbeddings,
)
@@ -149,6 +150,7 @@ __all__ = [
"BookendEmbeddings",
"VolcanoEmbeddings",
"OCIGenAIEmbeddings",
"QuantizedBiEncoderEmbeddings",
"NeMoEmbeddings",
]