community[minor]: Add QuantizedEmbedders (#17391)

**Description:** 
* adding Quantized embedders using optimum-intel and
intel-extension-for-pytorch.
* added mdx documentation and example notebooks 
* added embedding import testing.

**Dependencies:** 
optimum = {extras = ["neural-compressor"], version = "^1.14.0", optional
= true}
intel_extension_for_pytorch = {version = "^2.2.0", optional = true}

Dependencies have been added to pyproject.toml for the community lib.  

**Twitter handle:** @peter_izsak

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
This commit is contained in:
Moshe Berchansky
2024-02-15 21:01:24 +02:00
committed by GitHub
parent bccc9241ea
commit 20a56fe0a2
6 changed files with 1029 additions and 0 deletions

View File

@@ -58,6 +58,7 @@ EXPECTED_ALL = [
"BookendEmbeddings",
"VolcanoEmbeddings",
"OCIGenAIEmbeddings",
"QuantizedBiEncoderEmbeddings",
"NeMoEmbeddings",
]