github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-06-03 05:34:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
06f47678ae
langchain/libs/community/langchain_community/embeddings
History
keval dekivadiya 06f47678ae
community[minor]: Add TextEmbed Embedding Integration (#22946) 
**Description:**

**TextEmbed** is a high-performance embedding inference server designed
to provide a high-throughput, low-latency solution for serving
embeddings. It supports various sentence-transformer models and includes
the ability to deploy image and text embedding models. TextEmbed offers
flexibility and scalability for diverse applications.

- **PyPI Package:** [TextEmbed on
PyPI](https://pypi.org/project/textembed/)
- **Docker Image:** [TextEmbed on Docker
Hub](https://hub.docker.com/r/kevaldekivadiya/textembed)
- **GitHub Repository:** [TextEmbed on
GitHub](https://github.com/kevaldekivadiya2415/textembed)

**PR Description**
This PR adds functionality for embedding documents and queries using the
`TextEmbedEmbeddings` class. The implementation allows for both
synchronous and asynchronous embedding requests to a TextEmbed API
endpoint. The class handles batching and permuting of input texts to
optimize the embedding process.

**Example Usage:**

```python
from langchain_community.embeddings import TextEmbedEmbeddings

# Initialise the embeddings class
embeddings = TextEmbedEmbeddings(model="your-model-id", api_key="your-api-key", api_url="your_api_url")

# Define a list of documents
documents = [
    "Data science involves extracting insights from data.",
    "Artificial intelligence is transforming various industries.",
    "Cloud computing provides scalable computing resources over the internet.",
    "Big data analytics helps in understanding large datasets.",
    "India has a diverse cultural heritage."
]

# Define a query
query = "What is the cultural heritage of India?"

# Embed all documents
document_embeddings = embeddings.embed_documents(documents)

# Embed the query
query_embedding = embeddings.embed_query(query)

# Print embeddings for each document
for i, embedding in enumerate(document_embeddings):
    print(f"Document {i+1} Embedding:", embedding)

# Print the query embedding
print("Query Embedding:", query_embedding)

---------

Co-authored-by: Eugene Yurtsev <eugene@langchain.dev>
2024-07-19 17:30:25 +00:00
..
__init__.py community[minor]: Add TextEmbed Embedding Integration (#22946) 2024-07-19 17:30:25 +00:00
aleph_alpha.py community[patch]: Update @root_validators to use explicit pre=True or pre=False (#23737) 2024-07-02 10:47:21 -04:00
anyscale.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
ascend.py docs: Spell check fixes (#24217) 2024-07-15 15:51:43 +00:00
awa.py community[patch]: Update @root_validators to use explicit pre=True or pre=False (#23737) 2024-07-02 10:47:21 -04:00
azure_openai.py fix azure_openai.py: some keys do not exists (#24158) 2024-07-15 17:17:05 +00:00
baichuan.py community: Add Baichuan Embeddings batch size (#22942) 2024-06-17 14:11:04 -04:00
baidu_qianfan_endpoint.py community[patch]: change the class of qianfan_ak and qianfan_sk parameters (#24293) 2024-07-16 09:17:48 -04:00
bedrock.py community[patch]: Update @root_validators to use explicit pre=True or pre=False (#23737) 2024-07-02 10:47:21 -04:00
bookend.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
clarifai.py community[patch]: Update @root_validators to use explicit pre=True or pre=False (#23737) 2024-07-02 10:47:21 -04:00
cloudflare_workersai.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
clova.py community[minor]: Add Clova Embeddings for LangChain Community (#21890) 2024-05-22 22:08:47 +00:00
cohere.py community[patch]: Update @root_validators to use explicit pre=True or pre=False (#23737) 2024-07-02 10:47:21 -04:00
dashscope.py community[patch]: Update root_validators embeddings: llamacpp, jina, dashscope, mosaicml, huggingface_hub, Toolkits: Connery, ChatModels: PAI_EAS, (#22828) 2024-06-12 13:59:05 -04:00
databricks.py community[patch]: docstrings update (#20301) 2024-04-11 16:23:27 -04:00
deepinfra.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
edenai.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
elasticsearch.py infra: rm unused # noqa violations (#22049) 2024-05-22 15:21:08 -07:00
embaas.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
ernie.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
fake.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
fastembed.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
gigachat.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
google_palm.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
gpt4all.py community: add model_name param valid for GPT4AllEmbeddings (#23867) 2024-07-05 10:46:34 -04:00
gradient_ai.py infra: add -p to mkdir in lint steps (#17013) 2024-02-05 11:22:06 -08:00
huggingface_hub.py huggingface: Fix huggingface tei support (#22653) 2024-07-03 13:30:29 -07:00
huggingface.py community[minor]: implement huggingface show_progress consistently (#22682) 2024-06-12 17:30:56 +00:00
infinity_local.py community[patch]: docstrings update (#20301) 2024-04-11 16:23:27 -04:00
infinity.py infra: rm unused # noqa violations (#22049) 2024-05-22 15:21:08 -07:00
ipex_llm.py community[minor]: Add IPEX-LLM BGE embedding support on both Intel CPU and GPU (#22226) 2024-06-03 12:37:10 -07:00
itrex.py community[minor]: Add ITREX optimized Embeddings (#18474) 2024-03-27 07:22:06 +00:00
javelin_ai_gateway.py community[patch]: docstrings update (#20301) 2024-04-11 16:23:27 -04:00
jina.py community: docstrings (#23202) 2024-06-20 11:08:13 -04:00
johnsnowlabs.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
laser.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
llamacpp.py community[patch]: Update root_validators embeddings: llamacpp, jina, dashscope, mosaicml, huggingface_hub, Toolkits: Connery, ChatModels: PAI_EAS, (#22828) 2024-06-12 13:59:05 -04:00
llamafile.py community[minor]: llamafile embeddings support (#17976) 2024-03-01 13:49:18 -08:00
llm_rails.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
localai.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
minimax.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
mlflow_gateway.py community[patch]: docstrings update (#20301) 2024-04-11 16:23:27 -04:00
mlflow.py docs: fix databricks document url (#19096) 2024-03-15 22:25:11 +00:00
modelscope_hub.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
mosaicml.py community[patch]: Update root_validators embeddings: llamacpp, jina, dashscope, mosaicml, huggingface_hub, Toolkits: Connery, ChatModels: PAI_EAS, (#22828) 2024-06-12 13:59:05 -04:00
nemo.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
nlpcloud.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
oci_generative_ai.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
octoai_embeddings.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
ollama.py community[patch]: OllamaEmbeddings - Pass headers to post request (#16880) 2024-03-29 18:44:52 +00:00
openai.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
openvino.py community[patch]: Update OpenVINO embedding and reranker to support static input shape (#22171) 2024-06-03 13:27:17 -07:00
optimum_intel.py community: fix tqdm import (#20263) 2024-04-25 19:44:53 +00:00
oracleai.py community[patch]: update embeddings/oracleai.py (#22240) 2024-06-03 12:38:51 -07:00
ovhcloud.py community: Fix OVHcloud 401 Unauthorized on embedding. (#23260) 2024-06-24 12:58:32 -04:00
premai.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
sagemaker_endpoint.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
sambanova.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
self_hosted_hugging_face.py docs: Update documentation to use 'model_id' rather than 'model_name' to match actual API (#16615) 2024-01-26 15:01:12 -08:00
self_hosted.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
sentence_transformer.py infra: update mypy 1.10, ruff 0.5 (#23721) 2024-07-03 10:33:27 -07:00
solar.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
spacy_embeddings.py infra: add -p to mkdir in lint steps (#17013) 2024-02-05 11:22:06 -08:00
sparkllm.py community: docstrings (#23202) 2024-06-20 11:08:13 -04:00
tensorflow_hub.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
text2vec.py Update text2vec.py (#19657) 2024-03-27 13:13:30 -04:00
textembed.py community[minor]: Add TextEmbed Embedding Integration (#22946) 2024-07-19 17:30:25 +00:00
titan_takeoff.py community[patch]: fix public interface for embeddings module (#21650) 2024-05-22 11:42:15 -04:00
vertexai.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
volcengine.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
voyageai.py (all): update removal in deprecation warnings from 0.2 to 0.3 (#21265) 2024-05-03 14:29:36 -04:00
xinference.py community[patch]: Adopting the lighter-weight xinference_client (#21900) 2024-05-20 22:05:09 +00:00
yandex.py core[minor],community[minor]: Upgrade all @root_validator() to @pre_init (#23841) 2024-07-08 16:09:29 -04:00
zhipuai.py fix https://github.com/langchain-ai/langchain/issues/23215 (#23216) 2024-06-20 13:04:50 +00:00