community: add InfinityRerank (#27043)

**Description:** 

- Add a Reranker for Infinity server.

**Dependencies:** 

This wrapper uses
[infinity_client](https://github.com/michaelfeil/infinity/tree/main/libs/client_infinity/infinity_client)
to connect to an Infinity server.

**Tests and docs**

- integration test: test_infinity_rerank.py
- example notebook: infinity_rerank.ipynb
[here](https://github.com/baptiste-pasquier/langchain/blob/feat/infinity-rerank/docs/docs/integrations/document_transformers/infinity_rerank.ipynb)

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
This commit is contained in:
Baptiste Pasquier
2024-11-07 02:26:30 +01:00
committed by GitHub
parent 2494deb2a4
commit 81f7daa458
5 changed files with 578 additions and 0 deletions

View File

@@ -0,0 +1,32 @@
from langchain_core.documents import Document
from langchain_community.document_compressors.infinity_rerank import (
InfinityRerank,
)
def test_rerank() -> None:
reranker = InfinityRerank()
docs = [
Document(
page_content=(
"This is a document not related to the python package infinity_emb, "
"hence..."
)
),
Document(page_content="Paris is in France!"),
Document(
page_content=(
"infinity_emb is a package for sentence embeddings and rerankings using"
" transformer models in Python!"
)
),
Document(page_content="random text for nothing"),
]
compressed = reranker.compress_documents(
query="What is the python package infinity_emb?",
documents=docs,
)
assert len(compressed) == 3, "default top_n is 3"
assert compressed[0].page_content == docs[2].page_content, "rerank works"