From 080d977690c8eaafbb367cd35e6a3603351ee945 Mon Sep 17 00:00:00 2001 From: Anhui-tqhuang Date: Sun, 18 Feb 2024 15:46:10 +0800 Subject: [PATCH] docs: add doc to the reranker --- fern/docs.yml | 4 +++ fern/docs/pages/recipes/reranker.mdx | 45 ++++++++++++++++++++++++++++ models/.gitignore | 2 -- 3 files changed, 49 insertions(+), 2 deletions(-) create mode 100644 fern/docs/pages/recipes/reranker.mdx delete mode 100644 models/.gitignore diff --git a/fern/docs.yml b/fern/docs.yml index 2611dac8..93c799fe 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -76,6 +76,10 @@ navigation: # TODO: add recipes - page: List of LLMs path: ./docs/pages/recipes/list-llm.mdx + - section: Choice of Reranker + contents: + - page: Reranker + path: ./docs/pages/recipes/reranker.mdx # More advanced usage of privateGPT, by API - tab: api-reference layout: diff --git a/fern/docs/pages/recipes/reranker.mdx b/fern/docs/pages/recipes/reranker.mdx new file mode 100644 index 00000000..2a2fbf18 --- /dev/null +++ b/fern/docs/pages/recipes/reranker.mdx @@ -0,0 +1,45 @@ +# Reranker + +PrivateGPT supports the integration with the `Reranker` which has the potential to enhance the performance of the Retrieval-Augmented Generation (RAG) system. + +## Configurations + +The Reranker can be configured using the following parameters: + +- **top_n**: Represents the number of top documents to retrieve. +- **cut_off**: A threshold score for similarity below which documents are dismissed. +- **enabled**: A boolean flag to activate or deactivate the reranker. +- **hf_model_name**: The Hugging Face model identifier for the FlagReranker. + +## Behavior of Reranker + +The functionality of the `Reranker` is as follows: + +1. It evaluates the similarity between a query and documents retrieved by the retriever. +2. If the similarity score is less than `cut_off`, the document is excluded from the results. +3. In scenarios where the filtered documents are fewer than `top_n`, the system defaults to providing the top `top_n` documents ignoring the `cut_off` score. +4. The `hf_model_name` parameter allows users to specify the particular FlagReranker model from [Hugging Face](https://huggingface.co/) for the reranking process. + +Use the `enabled` flag to toggle the `Reranker` as per requirement for optimized results. + +## Example Usage + +To utilize the `Reranker` with your desired settings: + +```yml +reranker: + enabled: true + hf_model_name: BAAI/bge-reranker-large + top_n: 5 + cut_off: 0.75 +``` + +## Conclusion + +`Reranker` serves as a [Node Postprocessor](https://docs.llamaindex.ai/en/stable/module_guides/querying/node_postprocessors/root.html). With these settings, it offers a robust and flexible way to improve the performance of the RAG system by filtering and ranking the retrieved documents based on relevancy. + +## Moreover + +The llamaindex is already integrated with an LLM-based reranker. However, this integration faces stability issues due to the LLM’s output being somewhat unpredictable. Such erratic behavior occasionally leads to complications where the output cannot be effectively parsed by privateGPT. The expected format is a structured list of documents with associated relevance scores. The LLM reranker sometimes generates outputs with inconsistent formatting or adds extraneous summaries not conducive to parsing. + +Due to these inconsistencies, there is a consideration to transition towards a specialized model strictly dedicated to reranking, which would reliably output only similarity scores. Such a model promises a more stable and predictable behavior. diff --git a/models/.gitignore b/models/.gitignore deleted file mode 100644 index c96a04f0..00000000 --- a/models/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -* -!.gitignore \ No newline at end of file