community[patch]: Implement Unique ID Enforcement in FAISS (#17244)

**Description:**
Implemented unique ID validation in the FAISS component to ensure all
document IDs are distinct. This update resolves issues related to
non-unique IDs, such as inconsistent behavior during deletion processes.
This commit is contained in:
ByeongUk Choi
2024-02-09 05:03:33 +09:00
committed by GitHub
parent 88609565a3
commit b88329e9a5
2 changed files with 15 additions and 0 deletions

View File

@@ -190,6 +190,9 @@ class FAISS(VectorStore):
_len_check_if_sized(documents, embeddings, "documents", "embeddings")
_len_check_if_sized(documents, ids, "documents", "ids")
if ids and len(ids) != len(set(ids)):
raise ValueError("Duplicate ids found in the ids list.")
# Add to the index.
vector = np.array(embeddings, dtype=np.float32)
if self._normalize_L2: