Aleksandr Filippov f0b6baa0ef fix(core): track within-batch deduplication in indexing num_skipped count (#32273) ...

**Description:** Fixes incorrect `num_skipped` count in the LangChain
indexing API. The current implementation only counts documents that
already exist in RecordManager (cross-batch duplicates) but fails to
count documents removed during within-batch deduplication via
`_deduplicate_in_order()`.

This PR adds tracking of the original batch size before deduplication
and includes the difference in `num_skipped`, ensuring that `num_added +
num_skipped` equals the total number of input documents.

**Issue:** Fixes incorrect document count reporting in indexing
statistics

**Dependencies:** None

Fixes #32272

---------

Co-authored-by: Alex Feel <afilippov@spotware.com>

2025-07-28 09:58:51 -04:00

..

__init__.py

…

test_hashed_document.py

core[patch]: Add additional hashing options to indexing API, warn on SHA-1 (#31649)

2025-06-24 14:44:06 -04:00

test_in_memory_indexer.py

chore(core): unpin packaging dependency (#32032)

2025-07-14 21:42:32 +00:00

test_in_memory_record_manager.py

core: Ruff preview fixes (#31877)

2025-07-07 13:02:40 -04:00

test_indexing.py

fix(core): track within-batch deduplication in indexing num_skipped count (#32273)

2025-07-28 09:58:51 -04:00

test_public_api.py

core: Add ruff rules RUF (#29353)

2025-05-15 15:43:57 -04:00