mirror of
https://github.com/hwchase17/langchain.git
synced 2025-08-23 11:32:10 +00:00
eaf2fb287f
# Description ## Summary This PR adds support for handling multi-labeled page numbers in the **PyPDFLoader**. Some PDFs use complex page numbering systems where the actual content may begin after multiple introductory pages. The page_label field helps accurately reflect the document’s page structure, making it easier to handle such cases during document parsing. ## Motivation This feature improves document parsing accuracy by allowing users to access the actual page labels instead of relying only on the physical page numbers. This is particularly useful for documents where the first few pages have roman numerals or other non-standard page labels. ## Use Case This feature is especially useful for **Retrieval-Augmented Generation** (RAG) systems where users may reference page numbers when asking questions. Some PDFs have both labeled page numbers (like roman numerals for introductory sections) and index-based page numbers. For example, a user might ask: "What is mentioned on page 5?" The system can now check both: • **Index-based page number** (page) • **Labeled page number** (page_label) This dual-check helps improve retrieval accuracy. Additionally, the results can be validated with an **agent or tool** to ensure the retrieved pages match the user’s query contextually. ## Code Changes - Added a page_label field to the metadata of the Document class in **PyPDFLoader**. - Implemented support for retrieving page_label from the pdf_reader.page_labels. - Created a test case (test_pypdf_loader_with_multi_label_page_numbers) with a sample PDF containing multi-labeled pages (geotopo-komprimiert.pdf) [[Source of pdf](https://github.com/py-pdf/sample-files/blob/main/009-pdflatex-geotopo/GeoTopo-komprimiert.pdf)]. - Updated existing tests to ensure compatibility and verify page_label extraction. ## Tests Added - Added a new test case for a PDF with multi-labeled pages. - Verified both page and page_label metadata fields are correctly extracted. ## Screenshots <img width="549" alt="image" src="https://github.com/user-attachments/assets/65db9f5c-032e-4592-926f-824777c28f33" /> |
||
|---|---|---|
| .. | ||
| agent_toolkits | ||
| agents | ||
| callbacks | ||
| chains | ||
| chat_loaders | ||
| chat_message_histories | ||
| chat_models | ||
| cross_encoders | ||
| data | ||
| docstore | ||
| document_compressors | ||
| document_loaders | ||
| document_transformers | ||
| embeddings | ||
| evaluation | ||
| examples | ||
| graph_vectorstores | ||
| graphs | ||
| imports | ||
| indexes | ||
| llms | ||
| load | ||
| query_constructors | ||
| retrievers | ||
| storage | ||
| tools | ||
| utilities | ||
| utils | ||
| vectorstores | ||
| __init__.py | ||
| conftest.py | ||
| test_cache.py | ||
| test_dependencies.py | ||
| test_document_transformers.py | ||
| test_graph_vectorstores.py | ||
| test_imports.py | ||
| test_sql_database_schema.py | ||
| test_sql_database.py | ||
| test_sqlalchemy.py | ||