github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-09-28 06:48:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore: update branch with changes from master (#32277)

Browse Source 
Co-authored-by: Maxime Grenu <69890511+cluster2600@users.noreply.github.com>
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: jmaillefaud <jonathan.maillefaud@evooq.ch>
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
Co-authored-by: tanwirahmad <tanwirahmad@users.noreply.github.com>
Co-authored-by: Christophe Bornet <cbornet@hotmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: niceg <79145285+growmuye@users.noreply.github.com>
Co-authored-by: Chaitanya varma <varmac301@gmail.com>
Co-authored-by: dishaprakash <57954147+dishaprakash@users.noreply.github.com>
Co-authored-by: Chester Curme <chester.curme@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Kanav Bansal <13186335+bansalkanav@users.noreply.github.com>
Co-authored-by: Aleksandr Filippov <71711753+alex-feel@users.noreply.github.com>
Co-authored-by: Alex Feel <afilippov@spotware.com>
This commit is contained in:
Mason Daugherty
2025-07-28 10:39:41 -04:00
committed by GitHub
parent 3496e1739e
commit 5e9eb19a83
449 changed files with 16481 additions and 5327 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
libs/text-splitters/langchain_text_splitters/html.py
View File

@@ -107,6 +107,7 @@ class HTMLHeaderTextSplitter:
# content="Conclusion"
# - Document with metadata={"Main Topic": "Conclusion"} and
# content="Final thoughts."
"""
def __init__(
@@ -562,6 +563,7 @@ class HTMLSemanticPreservingSplitter(BaseDocumentTransformer):
preserve_images=True,
custom_handlers={"iframe": custom_iframe_extractor}
)
""" # noqa: E501, D214
def __init__(
@@ -842,6 +844,10 @@ class HTMLSemanticPreservingSplitter(BaseDocumentTransformer):
preserved_elements,
placeholder_count,
)
content = " ".join(elem.find_all(string=True, recursive=False))
if content:
content = self._normalize_and_clean_text(content)
current_content.append(content)
continue
if elem.name in [h[0] for h in self._headers_to_split_on]: