langchain

mirror of https://github.com/hwchase17/langchain.git synced 2025-09-21 02:19:31 +00:00

Files

Hyunjoon Jeong 9cc85387d1 fix(text-splitters): add validation to prevent infinite loop and prevent empty token splitter (#32205 )

### Description
1) Add validation to prevent infinite loop condition when
```tokenizer.tokens_per_chunk > tokenizer.chunk_overlap```
2) Avoid empty decoded chunk when splitter appends tokens

---------

Co-authored-by: Eugene Yurtsev <eugene@langchain.dev>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>

2025-09-11 16:55:32 -04:00

xsl

text-splitters[patch]: delete unused html_chunks_with_headers.xslt (#29340 )

2025-01-23 11:29:08 -05:00

__init__.py

chore: formatting across codebase (#32466 )

2025-08-08 10:20:10 -04:00

base.py

fix(text-splitters): add validation to prevent infinite loop and prevent empty token splitter (#32205 )

2025-09-11 16:55:32 -04:00

character.py

chore(text-splitters): select ALL rules with exclusions (#32325 )