Switches type checking for `langchain-text-splitters` from `mypy` to
[`ty`](https://docs.astral.sh/ty/), which is much faster. The `ollama`
package already [switched to
`ty`](https://github.com/langchain-ai/langchain/pull/36571).
## What changed
The core of this PR is the config swap (`[tool.mypy]` →
`[tool.ty.rules]`/`[tool.ty.analysis]`, `Makefile`, and the `typing`
dependency group). Because `ty` runs with `all = "error"`, a few modules
also needed source-level adjustments to satisfy the stricter analysis.
These are **behavior-preserving refactors** except for one intentional
fix, called out below so reviewers know where to look.
### Behavioral change (intentional fix)
- `SentenceTransformersTokenTextSplitter` now raises a clear
`ValueError` when the underlying model reports no maximum sequence
length **and** no `tokens_per_chunk` was provided. Previously this
combination reached a `None > None` comparison and surfaced as an opaque
`TypeError`. As a consequence, the public `maximum_tokens_per_chunk`
attribute is now honestly typed as `int | None` — it can remain `None`
when the caller supplies `tokens_per_chunk` explicitly for a model
without a limit.
### Behavior-preserving refactors (no user-visible change)
- `TokenTextSplitter.from_tiktoken_encoder` is now an explicit override
rather than the base method dispatching on `issubclass(cls,
TokenTextSplitter)`. The shared length-function logic moved into a
private helper. Public signatures and return types are unchanged.
- `NLTKTextSplitter` builds its tokenizer once at construction, so
`_tokenizer` is now always a `Callable[[str], list[str]]`. The private
attributes `_language` and `_use_span_tokenize` are no longer stored —
flagging in case any downstream code read those (they are
underscore-private). Tokenization output is unchanged.
- `HTMLSemanticPreservingSplitter` text extraction was rewritten from a
`cast`-based check to `isinstance(element, Tag)` narrowing; output is
equivalent for tags, text nodes, and comments.
---------
Co-authored-by: Mason Daugherty <github@mdrxy.com>
## Description
This pull request introduces a new text splitter,
`JSFrameworkTextSplitter`, to the Langchain library. The
`JSFrameworkTextSplitter` extends the `RecursiveCharacterTextSplitter`
to handle JavaScript framework code effectively, including React (JSX),
Vue, and Svelte. It identifies and utilizes framework-specific component
tags and syntax elements as splitting points, alongside standard
JavaScript syntax. This ensures that code is divided at natural
boundaries, enhancing the parsing and processing of JavaScript and
framework-specific code.
### Key Features
- Supports React (JSX), Vue, and Svelte frameworks.
- Identifies and uses framework-specific tags and syntax elements as
natural splitting points.
- Extends the existing `RecursiveCharacterTextSplitter` for seamless
integration.
## Issue
No specific issue addressed.
## Dependencies
No additional dependencies required.
---------
Co-authored-by: ccurme <chester.curme@gmail.com>
