Commit Graph

8 Commits

Author SHA1 Message Date
Balaji Seshadri
fd822b07c0 fix(text-splitters): restore lazy imports for heavy optional dependencies (#35469)
## Summary

- Moves `nltk`, `spacy`, `sentence-transformers`, and `konlpy` imports
back inside class constructors/functions so they are only loaded when
the respective splitter is actually instantiated
- Adds a subprocess-based regression test to verify no heavy packages
are imported at `langchain_text_splitters` load time

## Why

PR #32325 moved these optional dependency imports to module-level
`try/except` blocks (to satisfy ruff's `PLC0415` rule). Since
`__init__.py` imports all four splitter modules, this caused `import
langchain_text_splitters` to eagerly load all optional heavy packages,
resulting in:

- A PyTorch NVML warning (`UserWarning: Can't initialize NVML`) on
non-GPU machines
- A ~650MB memory spike on import (74MB → 736MB), vs ~50MB in 0.3.x

The fix restores the lazy import pattern with `# noqa: PLC0415` to
suppress the linter rule, which is the correct trade-off when a
dependency has high instantiation cost.

## Review notes

- The `PLC0415` suppressions are intentional — these are optional heavy
dependencies that should never be loaded unless the user explicitly
instantiates the splitter class
- The regression test uses a subprocess for proper isolation (the test
file itself imports `langchain_text_splitters` at the top, so
`sys.modules` checks within the same process would not reflect a clean
import state)

Fixes #35437.

> **AI disclaimer:** This PR was developed with assistance from Claude
Code (Anthropic AI).

---------

Co-authored-by: AshwathB-debug <ashwathbalaji04@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Mason Daugherty <github@mdrxy.com>
2026-07-05 23:19:16 -04:00
Christophe Bornet
fd69425439 style(text-splitters): fix some ruff preview rules (#34665)
Co-authored-by: Mason Daugherty <mason@langchain.dev>
Co-authored-by: Mason Daugherty <github@mdrxy.com>
2026-01-09 17:28:18 -05:00
Christophe Bornet
0c3e8ccd0e chore(text-splitters): select ALL rules with exclusions (#32325)
Co-authored-by: Mason Daugherty <mason@langchain.dev>
2025-09-08 14:46:09 +00:00
Mason Daugherty
457ce9c4b0 feat(text-splitters): ruff fixes and rules (#32502) 2025-08-11 13:28:22 -04:00
Christophe Bornet
451c90fefa text-splitters: Ruff autofixes (#31858)
Auto-fixes from ruff with rule `ALL`
2025-07-07 10:06:08 -04:00
Christophe Bornet
802d2bf249 text-splitters: Add ruff rule UP (pyupgrade) (#31841)
See https://docs.astral.sh/ruff/rules/#pyupgrade-up
All auto-fixed except `typing.AbstractSet` -> `collections.abc.Set`
2025-07-03 10:11:35 -04:00
Christophe Bornet
eab8484a80 text-splitters[patch]: fix some import-untyped errors (#31030) 2025-05-15 11:34:22 -04:00
Bagatur
5efb5c099f text-splitters[minor], langchain[minor], community[patch], templates, docs: langchain-text-splitters 0.0.1 (#18346) 2024-02-29 18:33:21 -08:00