mirror of
https://github.com/hwchase17/langchain.git
synced 2026-03-16 18:13:33 +00:00
8791ada8c0
Bumps [nltk](https://github.com/nltk/nltk) from 3.9.2 to 3.9.3. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/nltk/nltk/blob/develop/ChangeLog">nltk's changelog</a>.</em></p> <blockquote> <p>Version 3.9.3 2026-02-21</p> <ul> <li>Fix CVE-2025-14009: secure ZIP extraction in nltk.downloader (<a href="https://redirect.github.com/nltk/nltk/issues/3468">#3468</a>)</li> <li>Block path traversal/arbitrary reads in nltk.data for protocol-less refs (<a href="https://redirect.github.com/nltk/nltk/issues/3467">#3467</a>)</li> <li>Block path traversal/abs paths in corpus readers and FS pointers (<a href="https://redirect.github.com/nltk/nltk/issues/3479">#3479</a>, <a href="https://redirect.github.com/nltk/nltk/issues/3480">#3480</a>)</li> <li>Validate external StanfordSegmenter JARs using SHA256 (<a href="https://redirect.github.com/nltk/nltk/issues/3477">#3477</a>)</li> <li>Add optional sandbox enforcement for filestring() (<a href="https://redirect.github.com/nltk/nltk/issues/3485">#3485</a>)</li> <li>Maintenance: downloader/zipped models, CI/tooling updates</li> </ul> <p>Thanks to the following contributors to 3.9.3: Chris Clauss, Eric Kafe, HyperPS, purificant, Shivansh-Game, Christopher Smith</p> <p>Version 3.9.2 2025-10-01</p> <ul> <li>Update download checksums to use SHA256 in built index</li> <li>Fix percentage escape in new-style string formatting</li> <li>replace shortened URLs using goo.gl</li> <li>Make Wordnet interoperable with various taggers and tagged corpora</li> <li>Fix saving PerceptronTagger</li> <li>Document how to reproduce old Wordnet studies</li> <li>properly initialize Portuguese corpus reader</li> <li>support for mixed rules conversion into Chomsky Normal Form</li> <li>only import tkinter if a GUI is needed</li> <li>issue <a href="https://redirect.github.com/nltk/nltk/issues/2112">#2112</a> with Corenlp</li> <li>new environment variable NLTK_DOWNLOADER_FORCE_INTERACTIVE_SHELL</li> <li>Lesk defaults to most frequent sense in case of ties</li> </ul> <p>Thanks to the following contributors to 3.9.2: Jose Cols, Peter de Blanc, GeneralPoxter, Eric Kafe, William LaCroix, Jason Liu, Samer Masterson, Mike014, purificant, Andrew Ernest Ritz, samertm, Ikram Ul Haq, Christopher Smith, Ryan Mannion</p> <p>Version 3.9.1 2024-08-19</p> <ul> <li>Fixed bug that prevented wordnet from loading</li> </ul> <p>Version 3.9 2024-08-18</p> <ul> <li>Fix security vulnerability CVE-2024-39705 (breaking change)</li> <li>Replace pickled models (punkt, chunker, taggers) by new pickle-free "_tab" packages</li> <li>No longer sort Wordnet synsets and relations (sort in calling function when required)</li> <li>Only strip the last suffix in Wordnet Morphy, thus restricting synsets() results</li> <li>Add Python 3.12 support</li> <li>Many other minor fixes</li> </ul> <p>Thanks to the following contributors to 3.8.2: Tom Aarsen, Cat Lee Ball, Veralara Bernhard, Carlos Brandt, Konstantin Chernyshev, Michael Higgins, Eric Kafe, Vivek Kalyan, David Lukes, Rob Malouf, purificant, Alex Rudnick, Liling Tan, Akihiro Yamazaki.</p> <p>Version 3.8.1 2023-01-02</p> <ul> <li>Resolve RCE vulnerability in localhost WordNet Browser (<a href="https://redirect.github.com/nltk/nltk/issues/3100">#3100</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="4154eb85e8"><code>4154eb8</code></a> Merge pull request <a href="https://redirect.github.com/nltk/nltk/issues/3503">#3503</a> from ekaf/hotfix-3501</li> <li><a href="7a710cbc8b"><code>7a710cb</code></a> Prepare release 3.9.3</li> <li><a href="1056b323af"><code>1056b32</code></a> Merge pull request <a href="https://redirect.github.com/nltk/nltk/issues/3468">#3468</a> from HyperPS/fix/secure-unzip-rce</li> <li><a href="7dc5baa98f"><code>7dc5baa</code></a> Resolve merge conflict in tag mapping using normalized nltk resource URL</li> <li><a href="7ef38b8aa6"><code>7ef38b8</code></a> Merge pull request <a href="https://redirect.github.com/nltk/nltk/issues/3467">#3467</a> from HyperPS/develop</li> <li><a href="b2e1164bf8"><code>b2e1164</code></a> Merge pull request <a href="https://redirect.github.com/nltk/nltk/issues/3485">#3485</a> from HyperPS/fix-filestring-sandbox-update</li> <li><a href="ac0ce55daa"><code>ac0ce55</code></a> Merge pull request <a href="https://redirect.github.com/nltk/nltk/issues/3480">#3480</a> from HyperPS/fix/filesystem-sandbox-security</li> <li><a href="603e34d25a"><code>603e34d</code></a> Merge pull request <a href="https://redirect.github.com/nltk/nltk/issues/3479">#3479</a> from HyperPS/fix/corpusreader-path-traversal</li> <li><a href="b63a5014aa"><code>b63a501</code></a> Merge pull request <a href="https://redirect.github.com/nltk/nltk/issues/3477">#3477</a> from HyperPS/fix/stanford-segmenter-rce-sha256</li> <li><a href="df38955e50"><code>df38955</code></a> Merge pull request <a href="https://redirect.github.com/nltk/nltk/issues/3494">#3494</a> from ekaf/ewnv</li> <li>Additional commits viewable in <a href="https://github.com/nltk/nltk/compare/3.9.2...3.9.3">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/langchain-ai/langchain/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
🦜✂️ LangChain Text Splitters
Looking for the JS/TS version? Check out LangChain.js.
Quick Install
pip install langchain-text-splitters
🤔 What is this?
LangChain Text Splitters contains utilities for splitting into chunks a wide variety of text documents.
📖 Documentation
For full documentation, see the API reference.
📕 Releases & Versioning
See our Releases and Versioning policies.
We encourage pinning your version to a specific version in order to avoid breaking your CI when we publish new tests. We recommend upgrading to the latest version periodically to make sure you have the latest tests.
Not pinning your version will ensure you always have the latest tests, but it may also break your CI if we introduce tests that your integration doesn't pass.
💁 Contributing
As an open-source project in a rapidly developing field, we are extremely open to contributions, whether it be in the form of a new feature, improved infrastructure, or better documentation.
For detailed information on how to contribute, see the Contributing Guide.