mirror of
https://github.com/hwchase17/langchain.git
synced 2025-09-23 11:30:37 +00:00
text-splitters: Add JSFrameworkTextSplitter for Handling JavaScript Framework Code (#28972)
## Description This pull request introduces a new text splitter, `JSFrameworkTextSplitter`, to the Langchain library. The `JSFrameworkTextSplitter` extends the `RecursiveCharacterTextSplitter` to handle JavaScript framework code effectively, including React (JSX), Vue, and Svelte. It identifies and utilizes framework-specific component tags and syntax elements as splitting points, alongside standard JavaScript syntax. This ensures that code is divided at natural boundaries, enhancing the parsing and processing of JavaScript and framework-specific code. ### Key Features - Supports React (JSX), Vue, and Svelte frameworks. - Identifies and uses framework-specific tags and syntax elements as natural splitting points. - Extends the existing `RecursiveCharacterTextSplitter` for seamless integration. ## Issue No specific issue addressed. ## Dependencies No additional dependencies required. --------- Co-authored-by: ccurme <chester.curme@gmail.com>
This commit is contained in:
@@ -36,6 +36,7 @@ from langchain_text_splitters.html import (
|
||||
HTMLSemanticPreservingSplitter,
|
||||
)
|
||||
from langchain_text_splitters.json import RecursiveJsonSplitter
|
||||
from langchain_text_splitters.jsx import JSFrameworkTextSplitter
|
||||
from langchain_text_splitters.konlpy import KonlpyTextSplitter
|
||||
from langchain_text_splitters.latex import LatexTextSplitter
|
||||
from langchain_text_splitters.markdown import (
|
||||
@@ -60,6 +61,7 @@ __all__ = [
|
||||
"RecursiveCharacterTextSplitter",
|
||||
"RecursiveJsonSplitter",
|
||||
"LatexTextSplitter",
|
||||
"JSFrameworkTextSplitter",
|
||||
"PythonCodeTextSplitter",
|
||||
"KonlpyTextSplitter",
|
||||
"SpacyTextSplitter",
|
||||
|
Reference in New Issue
Block a user