mirror of
https://github.com/hwchase17/langchain.git
synced 2025-09-07 22:11:51 +00:00
chore: add support for TypeScript code splitting (#11160)
- **Description:** Adds typescript language to `TextSplitter` --------- Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
This commit is contained in:
@@ -18,6 +18,7 @@ from langchain.text_splitter import (
|
||||
'go',
|
||||
'java',
|
||||
'js',
|
||||
'ts',
|
||||
'php',
|
||||
'proto',
|
||||
'python',
|
||||
@@ -107,6 +108,36 @@ js_docs
|
||||
|
||||
</CodeOutputBlock>
|
||||
|
||||
## TS
|
||||
Here's an example using the TS text splitter:
|
||||
|
||||
|
||||
```python
|
||||
TS_CODE = """
|
||||
function helloWorld(): void {
|
||||
console.log("Hello, World!");
|
||||
}
|
||||
|
||||
// Call the function
|
||||
helloWorld();
|
||||
"""
|
||||
|
||||
ts_splitter = RecursiveCharacterTextSplitter.from_language(
|
||||
language=Language.TS, chunk_size=60, chunk_overlap=0
|
||||
)
|
||||
ts_docs = ts_splitter.create_documents([TS_CODE])
|
||||
ts_docs
|
||||
```
|
||||
|
||||
<CodeOutputBlock lang="python">
|
||||
|
||||
```
|
||||
[Document(page_content='function helloWorld(): void {\n console.log("Hello, World!");\n}', metadata={}),
|
||||
Document(page_content='// Call the function\nhelloWorld();', metadata={})]
|
||||
```
|
||||
|
||||
</CodeOutputBlock>
|
||||
|
||||
## Markdown
|
||||
|
||||
Here's an example using the Markdown text splitter:
|
||||
|
Reference in New Issue
Block a user