Vwp/docs improved document loaders (#4006)

Huge thanks to @leo-gan for improving the document loaders notebooks

---------

Co-authored-by: Leonid Ganeline <leo.gan.57@gmail.com>
This commit is contained in:
Zander Chase
2023-05-02 15:24:53 -07:00
committed by GitHub
parent 1c68cbdb28
commit aa38355999
57 changed files with 1227 additions and 779 deletions

View File

@@ -6,14 +6,33 @@
"metadata": {},
"source": [
"# Subtitle Files\n",
"How to load data from subtitle (`.srt`) files"
"\n",
">[The SubRip file format](https://en.wikipedia.org/wiki/SubRip#SubRip_file_format) is described on the `Matroska` multimedia container format website as \"perhaps the most basic of all subtitle formats.\" `SubRip (SubRip Text)` files are named with the extension `.srt`, and contain formatted lines of plain text in groups separated by a blank line. Subtitles are numbered sequentially, starting at 1. The timecode format used is hours:minutes:seconds,milliseconds with time units fixed to two zero-padded digits and fractions fixed to three zero-padded digits (00:00:00,000). The fractional separator used is the comma, since the program was written in France.\n",
"\n",
"How to load data from subtitle (`.srt`) files\n",
"\n",
"Please, download the [example .srt file from here](https://www.opensubtitles.org/en/subtitles/5575150/star-wars-the-clone-wars-crisis-at-the-heart-en)."
]
},
{
"cell_type": "code",
"execution_count": 24,
"execution_count": null,
"id": "c6eb0372-ad36-4747-8120-d1557fe632fd",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"!pip install pysrt"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "2cbb7f5c",
"metadata": {},
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.document_loaders import SRTLoader"
@@ -21,9 +40,11 @@
},
{
"cell_type": "code",
"execution_count": 25,
"execution_count": 5,
"id": "865d8a14",
"metadata": {},
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"loader = SRTLoader(\"example_data/Star_Wars_The_Clone_Wars_S06E07_Crisis_at_the_Heart.srt\")"
@@ -31,9 +52,11 @@
},
{
"cell_type": "code",
"execution_count": 26,
"execution_count": null,
"id": "173a9234",
"metadata": {},
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"docs = loader.load()"
@@ -59,14 +82,6 @@
"source": [
"docs[0].page_content[:100]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3b7a8dc4",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
@@ -85,7 +100,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
"version": "3.10.6"
}
},
"nbformat": 4,