bachr b3ed98dec0 community[patch]: avoid KeyError when language not in LANGUAGE_SEGMENTERS (#15212) ...

**Description:**

Handle unsupported languages in same way as when none is provided 
 
**Issue:**

The following line will throw a KeyError if the language is not
supported.
```python
self.Segmenter = LANGUAGE_SEGMENTERS[language]
```
E.g. when using `Language.CPP` we would get `KeyError: <Language.CPP:
'cpp'>`

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>

2024-01-23 21:09:43 -08:00

..

blob_loaders

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

parsers

community[patch]: avoid KeyError when language not in LANGUAGE_SEGMENTERS (#15212)

2024-01-23 21:09:43 -08:00

__init__.py

community[minor]: New documents loader for visio files (with extension .vsdx) (#16171)

2024-01-22 22:07:03 -08:00

acreom.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

airbyte_json.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

airbyte.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

airtable.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

apify_dataset.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

arcgis_loader.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

arxiv.py

Update arxiv.py with get_summaries_as_docs inside of Arxivloader (#14953)

2023-12-22 13:14:22 -08:00

assemblyai.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

astradb.py

Add AstraDB document loader (#14747)

2024-01-01 16:13:28 -08:00

async_html.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

azlyrics.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

azure_ai_data.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

azure_blob_storage_container.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

azure_blob_storage_file.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

baiducloud_bos_directory.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

baiducloud_bos_file.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

base_o365.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

base.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

bibtex.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

bigquery.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

bilibili.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

blackboard.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

blockchain.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

brave_search.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

browserless.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

cassandra.py

community[minor]: Add Cassandra document loader (#16215)

2024-01-18 18:49:02 -08:00

chatgpt.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

chm.py

Feat: add CHM file loader (#15519)

2024-01-07 09:28:52 -08:00

chromium.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

college_confidential.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

concurrent.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

confluence.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

conllu.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

couchbase.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

csv_loader.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

cube_semantic.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

datadog_logs.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

dataframe.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

diffbot.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

directory.py

community: correct spelling mistakes of "Suffle" and "reporoducibility" (#15172)

2023-12-26 11:22:59 -08:00

discord.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

doc_intelligence.py

community: fix the "page" mode in the AzureAIDocumentIntelligenceParser (bug) (#15958)

2024-01-12 11:01:28 -08:00

docugami.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

docusaurus.py

docs: docstrings langchain_community update (#14889)

2023-12-19 08:58:24 -05:00

dropbox.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

duckdb_loader.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

email.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

epub.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

etherscan.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

evernote.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

excel.py

Docs: fix excel document loader typo (#15470)

2024-01-07 09:33:35 -08:00

facebook_chat.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

fauna.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

figma.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

gcs_directory.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

gcs_file.py

fix: correct spelling mistakes of "seperate, intialise, pre-defined" (#14647)

2023-12-22 11:49:35 -08:00

generic.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

geodataframe.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

git.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

gitbook.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

github.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

google_speech_to_text.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

googledrive.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

gutenberg.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

helpers.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

hn.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

html_bs.py

fix: correct spelling mistakes of "seperate, intialise, pre-defined" (#14647)

2023-12-22 11:49:35 -08:00

html.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

hugging_face_dataset.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

ifixit.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

image_captions.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

image.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

imsdb.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

iugu.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

joplin.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

json_loader.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

lakefs.py

docs: docstrings langchain_community update (#14889)

2023-12-19 08:58:24 -05:00

larksuite.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

markdown.py

corrected outdated link (#15053)

2023-12-22 12:39:38 -08:00

mastodon.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

max_compute.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

mediawikidump.py

community:Lazy load wikipedia dump file (#15111)

2024-01-01 14:02:56 -08:00

merge.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

mhtml.py

fix: correct spelling mistakes of "seperate, intialise, pre-defined" (#14647)

2023-12-22 11:49:35 -08:00

modern_treasury.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

mongodb.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

news.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

notebook.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

notion.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

notiondb.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

nuclia.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

obs_directory.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

obs_file.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

obsidian.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

odt.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

onedrive_file.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

onedrive.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

onenote.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

open_city_data.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

org_mode.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

pdf.py

community: Include PDF ID in MathPix metadata (#15629)

2024-01-07 08:31:53 -08:00

polars_dataframe.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

powerpoint.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

psychic.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

pubmed.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

pyspark_dataframe.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

python.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

quip.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

readthedocs.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

recursive_url_loader.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

reddit.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

roam.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

rocksetdb.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

rspace.py

fix: correct spelling mistakes of "seperate, intialise, pre-defined" (#14647)

2023-12-22 11:49:35 -08:00

rss.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

rst.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

rtf.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

s3_directory.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

s3_file.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

sharepoint.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

sitemap.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

slack_directory.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

snowflake_loader.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

spreedly.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

srt.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

stripe.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

surrealdb.py

community[patch]: SurrealDB fix for asyncio (#16092)

2024-01-23 19:46:19 -08:00

telegram.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

tencent_cos_directory.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

tencent_cos_file.py

fix: correct spelling mistakes of "seperate, intialise, pre-defined" (#14647)

2023-12-22 11:49:35 -08:00

tensorflow_datasets.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

text.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

tomarkdown.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

toml.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

trello.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

tsv.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

twitter.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

unstructured.py

community[patch]: Load list of files using UnstructuredFileLoader (#16216)

2024-01-23 19:37:37 -08:00

url_playwright.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

url_selenium.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

url.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

vsdx.py

community[minor]: New documents loader for visio files (with extension .vsdx) (#16171)

2024-01-22 22:07:03 -08:00

weather.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

web_base.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

whatsapp_chat.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

wikipedia.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

word_document.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

xml.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

xorbits.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00

youtube.py

community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)

2023-12-11 13:53:30 -08:00