Nicolas ad04585e30 community[minor]: Firecrawl.dev integration (#20364) ...

Added the [FireCrawl](https://firecrawl.dev) document loader. Firecrawl
crawls and convert any website into LLM-ready data. It crawls all
accessible subpages and give you clean markdown for each.

    - **Description:** Adds FireCrawl data loader
    - **Dependencies:** firecrawl-py
    - **Twitter handle:** @mendableai 

ccing contributors: (@ericciarla @nickscamara)

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>

2024-04-12 19:13:48 +00:00

..

blob_loaders

community[patch]: speed up import times in the community package (#18928)

2024-03-11 16:37:36 -04:00

parsers

community[patch]: import flattening fix (#20110)

2024-04-10 13:01:19 -04:00

__init__.py

community[minor]: Firecrawl.dev integration (#20364)

2024-04-12 19:13:48 +00:00

acreom.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

airbyte_json.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

airbyte.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

airtable.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

apify_dataset.py

…

arcgis_loader.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

arxiv.py

community[minor]: Implement lazy_load() for ArxivLoader (#18664)

2024-03-06 09:16:49 -05:00

assemblyai.py

community[patch]: docstrings update (#20301)

2024-04-11 16:23:27 -04:00

astradb.py

community[patch]: Use langchain-astradb for AstraDB doc loader (#19071)

2024-03-15 22:57:25 +00:00

async_html.py

…

athena.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

azlyrics.py

…

azure_ai_data.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

azure_blob_storage_container.py

community[patch]: type ignore fixes (#18395)

2024-03-01 11:21:02 -08:00

azure_blob_storage_file.py

…

baiducloud_bos_directory.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

baiducloud_bos_file.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

base_o365.py

…

base.py

core: Move document loader interfaces to core (#17723)

2024-03-06 13:59:00 -05:00

bibtex.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

bigquery.py

deprecating integrations moved to langchain_google_community (#19841)

2024-04-02 17:06:07 -04:00

bilibili.py

community[patch]: docstrings update (#20301)

2024-04-11 16:23:27 -04:00

blackboard.py

community[patch]: type ignore fixes (#18395)

2024-03-01 11:21:02 -08:00

blockchain.py

…

brave_search.py

…

browserless.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

cassandra.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

chatgpt.py

…

chm.py

community[patch]: docstrings (#16810)

2024-02-09 12:48:57 -08:00

chromium.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

college_confidential.py

…

concurrent.py

community[patch]: import flattening fix (#20110)

2024-04-10 13:01:19 -04:00

confluence.py

community[patch]: expanding version in confluence loader (#19324)

2024-03-25 17:08:01 -07:00

conllu.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

couchbase.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

csv_loader.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

cube_semantic.py

community[patch]: Implement lazy_load() for CubeSemanticLoader (#18535)

2024-03-05 17:32:31 -08:00

datadog_logs.py

…

dataframe.py

community[patch]: support modin document loader (#18866)

2024-03-10 18:40:04 -07:00

diffbot.py

…

directory.py

community[patch]: fixed multithreading returning List[List[Documents]] instead of List[Documents] (#20230)

2024-04-09 17:06:37 -04:00

discord.py

…

doc_intelligence.py

community[patch]: import flattening fix (#20110)

2024-04-10 13:01:19 -04:00

docugami.py

deprecate community docugami loader (#19230)

2024-03-18 12:56:47 -07:00

docusaurus.py

…

dropbox.py

infra: add print rule to ruff (#16221)

2024-02-09 16:13:30 -08:00

duckdb_loader.py

…

email.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

epub.py

…

etherscan.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

evernote.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

excel.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

facebook_chat.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

fauna.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

figma.py

…

firecrawl.py

community[minor]: Firecrawl.dev integration (#20364)

2024-04-12 19:13:48 +00:00

gcs_directory.py

community[patch]: GCSDirectoryLoader bugfix (#20005)

2024-04-09 16:57:00 +00:00

gcs_file.py

deprecating integrations moved to langchain_google_community (#19841)

2024-04-02 17:06:07 -04:00

generic.py

community[patch]: import flattening fix (#20110)

2024-04-10 13:01:19 -04:00

geodataframe.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

git.py

Merge pull request #18539

2024-03-06 13:25:14 -05:00

gitbook.py

community[minor]: Implement lazy_load() for GitbookLoader (#18670)

2024-03-06 09:14:36 -05:00

github.py

community: Implement lazy_load() for GithubFileLoader (#18584)

2024-03-05 09:35:50 -08:00

google_speech_to_text.py

deprecating integrations moved to langchain_google_community (#19841)

2024-04-02 17:06:07 -04:00

googledrive.py

deprecating integrations moved to langchain_google_community (#19841)

2024-04-02 17:06:07 -04:00

gutenberg.py

…

helpers.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

hn.py

…

html_bs.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

html.py

…

hugging_face_dataset.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

hugging_face_model.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

ifixit.py

…

image_captions.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

image.py

…

imsdb.py

…

iugu.py

…

joplin.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

json_loader.py

community: Implement lazy_load() for JSONLoader (#18643)

2024-03-08 13:58:17 -05:00

lakefs.py

…

larksuite.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

llmsherpa.py

community[minor]: add support for llmsherpa (#19741)

2024-03-29 16:04:57 -07:00

markdown.py

…

mastodon.py

Merge pull request #18671

2024-03-06 13:23:14 -05:00

max_compute.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

mediawikidump.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

merge.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

mhtml.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

modern_treasury.py

…

mongodb.py

community[minor]: added a feature to filter documents in Mongoloader (#18253)

2024-03-08 12:06:35 -08:00

news.py

…

notebook.py

community[patch]: add NotebookLoader unit test (#17721)

2024-03-29 00:27:46 +00:00

notion.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

notiondb.py

community[patch]: Fix NotionDBLoader 400 Error by conditionally adding filter parameter (#19075)

2024-03-14 13:56:57 +00:00

nuclia.py

infra: add print rule to ruff (#16221)

2024-02-09 16:13:30 -08:00

obs_directory.py

…

obs_file.py

…

obsidian.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

odt.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

onedrive_file.py

…

onedrive.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

onenote.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

open_city_data.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

oracleadb_loader.py

community[minor]: add oracle autonomous database doc loader integration (#19536)

2024-03-26 17:02:18 -07:00

org_mode.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

pdf.py

community[patch]: import flattening fix (#20110)

2024-04-10 13:01:19 -04:00

pebblo.py

community[minor]: Add support for Pebblo cloud_api_key in PebbloSafeLoader (#19855)

2024-04-08 11:10:04 -04:00

polars_dataframe.py

…

powerpoint.py

…

psychic.py

Merge pull request #18656

2024-03-06 13:05:04 -05:00

pubmed.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

pyspark_dataframe.py

…

python.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

quip.py

…

readthedocs.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

recursive_url_loader.py

community[patch]: RecursiveUrlLoader: add base_url option (#19421)

2024-03-22 15:34:31 -07:00

reddit.py

…

roam.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

rocksetdb.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

rspace.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

rss.py

…

rst.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

rtf.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

s3_directory.py

community[patch]: Skip nested directories when using S3DirectoryLoader (#17829)

2024-03-08 16:50:58 -08:00

s3_file.py

community[patch]: support unstructured_kwargs for s3 loader (#15473)

2024-03-27 22:03:48 +00:00

sharepoint.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

sitemap.py

community[minor]: Implement lazy_load() for SitemapLoader (#18667)

2024-03-06 09:15:35 -05:00

slack_directory.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

snowflake_loader.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

spreedly.py

…

sql_database.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

srt.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

stripe.py

…

surrealdb.py

…

telegram.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

tencent_cos_directory.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

tencent_cos_file.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

tensorflow_datasets.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

text.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

tidb.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

tomarkdown.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

toml.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

trello.py

community: Implement lazy_load() for TrelloLoader (#18658)

2024-03-06 13:04:36 -05:00

tsv.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

twitter.py

…

unstructured.py

community[patch]: refactor the type hint of file_path in UnstructuredAPIFileLoader class (#18839)

2024-03-27 22:31:54 +00:00

url_playwright.py

community: Implement lazy_load() for PlaywrightURLLoader (#18676)

2024-03-06 16:52:13 -05:00

url_selenium.py

…

url.py

…

vsdx.py

community[patch]: import flattening fix (#20110)

2024-04-10 13:01:19 -04:00

weather.py

community: Use default load() implementation in doc loaders (#18385)

2024-03-01 14:46:52 -05:00

web_base.py

core[minor]: Add aload to document loader (#19936)

2024-04-03 10:46:47 -04:00

whatsapp_chat.py

community: Implement lazy_load() for WhatsAppChatLoader (#18677)

2024-03-06 13:03:46 -05:00

wikipedia.py

community[minor]: Implement lazy_load() for WikipediaLoader (#18680)

2024-03-06 13:03:21 -05:00

word_document.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

xml.py

community: better support of pathlib paths in document loaders (#18396)

2024-03-26 11:51:52 -04:00

xorbits.py

…

youtube.py

community[patch]: docstrings (#16810)

2024-02-09 12:48:57 -08:00

yuque.py

community[minor]: add Yuque document loader (#17924)

2024-03-05 15:54:07 -08:00