Add Xorbits Dataframe as a Document Loader (#7319)

- [Xorbits](https://doc.xorbits.io/en/latest/) is an open-source
computing framework that makes it easy to scale data science and machine
learning workloads in parallel. Xorbits can leverage multi cores or GPUs
to accelerate computation on a single machine, or scale out up to
thousands of machines to support processing terabytes of data.

- This PR added support for the Xorbits document loader, which allows
langchain to leverage Xorbits to parallelize and distribute the loading
of data.
- Dependencies: This change requires the Xorbits library to be installed
in order to be used.
`pip install xorbits`
- Request for review: @rlancemartin, @eyurtsev
- Twitter handle: https://twitter.com/Xorbitsio

Co-authored-by: Bagatur <baskaryan@gmail.com>
This commit is contained in:
Yifei Song
2023-07-10 16:24:47 +08:00
committed by GitHub
parent 21a353e9c2
commit 7d29bb2c02
6 changed files with 418 additions and 0 deletions

View File

@@ -660,6 +660,7 @@ Classes
document_loaders.word_document.Docx2txtLoader
document_loaders.word_document.UnstructuredWordDocumentLoader
document_loaders.xml.UnstructuredXMLLoader
document_loaders.xorbits.XorbitsLoader
document_loaders.youtube.GoogleApiYoutubeLoader
document_loaders.youtube.YoutubeLoader