mirror of
https://github.com/hwchase17/langchain.git
synced 2025-05-18 13:31:36 +00:00
e4cfaa5680
### Summary
This PR introduces a `SeleniumURLLoader` which, similar to
`UnstructuredURLLoader`, loads data from URLs. However, it utilizes
`selenium` to fetch page content, enabling it to work with
JavaScript-rendered pages. The `unstructured` library is also employed
for loading the HTML content.
### Testing
```bash
pip install selenium
pip install unstructured
```
```python
from langchain.document_loaders import SeleniumURLLoader
urls = [
"https://www.youtube.com/watch?v=dQw4w9WgXcQ",
"https://goo.gl/maps/NDSHwePEyaHMFGwh8"
]
loader = SeleniumURLLoader(urls=urls)
data = loader.load()
```
|
||
|---|---|---|
| .. | ||
| document_loaders/examples | ||
| retrievers/examples | ||
| text_splitters | ||
| vectorstores | ||
| document_loaders.rst | ||
| getting_started.ipynb | ||
| retrievers.rst | ||
| text_splitters.rst | ||
| vectorstores.rst | ||