mirror of
https://github.com/hwchase17/langchain.git
synced 2025-08-07 12:06:43 +00:00
[Community]: requests_kwargs
not being used in _fetch (#28646)
- **Description:** `requests_kwargs` is not being passed to `_fetch` which is fetching pages asynchronously. In this PR, making sure that we are passing `requests_kwargs` to `_fetch` just like `_scrape`. - **Issue:** #28634 --------- Co-authored-by: Erick Friis <erick@langchain.dev>
This commit is contained in:
parent
8c37808d47
commit
fa155a422f
@ -208,7 +208,10 @@ class WebBaseLoader(BaseLoader):
|
||||
)
|
||||
if not self.session.verify:
|
||||
kwargs["ssl"] = False
|
||||
async with session.get(url, **kwargs) as response:
|
||||
|
||||
async with session.get(
|
||||
url, **(self.requests_kwargs | kwargs)
|
||||
) as response:
|
||||
if self.raise_for_status:
|
||||
response.raise_for_status()
|
||||
return await response.text()
|
||||
|
Loading…
Reference in New Issue
Block a user