mirror of
https://github.com/hwchase17/langchain.git
synced 2025-06-22 06:39:52 +00:00
community[patch]: fixed multithreading returning List[List[Documents]] instead of List[Documents] (#20230)
Description: When multithreading is set to True and using the DirectoryLoader, there was a bug that caused the return type to be a double nested list. This resulted in other places upstream not being able to utilize the from_documents method as it was no longer a `List[Documents]` it was a `List[List[Documents]]`. The change made was to just loop through the `future.result()` and yield every item. Issue: #20093 Dependencies: N/A Twitter handle: N/A
This commit is contained in:
parent
230376f183
commit
806d4ae48f
@ -174,7 +174,8 @@ class DirectoryLoader(BaseLoader):
|
||||
)
|
||||
)
|
||||
for future in concurrent.futures.as_completed(futures):
|
||||
yield future.result()
|
||||
for item in future.result():
|
||||
yield item
|
||||
else:
|
||||
for i in items:
|
||||
yield from self._lazy_load_file(i, p, pbar)
|
||||
|
Loading…
Reference in New Issue
Block a user