mirror of
https://github.com/hwchase17/langchain.git
synced 2025-06-28 09:28:48 +00:00
community: fix DirectoryLoader
progress bar (#19821)
**Description:** currently, the `DirectoryLoader` progress-bar maximum value is based on an incorrect number of files to process In langchain_community/document_loaders/directory.py:127: ```python paths = p.rglob(self.glob) if self.recursive else p.glob(self.glob) items = [ path for path in paths if not (self.exclude and any(path.match(glob) for glob in self.exclude)) ] ``` `paths` returns both files and directories. `items` is later used to determine the maximum value of the progress-bar which gives an incorrect progress indication.
This commit is contained in:
parent
984e7e36c2
commit
463160c3f6
@ -129,6 +129,7 @@ class DirectoryLoader(BaseLoader):
|
||||
path
|
||||
for path in paths
|
||||
if not (self.exclude and any(path.match(glob) for glob in self.exclude))
|
||||
and path.is_file()
|
||||
]
|
||||
|
||||
if self.sample_size > 0:
|
||||
|
Loading…
Reference in New Issue
Block a user