github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-05-08 16:48:49 +00:00
Code Issues Packages Projects Releases Wiki Activity
b51a1eba4d
langchain/libs/community/tests/unit_tests/document_loaders
History
maang-h fc93bed8c4
community: Fix CSVLoader columns is None (#20701) 
- **Bug code**: In
langchain_community/document_loaders/csv_loader.py:100

- **Description**: currently, when 'CSVLoader' reads the column as None
in the 'csv' file, it will report an error because the 'CSVLoader' does
not verify whether the column is of str type and does not consider how
to handle the corresponding 'row_data' when the column is' None 'in the
csv. This pr provides a solution.

- **Issue:**  Fix #20699 

- **thinking:**

1. Refer to the processing method for
'langchain_community/document_loaders/csv_loader.py:100' when **'v'**
equals'None', and apply the same method to '**k**'.
(Reference`csv.DictReader` ,**'k'** will only be None when `
len(columns) < len(number_row_data)` is established)
2. **‘k’** equals None only holds when it is the last column, and its
corresponding **'v'** type is a list. Therefore, I referred to the data
format in 'Document' and used ',' to concatenated the elements in the
list.(But I'm not sure if you accept this form, if you have any other
ideas, communicate)

---------

Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>
2024-05-22 12:57:46 -07:00
..
blob_loaders community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
loaders community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
parsers community: Add PHP language parser to document_loaders (#19850) 2024-04-08 11:30:28 -04:00
sample_documents community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_docs community: Fix CSVLoader columns is None (#20701) 2024-05-22 12:57:46 -07:00
__init__.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_airbyte.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_arcgis_loader.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_assemblyai.py Merge pull request #18421 2024-03-06 13:16:05 -05:00
test_bibtex.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_bshtml.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_confluence.py Merge pull request #18436 2024-03-06 13:15:24 -05:00
test_couchbase.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_csv_loader.py community: Fix CSVLoader columns is None (#20701) 2024-05-22 12:57:46 -07:00
test_cube_semantic.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_detect_encoding.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_directory_loader.py community: Fix CSVLoader columns is None (#20701) 2024-05-22 12:57:46 -07:00
test_directory.py community[minor]: Implement DirectoryLoader lazy_load function (#19537) 2024-03-29 14:46:52 +00:00
test_evernote_loader.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_generic_loader.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_git.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_github.py community[patch]: upgrade to recent version of mypy (#21616) 2024-05-13 14:55:07 -04:00
test_hugging_face_model.py community[minor]: add hugging_face_model document loader (#17323) 2024-02-28 20:05:35 +00:00
test_hugging_face.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_imports.py community[minor]: Oraclevs integration (#21123) 2024-05-04 03:15:35 +00:00
test_json_loader.py community[minor]: use jq schema for content_key in json_loader (#18003) 2024-03-05 15:51:24 -08:00
test_lakefs.py community[minor]: import fix (#20995) 2024-04-29 10:32:50 -04:00
test_mediawikidump.py infra: add print rule to ruff (#16221) 2024-02-09 16:13:30 -08:00
test_mhtml.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_mongodb.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_notebook.py community[patch]: add NotebookLoader unit test (#17721) 2024-03-29 00:27:46 +00:00
test_obsidian.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_onenote.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_oracleadb.py community[minor]: add oracle autonomous database doc loader integration (#19536) 2024-03-26 17:02:18 -07:00
test_pebblo.py community[minor]: Add support for Pebblo cloud_api_key in PebbloSafeLoader (#19855) 2024-04-08 11:10:04 -04:00
test_psychic.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_readthedoc.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_rspace_loader.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_rss.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_trello.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_web_base.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
test_youtube.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00