github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-10-23 11:16:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add PythonLoader which auto-detects encoding of Python files (#3311)

Browse Source 
This PR contributes a `PythonLoader`, which inherits from
`TextLoader` but detects and sets the encoding automatically.
This commit is contained in:
Paul Garner
2023-04-21 18:47:57 +01:00
committed by GitHub
parent 1ecbeec24e
commit aa9d5707e0
7 changed files with 104 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
tests/integration_tests/examples/non-utf8-encoding.py Normal file
View File

@@ -0,0 +1,3 @@
# coding: iso-8859-5
# <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD> <- Cyrillic characters
u = "<EFBFBD><EFBFBD><EFBFBD><EFBFBD>"