github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-09-03 20:16:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: Add UnstructuredCSVLoader for CSV files (#5844)

Browse Source 
### Summary

Adds an `UnstructuredCSVLoader` for loading CSVs. One advantage of using
`UnstructuredCSVLoader` relative to the standard `CSVLoader` is that if
you use `UnstructuredCSVLoader` in `"elements"` mode, an HTML
representation of the table will be available in the metadata.

#### Who can review?

@hwchase17
 @eyurtsev
This commit is contained in:
Matt Robinson
2023-06-07 22:18:01 -04:00
committed by GitHub
parent 0b4a51930c
commit 11fec7d4d1
5 changed files with 248 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

211
docs/modules/indexes/document_loaders/examples/csv.ipynb
View File

@@ -29,7 +29,6 @@
"cell_type": "code",
"execution_count": 26,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
@@ -45,7 +44,6 @@
"cell_type": "code",
"execution_count": 27,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
@@ -76,7 +74,6 @@
"cell_type": "code",
"execution_count": 28,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
@@ -96,7 +93,6 @@
"cell_type": "code",
"execution_count": 29,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
@@ -152,6 +148,211 @@
"source": [
"print(data)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## `UnstructuredCSVLoader`\n",
"\n",
"You can also load the table using the `UnstructuredCSVLoader`. One advantage of using `UnstructuredCSVLoader` is that if you use it in `\"elements\"` mode, an HTML representation of the table will be available in the metadata."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders.csv_loader import UnstructuredCSVLoader"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"loader = UnstructuredCSVLoader(file_path='example_data/mlb_teams_2012.csv', mode=\"elements\")\n",
"docs = loader.load()"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<table border=\"1\" class=\"dataframe\">\n",
" <tbody>\n",
" <tr>\n",
" <td>Nationals</td>\n",
" <td>81.34</td>\n",
" <td>98</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Reds</td>\n",
" <td>82.20</td>\n",
" <td>97</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Yankees</td>\n",
" <td>197.96</td>\n",
" <td>95</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Giants</td>\n",
" <td>117.62</td>\n",
" <td>94</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Braves</td>\n",
" <td>83.31</td>\n",
" <td>94</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Athletics</td>\n",
" <td>55.37</td>\n",
" <td>94</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Rangers</td>\n",
" <td>120.51</td>\n",
" <td>93</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Orioles</td>\n",
" <td>81.43</td>\n",
" <td>93</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Rays</td>\n",
" <td>64.17</td>\n",
" <td>90</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Angels</td>\n",
" <td>154.49</td>\n",
" <td>89</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Tigers</td>\n",
" <td>132.30</td>\n",
" <td>88</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Cardinals</td>\n",
" <td>110.30</td>\n",
" <td>88</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Dodgers</td>\n",
" <td>95.14</td>\n",
" <td>86</td>\n",
" </tr>\n",
" <tr>\n",
" <td>White Sox</td>\n",
" <td>96.92</td>\n",
" <td>85</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Brewers</td>\n",
" <td>97.65</td>\n",
" <td>83</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Phillies</td>\n",
" <td>174.54</td>\n",
" <td>81</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Diamondbacks</td>\n",
" <td>74.28</td>\n",
" <td>81</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Pirates</td>\n",
" <td>63.43</td>\n",
" <td>79</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Padres</td>\n",
" <td>55.24</td>\n",
" <td>76</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Mariners</td>\n",
" <td>81.97</td>\n",
" <td>75</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Mets</td>\n",
" <td>93.35</td>\n",
" <td>74</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Blue Jays</td>\n",
" <td>75.48</td>\n",
" <td>73</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Royals</td>\n",
" <td>60.91</td>\n",
" <td>72</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Marlins</td>\n",
" <td>118.07</td>\n",
" <td>69</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Red Sox</td>\n",
" <td>173.18</td>\n",
" <td>69</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Indians</td>\n",
" <td>78.43</td>\n",
" <td>68</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Twins</td>\n",
" <td>94.08</td>\n",
" <td>66</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Rockies</td>\n",
" <td>78.06</td>\n",
" <td>64</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Cubs</td>\n",
" <td>88.19</td>\n",
" <td>61</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Astros</td>\n",
" <td>60.65</td>\n",
" <td>55</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n"
]
}
],
"source": [
"print(docs[0].metadata[\"text_as_html\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
@@ -170,7 +371,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
"version": "3.8.13"
}
},
"nbformat": 4,