mirror of
https://github.com/hwchase17/langchain.git
synced 2025-09-14 14:05:37 +00:00
community[minor]: Azure DocumentIntelligenceLoader/Parser support update with latest SDK (#14389)
- **Description:** Add DocumentIntelligenceLoader & DocumentIntelligenceParser implementation using the latest Azure Document Intelligence SDK with markdown support. The core logic resides in DocumentIntelligenceParser and DocumentIntelligenceLoader is a mere wrapper of the parser. The parser will takes api_endpoint and api_key and creates DocumentIntelligenceClient for the user. 4 parsing modes are supported: 1. Markdown (default) 2. Single 3. Page 4. Object UT and notebook are also updated accordingly. - **Dependencies:** Azure Document Intelligence SDK: azure-ai-documentintelligence [azure-sdk-for-python/sdk/documentintelligence/azure-ai-documentintelligence at 7c42462ac662522a6fd21b17d2a20f4cd40d0356 · Azure/azure-sdk-for-python (github.com)](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-python%2Ftree%2F7c42462ac662522a6fd21b17d2a20f4cd40d0356%2Fsdk%2Fdocumentintelligence%2Fazure-ai-documentintelligence&data=05%7C01%7CZifei.Qian%40microsoft.com%7C298225aa3e31468a863108dbf07374ff%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638368150928704292%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=oE0Sl4HERnMKdbkV9KgBV46Z2xytcQAShdTWf7ZNl%2Bs%3D&reserved=0). --------- Co-authored-by: Erick Friis <erick@langchain.dev>
This commit is contained in:
@@ -5,7 +5,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Azure Document Intelligence"
|
||||
"# Azure AI Document Intelligence"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -13,7 +13,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Azure Document Intelligence (formerly known as Azure Forms Recognizer) is machine-learning \n",
|
||||
"Azure AI Document Intelligence (formerly known as Azure Form Recognizer) is machine-learning \n",
|
||||
"based service that extracts text (including handwriting), tables or key-value-pairs from\n",
|
||||
"scanned documents or images.\n",
|
||||
"\n",
|
||||
@@ -21,7 +21,7 @@
|
||||
"\n",
|
||||
"Document Intelligence supports PDF, JPEG, PNG, BMP, or TIFF.\n",
|
||||
"\n",
|
||||
"Further documentation is available at https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/?view=doc-intel-3.1.0.\n"
|
||||
"Further documentation is available at https://aka.ms/doc-intelligence.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -30,7 +30,7 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install langchain azure-ai-formrecognizer -q"
|
||||
"%pip install langchain langchain-community azure-ai-documentintelligence -q"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -46,23 +46,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The first example uses a local file which will be sent to Azure Document Intelligence.\n",
|
||||
"\n",
|
||||
"First, an instance of a DocumentAnalysisClient is created with endpoint and key for the Azure service. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azure.ai.formrecognizer import DocumentAnalysisClient\n",
|
||||
"from azure.core.credentials import AzureKeyCredential\n",
|
||||
"\n",
|
||||
"document_analysis_client = DocumentAnalysisClient(\n",
|
||||
" endpoint=\"<service_endpoint>\", credential=AzureKeyCredential(\"<service_key>\")\n",
|
||||
")"
|
||||
"The first example uses a local file which will be sent to Azure AI Document Intelligence."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -75,15 +59,18 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langchain.document_loaders.pdf import DocumentIntelligenceLoader\n",
|
||||
"from langchain_community.document_loaders import AzureAIDocumentIntelligenceLoader\n",
|
||||
"\n",
|
||||
"loader = DocumentIntelligenceLoader(\n",
|
||||
" \"<Local_filename>\", client=document_analysis_client, model=\"<model_name>\"\n",
|
||||
") # e.g. prebuilt-document\n",
|
||||
"file_path = \"<filepath>\"\n",
|
||||
"endpoint = \"<endpoint>\"\n",
|
||||
"key = \"<key>\"\n",
|
||||
"loader = AzureAIDocumentIntelligenceLoader(\n",
|
||||
" api_endpoint=endpoint, api_key=key, file_path=file_path, api_model=\"prebuilt-layout\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"documents = loader.load()"
|
||||
]
|
||||
@@ -93,25 +80,45 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The output contains each page of the source document as a LangChain document: "
|
||||
"The default output contains one LangChain document with markdown format content: "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[Document(page_content='...', metadata={'source': '...', 'page': 1})]"
|
||||
]
|
||||
},
|
||||
"execution_count": 18,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"documents"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Example 2\n",
|
||||
"The input file can also be URL path."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"url_path = \"<url>\"\n",
|
||||
"loader = AzureAIDocumentIntelligenceLoader(\n",
|
||||
" api_endpoint=endpoint, api_key=key, url_path=url_path, api_model=\"prebuilt-layout\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"documents = loader.load()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"documents"
|
||||
]
|
||||
@@ -124,8 +131,16 @@
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"version": "3.9.5"
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.10"
|
||||
},
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
|
Reference in New Issue
Block a user