mirror of
https://github.com/hwchase17/langchain.git
synced 2025-07-02 03:15:11 +00:00
merge pages into google
and AWS
pages (#11312)
There are several pages in `integrations/providers/more` that belongs to Google and AWS `integrations/providers`. - moved content of these pages into the Google and AWS `integrations/providers` pages - removed these individual pages
This commit is contained in:
parent
70be04a816
commit
22165cb2fc
@ -1,6 +1,6 @@
|
|||||||
# AWS
|
# AWS
|
||||||
|
|
||||||
All functionality related to AWS platform
|
All functionality related to [Amazon AWS](https://aws.amazon.com/) platform
|
||||||
|
|
||||||
## LLMs
|
## LLMs
|
||||||
|
|
||||||
@ -70,7 +70,7 @@ from langchain.llms.sagemaker_endpoint import ContentHandlerBase
|
|||||||
|
|
||||||
## Document loaders
|
## Document loaders
|
||||||
|
|
||||||
### AWS S3 Directory
|
### AWS S3 Directory and File
|
||||||
>[Amazon Simple Storage Service (Amazon S3)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html) is an object storage service.
|
>[Amazon Simple Storage Service (Amazon S3)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html) is an object storage service.
|
||||||
>[AWS S3 Directory](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html)
|
>[AWS S3 Directory](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html)
|
||||||
>[AWS S3 Buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html)
|
>[AWS S3 Buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html)
|
||||||
@ -82,3 +82,24 @@ See a [usage example for S3FileLoader](/docs/integrations/document_loaders/aws_s
|
|||||||
```python
|
```python
|
||||||
from langchain.document_loaders import S3DirectoryLoader, S3FileLoader
|
from langchain.document_loaders import S3DirectoryLoader, S3FileLoader
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Memory
|
||||||
|
|
||||||
|
### AWS DynamoDB
|
||||||
|
|
||||||
|
>[AWS DynamoDB](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/dynamodb/index.html)
|
||||||
|
> is a fully managed `NoSQL` database service that provides fast and predictable performance with seamless scalability.
|
||||||
|
|
||||||
|
We have to configur the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html).
|
||||||
|
|
||||||
|
We need to install the `boto3` library.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install boto3
|
||||||
|
```
|
||||||
|
|
||||||
|
See a [usage example](/docs/integrations/memory/aws_dynamodb).
|
||||||
|
|
||||||
|
```python
|
||||||
|
from langchain.memory import DynamoDBChatMessageHistory
|
||||||
|
```
|
@ -1,6 +1,6 @@
|
|||||||
# Google
|
# Google
|
||||||
|
|
||||||
All functionality related to Google Platform
|
All functionality related to [Google Cloud Platform](https://cloud.google.com/)
|
||||||
|
|
||||||
## LLMs
|
## LLMs
|
||||||
|
|
||||||
@ -37,7 +37,7 @@ from langchain.chat_models import ChatVertexAI
|
|||||||
>[Google BigQuery](https://cloud.google.com/bigquery) is a serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data.
|
>[Google BigQuery](https://cloud.google.com/bigquery) is a serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data.
|
||||||
`BigQuery` is a part of the `Google Cloud Platform`.
|
`BigQuery` is a part of the `Google Cloud Platform`.
|
||||||
|
|
||||||
First, you need to install `google-cloud-bigquery` python package.
|
First, we need to install `google-cloud-bigquery` python package.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install google-cloud-bigquery
|
pip install google-cloud-bigquery
|
||||||
@ -53,7 +53,7 @@ from langchain.document_loaders import BigQueryLoader
|
|||||||
|
|
||||||
>[Google Cloud Storage](https://en.wikipedia.org/wiki/Google_Cloud_Storage) is a managed service for storing unstructured data.
|
>[Google Cloud Storage](https://en.wikipedia.org/wiki/Google_Cloud_Storage) is a managed service for storing unstructured data.
|
||||||
|
|
||||||
First, you need to install `google-cloud-storage` python package.
|
First, we need to install `google-cloud-storage` python package.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install google-cloud-storage
|
pip install google-cloud-storage
|
||||||
@ -78,7 +78,7 @@ from langchain.document_loaders import GCSFileLoader
|
|||||||
|
|
||||||
Currently, only `Google Docs` are supported.
|
Currently, only `Google Docs` are supported.
|
||||||
|
|
||||||
First, you need to install several python package.
|
First, we need to install several python package.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install google-api-python-client google-auth-httplib2 google-auth-oauthlib
|
pip install google-api-python-client google-auth-httplib2 google-auth-oauthlib
|
||||||
@ -109,6 +109,32 @@ See a [usage example](/docs/integrations/vectorstores/matchingengine).
|
|||||||
from langchain.vectorstores import MatchingEngine
|
from langchain.vectorstores import MatchingEngine
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Google ScaNN
|
||||||
|
|
||||||
|
>[Google ScaNN](https://github.com/google-research/google-research/tree/master/scann)
|
||||||
|
> (Scalable Nearest Neighbors) is a python package.
|
||||||
|
>
|
||||||
|
>`ScaNN` is a method for efficient vector similarity search at scale.
|
||||||
|
|
||||||
|
>`ScaNN` includes search space pruning and quantization for Maximum Inner
|
||||||
|
> Product Search and also supports other distance functions such as
|
||||||
|
> Euclidean distance. The implementation is optimized for x86 processors
|
||||||
|
> with AVX2 support. See its [Google Research github](https://github.com/google-research/google-research/tree/master/scann)
|
||||||
|
> for more details.
|
||||||
|
|
||||||
|
We need to install `scann` python package.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install scann
|
||||||
|
```
|
||||||
|
|
||||||
|
See a [usage example](/docs/integrations/vectorstores/scann).
|
||||||
|
|
||||||
|
```python
|
||||||
|
from langchain.vectorstores import ScaNN
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
## Tools
|
## Tools
|
||||||
### Google Search
|
### Google Search
|
||||||
|
|
||||||
@ -123,8 +149,36 @@ from langchain.utilities import GoogleSearchAPIWrapper
|
|||||||
```
|
```
|
||||||
For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/google_search.html).
|
For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/google_search.html).
|
||||||
|
|
||||||
You can easily load this wrapper as a Tool (to use with an Agent). You can do this with:
|
We can easily load this wrapper as a Tool (to use with an Agent). We can do this with:
|
||||||
```python
|
```python
|
||||||
from langchain.agents import load_tools
|
from langchain.agents import load_tools
|
||||||
tools = load_tools(["google-search"])
|
tools = load_tools(["google-search"])
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Document Transformer
|
||||||
|
### Google Document AI
|
||||||
|
|
||||||
|
>[Document AI](https://cloud.google.com/document-ai/docs/overview) is a `Google Cloud Platform`
|
||||||
|
> service to transform unstructured data from documents into structured data, making it easier
|
||||||
|
> to understand, analyze, and consume.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
We need to set up a [`GCS` bucket and create your own OCR processor](https://cloud.google.com/document-ai/docs/create-processor)
|
||||||
|
The `GCS_OUTPUT_PATH` should be a path to a folder on GCS (starting with `gs://`)
|
||||||
|
and a processor name should look like `projects/PROJECT_NUMBER/locations/LOCATION/processors/PROCESSOR_ID`.
|
||||||
|
We can get it either programmatically or copy from the `Prediction endpoint` section of the `Processor details`
|
||||||
|
tab in the Google Cloud Console.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install google-cloud-documentai
|
||||||
|
pip install google-cloud-documentai-toolbox
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
See a [usage example](/docs/integrations/document_transformers/docai).
|
||||||
|
|
||||||
|
```python
|
||||||
|
from langchain.document_loaders.blob_loaders import Blob
|
||||||
|
from langchain.document_loaders.parsers import DocAIParser
|
||||||
|
```
|
||||||
|
@ -1,23 +0,0 @@
|
|||||||
# AWS DynamoDB
|
|
||||||
|
|
||||||
>[AWS DynamoDB](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/dynamodb/index.html)
|
|
||||||
> is a fully managed `NoSQL` database service that provides fast and predictable performance with seamless scalability.
|
|
||||||
|
|
||||||
## Installation and Setup
|
|
||||||
|
|
||||||
We have to configur the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html).
|
|
||||||
|
|
||||||
We need to install the `boto3` library.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install boto3
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
## Memory
|
|
||||||
|
|
||||||
See a [usage example](/docs/integrations/memory/aws_dynamodb).
|
|
||||||
|
|
||||||
```python
|
|
||||||
from langchain.memory import DynamoDBChatMessageHistory
|
|
||||||
```
|
|
@ -1,28 +0,0 @@
|
|||||||
# Google Document AI
|
|
||||||
|
|
||||||
>[Document AI](https://cloud.google.com/document-ai/docs/overview) is a `Google Cloud Platform`
|
|
||||||
> service to transform unstructured data from documents into structured data, making it easier
|
|
||||||
> to understand, analyze, and consume.
|
|
||||||
|
|
||||||
|
|
||||||
## Installation and Setup
|
|
||||||
|
|
||||||
You need to set up a [`GCS` bucket and create your own OCR processor](https://cloud.google.com/document-ai/docs/create-processor)
|
|
||||||
The `GCS_OUTPUT_PATH` should be a path to a folder on GCS (starting with `gs://`)
|
|
||||||
and a processor name should look like `projects/PROJECT_NUMBER/locations/LOCATION/processors/PROCESSOR_ID`.
|
|
||||||
You can get it either programmatically or copy from the `Prediction endpoint` section of the `Processor details`
|
|
||||||
tab in the Google Cloud Console.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install google-cloud-documentai
|
|
||||||
pip install google-cloud-documentai-toolbox
|
|
||||||
```
|
|
||||||
|
|
||||||
## Document Transformer
|
|
||||||
|
|
||||||
See a [usage example](/docs/integrations/document_transformers/docai).
|
|
||||||
|
|
||||||
```python
|
|
||||||
from langchain.document_loaders.blob_loaders import Blob
|
|
||||||
from langchain.document_loaders.parsers import DocAIParser
|
|
||||||
```
|
|
@ -1,29 +0,0 @@
|
|||||||
# ScaNN
|
|
||||||
|
|
||||||
>[Google ScaNN](https://github.com/google-research/google-research/tree/master/scann)
|
|
||||||
> (Scalable Nearest Neighbors) is a python package.
|
|
||||||
>
|
|
||||||
>`ScaNN` is a method for efficient vector similarity search at scale.
|
|
||||||
|
|
||||||
>ScaNN includes search space pruning and quantization for Maximum Inner
|
|
||||||
> Product Search and also supports other distance functions such as
|
|
||||||
> Euclidean distance. The implementation is optimized for x86 processors
|
|
||||||
> with AVX2 support. See its [Google Research github](https://github.com/google-research/google-research/tree/master/scann)
|
|
||||||
> for more details.
|
|
||||||
|
|
||||||
## Installation and Setup
|
|
||||||
|
|
||||||
We need to install `scann` python package.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install scann
|
|
||||||
```
|
|
||||||
|
|
||||||
## Vector Store
|
|
||||||
|
|
||||||
See a [usage example](/docs/integrations/vectorstores/scann).
|
|
||||||
|
|
||||||
```python
|
|
||||||
from langchain.vectorstores import ScaNN
|
|
||||||
```
|
|
||||||
|
|
Loading…
Reference in New Issue
Block a user