From 3dabd3f2146c0dbabe51c633f44ba369f1271a68 Mon Sep 17 00:00:00 2001 From: Leonid Ganeline Date: Sat, 9 Mar 2024 13:34:33 -0800 Subject: [PATCH] docs: platform pages update (#17836) `Integrations` platform page ToC-s: sections there are placed without order. For example, the [google](https://python.langchain.com/docs/integrations/platforms/google) page. The `LLM` section is not the first section, as it is in the [Components](https://python.langchain.com/docs/integrations/components) menu. Updates: * reorganized the page sections so they follow the Component menu order. * fixed names for the section names: "Text Embedding Models" -> "Embedding Models" --- docs/docs/integrations/platforms/aws.mdx | 170 +++++++++--------- docs/docs/integrations/platforms/google.mdx | 77 ++++---- .../docs/integrations/platforms/microsoft.mdx | 47 +++-- docs/docs/integrations/platforms/openai.mdx | 63 ++++--- 4 files changed, 178 insertions(+), 179 deletions(-) diff --git a/docs/docs/integrations/platforms/aws.mdx b/docs/docs/integrations/platforms/aws.mdx index e64bad38c51..b6b7aa9ab6d 100644 --- a/docs/docs/integrations/platforms/aws.mdx +++ b/docs/docs/integrations/platforms/aws.mdx @@ -67,7 +67,7 @@ See a [usage example](/docs/integrations/chat/bedrock). from langchain_community.chat_models import BedrockChat ``` -## Text Embedding Models +## Embedding Models ### Bedrock @@ -84,26 +84,6 @@ from langchain_community.embeddings import SagemakerEndpointEmbeddings from langchain_community.llms.sagemaker_endpoint import ContentHandlerBase ``` -## Chains - -### Amazon Comprehend Moderation Chain - ->[Amazon Comprehend](https://aws.amazon.com/comprehend/) is a natural-language processing (NLP) service that -> uses machine learning to uncover valuable insights and connections in text. - - -We need to install the `boto3` and `nltk` libraries. - -```bash -pip install boto3 nltk -``` - -See a [usage example](/docs/guides/safety/amazon_comprehend_chain). - -```python -from langchain_experimental.comprehend_moderation import AmazonComprehendModerationChain -``` - ## Document loaders ### AWS S3 Directory and File @@ -132,71 +112,6 @@ See a [usage example](/docs/integrations/document_loaders/amazon_textract). from langchain_community.document_loaders import AmazonTextractPDFLoader ``` -## Memory - -### AWS DynamoDB - ->[AWS DynamoDB](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/dynamodb/index.html) -> is a fully managed `NoSQL` database service that provides fast and predictable performance with seamless scalability. - -We have to configure the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html). - -We need to install the `boto3` library. - -```bash -pip install boto3 -``` - -See a [usage example](/docs/integrations/memory/aws_dynamodb). - -```python -from langchain.memory import DynamoDBChatMessageHistory -``` - -## Retrievers - -### Amazon Kendra - -> [Amazon Kendra](https://docs.aws.amazon.com/kendra/latest/dg/what-is-kendra.html) is an intelligent search service -> provided by `Amazon Web Services` (`AWS`). It utilizes advanced natural language processing (NLP) and machine -> learning algorithms to enable powerful search capabilities across various data sources within an organization. -> `Kendra` is designed to help users find the information they need quickly and accurately, -> improving productivity and decision-making. - -> With `Kendra`, we can search across a wide range of content types, including documents, FAQs, knowledge bases, -> manuals, and websites. It supports multiple languages and can understand complex queries, synonyms, and -> contextual meanings to provide highly relevant search results. - -We need to install the `boto3` library. - -```bash -pip install boto3 -``` - -See a [usage example](/docs/integrations/retrievers/amazon_kendra_retriever). - -```python -from langchain.retrievers import AmazonKendraRetriever -``` - -### Amazon Bedrock (Knowledge Bases) - -> [Knowledge bases for Amazon Bedrock](https://aws.amazon.com/bedrock/knowledge-bases/) is an -> `Amazon Web Services` (`AWS`) offering which lets you quickly build RAG applications by using your -> private data to customize foundation model response. - -We need to install the `boto3` library. - -```bash -pip install boto3 -``` - -See a [usage example](/docs/integrations/retrievers/bedrock). - -```python -from langchain.retrievers import AmazonKnowledgeBasesRetriever -``` - ## Vector stores ### Amazon OpenSearch Service @@ -248,6 +163,49 @@ See a [usage example](/docs/integrations/vectorstores/documentdb). from langchain.vectorstores import DocumentDBVectorSearch ``` +## Retrievers + +### Amazon Kendra + +> [Amazon Kendra](https://docs.aws.amazon.com/kendra/latest/dg/what-is-kendra.html) is an intelligent search service +> provided by `Amazon Web Services` (`AWS`). It utilizes advanced natural language processing (NLP) and machine +> learning algorithms to enable powerful search capabilities across various data sources within an organization. +> `Kendra` is designed to help users find the information they need quickly and accurately, +> improving productivity and decision-making. + +> With `Kendra`, we can search across a wide range of content types, including documents, FAQs, knowledge bases, +> manuals, and websites. It supports multiple languages and can understand complex queries, synonyms, and +> contextual meanings to provide highly relevant search results. + +We need to install the `boto3` library. + +```bash +pip install boto3 +``` + +See a [usage example](/docs/integrations/retrievers/amazon_kendra_retriever). + +```python +from langchain.retrievers import AmazonKendraRetriever +``` + +### Amazon Bedrock (Knowledge Bases) + +> [Knowledge bases for Amazon Bedrock](https://aws.amazon.com/bedrock/knowledge-bases/) is an +> `Amazon Web Services` (`AWS`) offering which lets you quickly build RAG applications by using your +> private data to customize foundation model response. + +We need to install the `boto3` library. + +```bash +pip install boto3 +``` + +See a [usage example](/docs/integrations/retrievers/bedrock). + +```python +from langchain.retrievers import AmazonKnowledgeBasesRetriever +``` ## Tools @@ -267,6 +225,26 @@ pip install boto3 See a [usage example](/docs/integrations/tools/awslambda). +## Memory + +### AWS DynamoDB + +>[AWS DynamoDB](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/dynamodb/index.html) +> is a fully managed `NoSQL` database service that provides fast and predictable performance with seamless scalability. + +We have to configure the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html). + +We need to install the `boto3` library. + +```bash +pip install boto3 +``` + +See a [usage example](/docs/integrations/memory/aws_dynamodb). + +```python +from langchain.memory import DynamoDBChatMessageHistory +``` ## Callbacks @@ -290,3 +268,23 @@ See a [usage example](/docs/integrations/callbacks/sagemaker_tracking). ```python from langchain.callbacks import SageMakerCallbackHandler ``` + +## Chains + +### Amazon Comprehend Moderation Chain + +>[Amazon Comprehend](https://aws.amazon.com/comprehend/) is a natural-language processing (NLP) service that +> uses machine learning to uncover valuable insights and connections in text. + + +We need to install the `boto3` and `nltk` libraries. + +```bash +pip install boto3 nltk +``` + +See a [usage example](/docs/guides/safety/amazon_comprehend_chain). + +```python +from langchain_experimental.comprehend_moderation import AmazonComprehendModerationChain +``` diff --git a/docs/docs/integrations/platforms/google.mdx b/docs/docs/integrations/platforms/google.mdx index c504ab65c29..d0ccca1a0ab 100644 --- a/docs/docs/integrations/platforms/google.mdx +++ b/docs/docs/integrations/platforms/google.mdx @@ -20,25 +20,9 @@ See a [usage example](/docs/integrations/llms/google_ai). from langchain_google_genai import GoogleGenerativeAI ``` -### Vertex AI +### Vertex AI Model Garden -Access to `Gemini` and `PaLM` LLMs (like `text-bison` and `code-bison`) via `Vertex AI` on Google Cloud. - -We need to install `langchain-google-vertexai` python package. - -```bash -pip install langchain-google-vertexai -``` - -See a [usage example](/docs/integrations/llms/google_vertex_ai_palm). - -```python -from langchain_google_vertexai import VertexAI -``` - -### Model Garden - -Access PaLM and hundreds of OSS models via `Vertex AI Model Garden` on Google Cloud. +Access `PaLM` and hundreds of OSS models via `Vertex AI Model Garden` service. We need to install `langchain-google-vertexai` python package. @@ -52,6 +36,7 @@ See a [usage example](/docs/integrations/llms/google_vertex_ai_palm#vertex-model from langchain_google_vertexai import VertexAIModelGarden ``` + ## Chat models ### Google Generative AI @@ -119,6 +104,40 @@ See a [usage example](/docs/integrations/chat/google_vertex_ai_palm). from langchain_google_vertexai import ChatVertexAI ``` +## Embedding models + +### Google Generative AI Embeddings + +See a [usage example](/docs/integrations/text_embedding/google_generative_ai). + +```bash +pip install -U langchain-google-genai +``` + +Configure your API key. + +```bash +export GOOGLE_API_KEY=your-api-key +``` + +```python +from langchain_google_genai import GoogleGenerativeAIEmbeddings +``` + +### Vertex AI + +We need to install `langchain-google-vertexai` python package. + +```bash +pip install langchain-google-vertexai +``` + +See a [usage example](/docs/integrations/text_embedding/google_vertex_ai_palm). + +```python +from langchain_google_vertexai import VertexAIEmbeddings +``` + ## Document Loaders ### AlloyDB for PostgreSQL @@ -797,22 +816,6 @@ See [usage example](/docs/integrations/memory/google_cloud_sql_mssql). from langchain_google_cloud_sql_mssql import MSSQLEngine, MSSQLChatMessageHistory ``` -## El Carro for Oracle Workloads - -> Google [El Carro Oracle Operator](https://github.com/GoogleCloudPlatform/elcarro-oracle-operator) -offers a way to run Oracle databases in Kubernetes as a portable, open source, -community driven, no vendor lock-in container orchestration system. - -```bash -pip install langchain-google-el-carro -``` - -See [usage example](/docs/integrations/memory/google_el_carro). - -```python -from langchain_google_el_carro import ElCarroChatMessageHistory -``` - ### Spanner > [Google Cloud Spanner](https://cloud.google.com/spanner/docs) is a fully managed, mission-critical, relational database service on Google Cloud that offers transactional consistency at global scale, automatic, synchronous replication for high availability, and support for two SQL dialects: GoogleSQL (ANSI 2011 with extensions) and PostgreSQL. @@ -889,10 +892,10 @@ See [usage example](/docs/integrations/memory/google_datastore). from langchain_google_datastore import DatastoreChatMessageHistory ``` -## El Carro Oracle Operator +### El Carro: The Oracle Operator for Kubernetes -> Google [El Carro Oracle Operator](https://github.com/GoogleCloudPlatform/elcarro-oracle-operator) -offers a way to run Oracle databases in Kubernetes as a portable, open source, +> Google [El Carro Oracle Operator for Kubernetes](https://github.com/GoogleCloudPlatform/elcarro-oracle-operator) +offers a way to run `Oracle` databases in `Kubernetes` as a portable, open source, community driven, no vendor lock-in container orchestration system. ```bash diff --git a/docs/docs/integrations/platforms/microsoft.mdx b/docs/docs/integrations/platforms/microsoft.mdx index 898ff688cd1..8eaf7a3a04a 100644 --- a/docs/docs/integrations/platforms/microsoft.mdx +++ b/docs/docs/integrations/platforms/microsoft.mdx @@ -2,6 +2,15 @@ All functionality related to `Microsoft Azure` and other `Microsoft` products. +## LLMs +### Azure OpenAI + +See a [usage example](/docs/integrations/llms/azure_openai). + +```python +from langchain_openai import AzureOpenAI +``` + ## Chat Models ### Azure OpenAI @@ -29,7 +38,7 @@ See a [usage example](/docs/integrations/chat/azure_chat_openai) from langchain_openai import AzureChatOpenAI ``` -## Text Embedding Models +## Embedding Models ### Azure OpenAI See a [usage example](/docs/integrations/text_embedding/azureopenai) @@ -38,15 +47,6 @@ See a [usage example](/docs/integrations/text_embedding/azureopenai) from langchain_openai import AzureOpenAIEmbeddings ``` -## LLMs -### Azure OpenAI - -See a [usage example](/docs/integrations/llms/azure_openai). - -```python -from langchain_openai import AzureOpenAI -``` - ## Document loaders ### Azure AI Data @@ -209,7 +209,6 @@ See a [usage example](/docs/integrations/document_loaders/microsoft_onenote). from langchain_community.document_loaders.onenote import OneNoteLoader ``` - ## Vector stores ### Azure Cosmos DB @@ -262,19 +261,6 @@ See a [usage example](/docs/integrations/retrievers/azure_cognitive_search). from langchain.retrievers import AzureCognitiveSearchRetriever ``` -## Utilities - -### Bing Search API - ->[Microsoft Bing](https://www.bing.com/), commonly referred to as `Bing` or `Bing Search`, -> is a web search engine owned and operated by `Microsoft`. - -See a [usage example](/docs/integrations/tools/bing_search). - -```python -from langchain_community.utilities import BingSearchAPIWrapper -``` - ## Toolkits ### Azure Cognitive Services @@ -320,6 +306,19 @@ from langchain_community.agent_toolkits import PowerBIToolkit from langchain_community.utilities.powerbi import PowerBIDataset ``` +## Utilities + +### Bing Search API + +>[Microsoft Bing](https://www.bing.com/), commonly referred to as `Bing` or `Bing Search`, +> is a web search engine owned and operated by `Microsoft`. + +See a [usage example](/docs/integrations/tools/bing_search). + +```python +from langchain_community.utilities import BingSearchAPIWrapper +``` + ## More ### Microsoft Presidio diff --git a/docs/docs/integrations/platforms/openai.mdx b/docs/docs/integrations/platforms/openai.mdx index 415aed521b5..0e8f25d266c 100644 --- a/docs/docs/integrations/platforms/openai.mdx +++ b/docs/docs/integrations/platforms/openai.mdx @@ -36,7 +36,6 @@ from langchain_openai import AzureOpenAI ``` For a more detailed walkthrough of the `Azure` wrapper, see [here](/docs/integrations/llms/azure_openai) - ## Chat model See a [usage example](/docs/integrations/chat/openai). @@ -51,8 +50,7 @@ from langchain_openai import AzureChatOpenAI ``` For a more detailed walkthrough of the `Azure` wrapper, see [here](/docs/integrations/chat/azure_chat_openai) - -## Text Embedding Model +## Embedding Model See a [usage example](/docs/integrations/text_embedding/openai) @@ -60,19 +58,6 @@ See a [usage example](/docs/integrations/text_embedding/openai) from langchain_openai import OpenAIEmbeddings ``` - -## Tokenizer - -There are several places you can use the `tiktoken` tokenizer. By default, it is used to count tokens -for OpenAI LLMs. - -You can also use it to count tokens when splitting documents with -```python -from langchain_text_splitters import CharacterTextSplitter -CharacterTextSplitter.from_tiktoken_encoder(...) -``` -For a more detailed walkthrough of this, see [this notebook](/docs/modules/data_connection/document_transformers/split_by_token#tiktoken) - ## Document Loader See a [usage example](/docs/integrations/document_loaders/chatgpt_loader). @@ -89,22 +74,6 @@ See a [usage example](/docs/integrations/retrievers/chatgpt-plugin). from langchain.retrievers import ChatGPTPluginRetriever ``` -## Chain - -See a [usage example](/docs/guides/safety/moderation). - -```python -from langchain.chains import OpenAIModerationChain -``` - -## Adapter - -See a [usage example](/docs/integrations/adapters/openai). - -```python -from langchain.adapters import openai as lc_openai -``` - ## Tools ### Dall-E Image Generator @@ -119,3 +88,33 @@ See a [usage example](/docs/integrations/tools/dalle_image_generator). ```python from langchain_community.utilities.dalle_image_generator import DallEAPIWrapper ``` + +## Adapter + +See a [usage example](/docs/integrations/adapters/openai). + +```python +from langchain.adapters import openai as lc_openai +``` + +## Tokenizer + +There are several places you can use the `tiktoken` tokenizer. By default, it is used to count tokens +for OpenAI LLMs. + +You can also use it to count tokens when splitting documents with +```python +from langchain.text_splitter import CharacterTextSplitter +CharacterTextSplitter.from_tiktoken_encoder(...) +``` +For a more detailed walkthrough of this, see [this notebook](/docs/modules/data_connection/document_transformers/split_by_token#tiktoken) + +## Chain + +See a [usage example](/docs/guides/safety/moderation). + +```python +from langchain.chains import OpenAIModerationChain +``` + +