mirror of
https://github.com/hwchase17/langchain.git
synced 2025-07-02 03:15:11 +00:00
Wfh/google docs update (#14676)
- Add gemini references - Fix the notebook (ultra isn't generally available; also gemini will randomly filter out responses, so added a fallback) --------- Co-authored-by: Leonid Kuligin <lkuligin@yandex.ru>
This commit is contained in:
parent
73382a579f
commit
6c031e0ebf
699
cookbook/Multi_modal_RAG_google.ipynb
Normal file
699
cookbook/Multi_modal_RAG_google.ipynb
Normal file
File diff suppressed because one or more lines are too long
@ -2,42 +2,57 @@
|
|||||||
|
|
||||||
All functionality related to [Google Cloud Platform](https://cloud.google.com/) and other `Google` products.
|
All functionality related to [Google Cloud Platform](https://cloud.google.com/) and other `Google` products.
|
||||||
|
|
||||||
## LLMs
|
|
||||||
|
|
||||||
### Vertex AI
|
|
||||||
|
|
||||||
Access `PaLM` LLMs like `text-bison` and `code-bison` via `Google Vertex AI`.
|
|
||||||
|
|
||||||
We need to install `google-cloud-aiplatform` python package.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install google-cloud-aiplatform
|
|
||||||
```
|
|
||||||
|
|
||||||
See a [usage example](/docs/integrations/llms/google_vertex_ai_palm).
|
|
||||||
|
|
||||||
```python
|
|
||||||
from langchain.llms import VertexAI
|
|
||||||
```
|
|
||||||
|
|
||||||
### Model Garden
|
|
||||||
|
|
||||||
Access PaLM and hundreds of OSS models via `Vertex AI Model Garden`.
|
|
||||||
|
|
||||||
We need to install `google-cloud-aiplatform` python package.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install google-cloud-aiplatform
|
|
||||||
```
|
|
||||||
|
|
||||||
See a [usage example](/docs/integrations/llms/google_vertex_ai_palm#vertex-model-garden).
|
|
||||||
|
|
||||||
```python
|
|
||||||
from langchain.llms import VertexAIModelGarden
|
|
||||||
```
|
|
||||||
|
|
||||||
## Chat models
|
## Chat models
|
||||||
|
|
||||||
|
### ChatGoogleGenerativeAI
|
||||||
|
|
||||||
|
Access `Gemini` models such as `gemini-pro` and `gemini-pro-vision` through the `ChatGoogleGenerativeAI` class.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -U langchain-google-genai
|
||||||
|
```
|
||||||
|
|
||||||
|
Configure your API key.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export GOOGLE_API_KEY=your-api-key
|
||||||
|
```
|
||||||
|
|
||||||
|
```python
|
||||||
|
from langchain_google_genai import ChatGoogleGenerativeAI
|
||||||
|
|
||||||
|
llm = ChatGoogleGenerativeAI(model="gemini-pro")
|
||||||
|
llm.invoke("Sing a ballad of LangChain.")
|
||||||
|
```
|
||||||
|
|
||||||
|
Gemini vision model supports image inputs when providing a single chat message. Example:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from langchain_core.messages import HumanMessage
|
||||||
|
from langchain_google_genai import ChatGoogleGenerativeAI
|
||||||
|
|
||||||
|
llm = ChatGoogleGenerativeAI(model="gemini-pro-vision")
|
||||||
|
# example
|
||||||
|
message = HumanMessage(
|
||||||
|
content=[
|
||||||
|
{
|
||||||
|
"type": "text",
|
||||||
|
"text": "What's in this image?",
|
||||||
|
}, # You can optionally provide text parts
|
||||||
|
{"type": "image_url", "image_url": "https://picsum.photos/seed/picsum/200/300"},
|
||||||
|
]
|
||||||
|
)
|
||||||
|
llm.invoke([message])
|
||||||
|
```
|
||||||
|
|
||||||
|
The value of image_url can be any of the following:
|
||||||
|
|
||||||
|
- A public image URL
|
||||||
|
- A gcs file (e.g., "gcs://path/to/file.png")
|
||||||
|
- A local file path
|
||||||
|
- A base64 encoded image (e.g., data:image/png;base64,abcd124)
|
||||||
|
- A PIL image
|
||||||
|
|
||||||
### Vertex AI
|
### Vertex AI
|
||||||
|
|
||||||
Access PaLM chat models like `chat-bison` and `codechat-bison` via Google Cloud.
|
Access PaLM chat models like `chat-bison` and `codechat-bison` via Google Cloud.
|
||||||
@ -72,6 +87,41 @@ See a [usage example](/docs/integrations/document_loaders/google_bigquery).
|
|||||||
from langchain.document_loaders import BigQueryLoader
|
from langchain.document_loaders import BigQueryLoader
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## LLMs
|
||||||
|
|
||||||
|
### Vertex AI
|
||||||
|
|
||||||
|
Access to `Gemini` and `PaLM` LLMs (like `text-bison` and `code-bison`) via `Google Vertex AI`.
|
||||||
|
|
||||||
|
We need to install `google-cloud-aiplatform` python package.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install google-cloud-aiplatform
|
||||||
|
```
|
||||||
|
|
||||||
|
See a [usage example](/docs/integrations/llms/google_vertex_ai_palm).
|
||||||
|
|
||||||
|
```python
|
||||||
|
from langchain.llms import VertexAI
|
||||||
|
```
|
||||||
|
|
||||||
|
### Model Garden
|
||||||
|
|
||||||
|
Access PaLM and hundreds of OSS models via `Vertex AI Model Garden`.
|
||||||
|
|
||||||
|
We need to install `google-cloud-aiplatform` python package.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install google-cloud-aiplatform
|
||||||
|
```
|
||||||
|
|
||||||
|
See a [usage example](/docs/integrations/llms/google_vertex_ai_palm#vertex-model-garden).
|
||||||
|
|
||||||
|
```python
|
||||||
|
from langchain.llms import VertexAIModelGarden
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
### Google Cloud Storage
|
### Google Cloud Storage
|
||||||
|
|
||||||
>[Google Cloud Storage](https://en.wikipedia.org/wiki/Google_Cloud_Storage) is a managed service for storing unstructured data.
|
>[Google Cloud Storage](https://en.wikipedia.org/wiki/Google_Cloud_Storage) is a managed service for storing unstructured data.
|
||||||
|
Loading…
Reference in New Issue
Block a user