Add support Vertex AI Gemini uses a public image URL (#14949)

## What

Since `langchain_google_genai.ChatGoogleGenerativeAI` supported A public
image URL, we add to support it in `langchain.chat_models.ChatVertexAI`
as well.

### Example

```py
from langchain.chat_models.vertexai import ChatVertexAI
from langchain_core.messages import HumanMessage

llm = ChatVertexAI(model_name="gemini-pro-vision")
image_message = {
    "type": "image_url",
    "image_url": {
        "url": "https://python.langchain.com/assets/images/cell-18-output-1-0c7fb8b94ff032d51bfe1880d8370104.png",
    },
}
text_message = {
    "type": "text",
    "text": "What is shown in this image?",
}
message = HumanMessage(content=[text_message, image_message])

output = llm([message])
print(output.content)
```

## Refs

-
https://python.langchain.com/docs/integrations/llms/google_vertex_ai_palm
-
https://python.langchain.com/docs/integrations/chat/google_generative_ai
This commit is contained in:
Takuya Igei
2023-12-23 06:19:09 +09:00
committed by GitHub
parent affa3e755a
commit 6da2246215
2 changed files with 44 additions and 0 deletions

View File

@@ -539,6 +539,35 @@
"print(output2.content)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can also use the public image URL:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"image_message = {\n",
" \"type\": \"image_url\",\n",
" \"image_url\": {\n",
" \"url\": \"https://python.langchain.com/assets/images/cell-18-output-1-0c7fb8b94ff032d51bfe1880d8370104.png\",\n",
" },\n",
"}\n",
"text_message = {\n",
" \"type\": \"text\",\n",
" \"text\": \"What is shown in this image?\",\n",
"}\n",
"message = HumanMessage(content=[text_message, image_message])\n",
"\n",
"output = llm([message])\n",
"print(output.content)"
]
},
{
"cell_type": "markdown",
"metadata": {