github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-06-28 17:38:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

VertexAI LLM count_tokens method requires list of prompts (#13451)

Browse Source 
I encountered this during summarization with VertexAI. I was receiving
an INVALID_ARGUMENT error, as it was trying to send a list of about
17000 single characters.

The [count_tokens
method](https://github.com/googleapis/python-aiplatform/blob/main/vertexai/language_models/_language_models.py#L658)
made available by Google takes in a list of prompts. It does not fail
for small texts, but it does for longer documents because the argument
list will be exceeding Googles allowed limit. Enforcing the list type
makes it work successfully.

This change will cast the input text to count to a list of that single
text so that the input format is always correct.

[Twitter](https://www.x.com/stijn_tratsaert)
This commit is contained in:
Stijn Tratsaert 2023-11-20 18:40:48 +01:00 committed by GitHub
parent fe7b40cb2a
commit b6f70d776b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
libs/langchain/langchain/llms/vertexai.py
View File

@ -288,7 +288,7 @@ class VertexAI(_VertexAICommon, BaseLLM):
The integer number of tokens in the text.
"""
try:
result = self.client.count_tokens(text)
result = self.client.count_tokens([text])
except AttributeError:
raise NotImplementedError(
"Your google-cloud-aiplatform version didn't implement count_tokens."