github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-07-05 12:48:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

chat_models.openai: Set tenacity timeout to openai's recommendation (#2768)

Browse Source 
[OpenAI's
cookbook](https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb)
suggest a tenacity backoff between 1 and 60 seconds. Currently
langchain's backoff is between 4 and 10 seconds, which causes frequent
timeout errors on my end.

This PR changes the timeout to the suggested values.
This commit is contained in:
rafael 2023-04-14 07:08:46 +02:00 committed by GitHub
parent 705596b46a
commit 1cc7ea333c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
langchain/chat_models/openai.py
View File

@ -32,8 +32,8 @@ logger = logging.getLogger(__file__)
def _create_retry_decorator(llm: ChatOpenAI) -> Callable[[Any], Any]:
import openai
min_seconds = 4
max_seconds = 10
min_seconds = 1
max_seconds = 60
# Wait 2^x * 1 second between each retry starting with
# 4 seconds, then up to 10 seconds, then 10 seconds afterwards
return retry(
@ -199,8 +199,8 @@ class ChatOpenAI(BaseChatModel):
def _create_retry_decorator(self) -> Callable[[Any], Any]:
import openai
min_seconds = 4
max_seconds = 10
min_seconds = 1
max_seconds = 60
# Wait 2^x * 1 second between each retry starting with
# 4 seconds, then up to 10 seconds, then 10 seconds afterwards
return retry(