github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-08-13 14:50:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

community[patch]: OpenLLM Async Client Fixes and Timeout Parameter (#20007)

Browse Source 
Same changes as this merged
[PR](https://github.com/langchain-ai/langchain/pull/17478)
(https://github.com/langchain-ai/langchain/pull/17478), but for the
async client, as the same issues persist.

- Replaced 'responses' attribute of OpenLLM's GenerationOutput schema to
'outputs'.
reference:
66de54eae7/openllm-core/src/openllm_core/_schemas.py (L135)

- Added timeout parameter for the async client.

---------

Co-authored-by: Seray Arslan <seray.arslan@knime.com>
This commit is contained in:
seray 2024-04-09 22:34:56 +02:00 committed by GitHub
parent 37a9e23c05
commit add31f46d0
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 5 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
libs/community/langchain_community/llms/openllm.py
View File

@ -308,10 +308,12 @@ class OpenLLM(LLM):
self._identifying_params["model_name"], **copied self._identifying_params["model_name"], **copied
) )
if self._client: if self._client:
async_client = openllm.client.AsyncHTTPClient(self.server_url) async_client = openllm.client.AsyncHTTPClient(self.server_url, self.timeout)
res = ( res = (
await async_client.generate(prompt, **config.model_dump(flatten=True)) (await async_client.generate(prompt, **config.model_dump(flatten=True)))
).responses[0] .outputs[0]
.text
)
else: else:
assert self._runner is not None assert self._runner is not None
( (