github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-06-23 07:09:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

community[patch]: Invoke callback prior to yielding token fix [DeepInfra] (#20427)

Browse Source 
- [x] **PR title**: community[patch]: Invoke callback prior to yielding
token fix for [DeepInfra]


- [x] **PR message**: 
- **Description:** Invoke callback prior to yielding token in stream
method in [DeepInfra]
    - **Issue:** https://github.com/langchain-ai/langchain/issues/16913
    - **Dependencies:** None
    - **Twitter handle:** @bolun_zhang

If no one reviews your PR within a few days, please @-mention one of
baskaryan, efriis, eyurtsev, hwchase17.
This commit is contained in:
balloonio 2024-04-14 14:32:52 -04:00 committed by GitHub
parent 450c458f8f
commit b66a4f48fa
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
libs/community/langchain_community/llms/deepinfra.py
View File

@ -155,9 +155,9 @@ class DeepInfra(LLM):
for line in _parse_stream(response.iter_lines()): for line in _parse_stream(response.iter_lines()):
chunk = _handle_sse_line(line) chunk = _handle_sse_line(line)
if chunk: if chunk:
yield chunk
if run_manager: if run_manager:
run_manager.on_llm_new_token(chunk.text) run_manager.on_llm_new_token(chunk.text)
yield chunk
async def _astream( async def _astream(
self, self,
@ -174,9 +174,9 @@ class DeepInfra(LLM):
async for line in _parse_stream_async(response.content): async for line in _parse_stream_async(response.content):
chunk = _handle_sse_line(line) chunk = _handle_sse_line(line)
if chunk: if chunk:
yield chunk
if run_manager: if run_manager:
await run_manager.on_llm_new_token(chunk.text) await run_manager.on_llm_new_token(chunk.text)
yield chunk
def _parse_stream(rbody: Iterator[bytes]) -> Iterator[str]: def _parse_stream(rbody: Iterator[bytes]) -> Iterator[str]: