github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-08-11 05:45:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

community[patch]: Invoke callback prior to yielding token fix for Llamafile (#20365)

Browse Source 
- [x] **PR title**: community[patch]: Invoke callback prior to yielding
token fix for Llamafile


- [x] **PR message**: 
- **Description:** Invoke callback prior to yielding token in stream
method in community llamafile.py
    - **Issue:** https://github.com/langchain-ai/langchain/issues/16913
    - **Dependencies:** None
    - **Twitter handle:** @bolun_zhang

If no one reviews your PR within a few days, please @-mention one of
baskaryan, efriis, eyurtsev, hwchase17.
This commit is contained in:
balloonio 2024-04-12 15:26:12 -04:00 committed by GitHub
parent 1b272fa2f4
commit e7b1a44c5b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
libs/community/langchain_community/llms/llamafile.py
View File

@ -297,9 +297,10 @@ class Llamafile(LLM):
for raw_chunk in response.iter_lines(decode_unicode=True): for raw_chunk in response.iter_lines(decode_unicode=True):
content = self._get_chunk_content(raw_chunk) content = self._get_chunk_content(raw_chunk)
chunk = GenerationChunk(text=content) chunk = GenerationChunk(text=content)
yield chunk
if run_manager: if run_manager:
run_manager.on_llm_new_token(token=chunk.text) run_manager.on_llm_new_token(token=chunk.text)
yield chunk
def _get_chunk_content(self, chunk: str) -> str: def _get_chunk_content(self, chunk: str) -> str:
"""When streaming is turned on, llamafile server returns lines like: """When streaming is turned on, llamafile server returns lines like: