mirror of
https://github.com/hwchase17/langchain.git
synced 2025-07-19 19:11:33 +00:00
mistralai: Fixed streaming in MistralAI with ainvoke and callbacks (#22000)
# Fix streaming in mistral with ainvoke - [x] **PR title** - [x] **PR message** - [x] **Add tests and docs**: 1. [x] Added a test for the fixed integration. 2. [x] An example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] **Lint and test**: Ran `make format`, `make lint` and `make test` from the root of the package(s) I've modified. Hello * I Identified an issue in the mistral package where the callback streaming (see on_llm_new_token) was not functioning correctly when the streaming parameter was set to True and call with `ainvoke`. * The root cause of the problem was the streaming not taking into account. ( I think it's an oversight ) * To resolve the issue, I added the `streaming` attribut. * Now, the callback with streaming works as expected when the streaming parameter is set to True. ## How to reproduce ``` from langchain_mistralai.chat_models import ChatMistralAI chain = ChatMistralAI(streaming=True) # Add a callback chain.ainvoke(..) # Oberve on_llm_new_token # Now, the callback is given as streaming tokens, before it was in grouped format. ``` Co-authored-by: Erick Friis <erick@langchain.dev>
This commit is contained in:
parent
f4b2e553e7
commit
5b1de2ae93
@ -586,7 +586,7 @@ class ChatMistralAI(BaseChatModel):
|
||||
stream: Optional[bool] = None,
|
||||
**kwargs: Any,
|
||||
) -> ChatResult:
|
||||
should_stream = stream if stream is not None else False
|
||||
should_stream = stream if stream is not None else self.streaming
|
||||
if should_stream:
|
||||
stream_iter = self._astream(
|
||||
messages=messages, stop=stop, run_manager=run_manager, **kwargs
|
||||
|
Loading…
Reference in New Issue
Block a user