langchain

mirror of https://github.com/hwchase17/langchain.git synced 2025-07-19 11:08:55 +00:00

⚡ Building applications with LLMs through composability ⚡

Go to file

Bob Merkus 5700646cc5 ollama: add reasoning model support (e.g. deepseek) (#29689 ) # Description This PR adds reasoning model support for `langchain-ollama` by extracting reasoning token blocks, like those used in deepseek. It was inspired by [ollama-deep-researcher](https://github.com/langchain-ai/ollama-deep-researcher), specifically the parsing of [thinking blocks](`6d1aaf2139/src/assistant/graph.py (L91)`): ```python # TODO: This is a hack to remove the <think> tags w/ Deepseek models # It appears very challenging to prompt them out of the responses while "<think>" in running_summary and "</think>" in running_summary: start = running_summary.find("<think>") end = running_summary.find("</think>") + len("</think>") running_summary = running_summary[:start] + running_summary[end:] ``` This notes that it is very hard to remove the reasoning block from prompting, but we actually want the model to reason in order to increase model performance. This implementation extracts the thinking block, so the client can still expect a proper message to be returned by `ChatOllama` (and use the reasoning content separately when desired). This implementation takes the same approach as [ChatDeepseek](`5d581ba22c/libs/partners/deepseek/langchain_deepseek/chat_models.py (L215)`), which adds the reasoning content to chunk.additional_kwargs.reasoning_content; ```python if hasattr(response.choices[0].message, "reasoning_content"): # type: ignore rtn.generations[0].message.additional_kwargs["reasoning_content"] = ( response.choices[0].message.reasoning_content # type: ignore ) ``` This should probably be handled upstream in ollama + ollama-python, but this seems like a reasonably effective solution. This is a standalone example of what is happening; ```python async def deepseek_message_astream( llm: BaseChatModel, messages: list[BaseMessage], config: RunnableConfig \| None = None, , model_target: str = "deepseek-r1", kwargs: Any, ) -> AsyncIterator[BaseMessageChunk]: """Stream responses from Deepseek models, filtering out <think> tags. Args: llm: The language model to stream from messages: The messages to send to the model Yields: Filtered chunks from the model response """ # check if the model is deepseek based if (llm.name and model_target not in llm.name) or (hasattr(llm, "model") and model_target not in llm.model): async for chunk in llm.astream(messages, config=config, kwargs): yield chunk return # Yield with a buffer, upon completing the <think></think> tags, move them to the reasoning content and start over buffer = "" async for chunk in llm.astream(messages, config=config, *kwargs): # start or append if not buffer: buffer = chunk.content else: buffer += chunk.content if hasattr(chunk, "content") else chunk # Process buffer to remove <think> tags if "<think>" in buffer or "</think>" in buffer: if hasattr(chunk, "tool_calls") and chunk.tool_calls: raise NotImplementedError("tool calls during reasoning should be removed?") if "<think>" in chunk.content or "</think>" in chunk.content: continue chunk.additional_kwargs["reasoning_content"] = chunk.content chunk.content = "" # upon block completion, reset the buffer if "<think>" in buffer and "</think>" in buffer: buffer = "" yield chunk ``` # Issue Integrating reasoning models (e.g. deepseek-r1) into existing LangChain based workflows is hard due to the thinking blocks that are included in the message contents. To avoid this, we could match the `ChatOllama` integration with `ChatDeepseek` to return the reasoning content inside `message.additional_arguments.reasoning_content` instead. # Dependenices None --------- Co-authored-by: Chester Curme <chester.curme@gmail.com>		2025-03-21 15:44:54 +00:00
.devcontainer	community[minor]: Add ApertureDB as a vectorstore (#24088 )	2024-07-16 09:32:59 -07:00
.github	infra(GHA): description is required based on schema definition (#30305 )	2025-03-17 18:42:42 +00:00
cookbook	docs: Correct grammatical typos in various documentation files (#29983 )	2025-02-25 19:13:31 +00:00
docs	docs: add links in Writer provider page (#30399 )	2025-03-20 16:13:48 -04:00
libs	ollama: add reasoning model support (e.g. deepseek) (#29689 )	2025-03-21 15:44:54 +00:00
scripts	infra: update mypy 1.10, ruff 0.5 (#23721 )	2024-07-03 10:33:27 -07:00
.gitattributes
.gitignore	infra: gitignore api_ref mds (#25705 )	2024-08-23 09:50:30 -07:00
.pre-commit-config.yaml	docs: fix builds (#29890 )	2025-02-19 13:35:59 -05:00
.readthedocs.yaml	docs(readthedocs): streamline config (#30307 )	2025-03-18 11:47:45 -04:00
CITATION.cff
LICENSE
Makefile	langchain: clean pyproject ruff section (#30070 )	2025-03-09 15:06:02 -04:00
MIGRATE.md	Proofreading and Editing Report for Migration Guide (#28084 )	2024-11-13 11:03:09 -05:00
poetry.toml	multiple: use modern installer in poetry (#23998 )	2024-07-08 18:50:48 -07:00
pyproject.toml	langchain: clean pyproject ruff section (#30070 )	2025-03-09 15:06:02 -04:00
README.md	docs: update readme (#30239 )	2025-03-12 13:45:13 -04:00
SECURITY.md	docs: single security doc (#28515 )	2024-12-04 18:15:34 +00:00
uv.lock	openai[patch]: support Responses API (#30231 )	2025-03-12 12:25:46 -04:00
yarn.lock	box: add langchain box package and DocumentLoader (#25506 )	2024-08-21 02:23:43 +00:00

README.md

Note

Looking for the JS/TS library? Check out LangChain.js.

LangChain is a framework for building LLM-powered applications. It helps you chain together interoperable components and third-party integrations to simplify AI application development — all while future-proofing decisions as the underlying technology evolves.

pip install -U langchain

To learn more about LangChain, check out the docs. If you’re looking for more advanced customization or agent orchestration, check out LangGraph, our framework for building controllable agent workflows.

Why use LangChain?

LangChain helps developers build applications powered by LLMs through a standard interface for models, embeddings, vector stores, and more.

Use LangChain for:

Real-time data augmentation. Easily connect LLMs to diverse data sources and external / internal systems, drawing from LangChain’s vast library of integrations with model providers, tools, vector stores, retrievers, and more.
Model interoperability. Swap models in and out as your engineering team experiments to find the best choice for your application’s needs. As the industry frontier evolves, adapt quickly — LangChain’s abstractions keep you moving without losing momentum.

LangChain’s ecosystem

While the LangChain framework can be used standalone, it also integrates seamlessly with any LangChain product, giving developers a full suite of tools when building LLM applications.

To improve your LLM application development, pair LangChain with:

LangSmith - Helpful for agent evals and observability. Debug poor-performing LLM app runs, evaluate agent trajectories, gain visibility in production, and improve performance over time.
LangGraph - Build agents that can reliably handle complex tasks with LangGraph, our low-level agent orchestration framework. LangGraph offers customizable architecture, long-term memory, and human-in-the-loop workflows — and is trusted in production by companies like LinkedIn, Uber, Klarna, and GitLab.
LangGraph Platform - Deploy and scale agents effortlessly with a purpose-built deployment platform for long running, stateful workflows. Discover, reuse, configure, and share agents across teams — and iterate quickly with visual prototyping in LangGraph Studio.

Additional resources

Tutorials: Simple walkthroughs with guided examples on getting started with LangChain.
How-to Guides: Quick, actionable code snippets for topics such as tool calling, RAG use cases, and more.
Conceptual Guides: Explanations of key concepts behind the LangChain framework.
API Reference: Detailed reference on navigating base packages and integrations for LangChain.

README.md Unescape Escape

Why use LangChain?

LangChain’s ecosystem

Additional resources

README.md