mirror of
https://github.com/hwchase17/langchain.git
synced 2025-09-24 03:52:10 +00:00
mongodb[minor]: Add MongoDB LLM Cache (#17470)
# Description - **Description:** Adding MongoDB LLM Caching Layer abstraction - **Issue:** N/A - **Dependencies:** None - **Twitter handle:** @mongodb Checklist: - [x] PR title: Please title your PR "package: description", where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [x] PR Message (above) - [x] Pass lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified to check that you're passing lint and testing. See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/ - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @efriis, @eyurtsev, @hwchase17. --------- Co-authored-by: Jib <jib@byblack.us>
This commit is contained in:
@@ -22,3 +22,61 @@ See a [usage example](/docs/integrations/vectorstores/mongodb_atlas).
|
||||
from langchain_mongodb import MongoDBAtlasVectorSearch
|
||||
```
|
||||
|
||||
|
||||
## LLM Caches
|
||||
|
||||
### MongoDBCache
|
||||
An abstraction to store a simple cache in MongoDB. This does not use Semantic Caching, nor does it require an index to be made on the collection before generation.
|
||||
|
||||
To import this cache:
|
||||
```python
|
||||
from langchain_mongodb.cache import MongoDBCache
|
||||
```
|
||||
|
||||
To use this cache with your LLMs:
|
||||
```python
|
||||
from langchain_core.globals import set_llm_cache
|
||||
|
||||
# use any embedding provider...
|
||||
from tests.integration_tests.vectorstores.fake_embeddings import FakeEmbeddings
|
||||
|
||||
mongodb_atlas_uri = "<YOUR_CONNECTION_STRING>"
|
||||
COLLECTION_NAME="<YOUR_CACHE_COLLECTION_NAME>"
|
||||
DATABASE_NAME="<YOUR_DATABASE_NAME>"
|
||||
|
||||
set_llm_cache(MongoDBCache(
|
||||
connection_string=mongodb_atlas_uri,
|
||||
collection_name=COLLECTION_NAME,
|
||||
database_name=DATABASE_NAME,
|
||||
))
|
||||
```
|
||||
|
||||
|
||||
### MongoDBAtlasSemanticCache
|
||||
Semantic caching allows users to retrieve cached prompts based on semantic similarity between the user input and previously cached results. Under the hood it blends MongoDBAtlas as both a cache and a vectorstore.
|
||||
The MongoDBAtlasSemanticCache inherits from `MongoDBAtlasVectorSearch` and needs an Atlas Vector Search Index defined to work. Please look at the [usage example](/docs/integrations/vectorstores/mongodb_atlas) on how to set up the index.
|
||||
|
||||
To import this cache:
|
||||
```python
|
||||
from langchain_mongodb.cache import MongoDBAtlasSemanticCache
|
||||
```
|
||||
|
||||
To use this cache with your LLMs:
|
||||
```python
|
||||
from langchain_core.globals import set_llm_cache
|
||||
|
||||
# use any embedding provider...
|
||||
from tests.integration_tests.vectorstores.fake_embeddings import FakeEmbeddings
|
||||
|
||||
mongodb_atlas_uri = "<YOUR_CONNECTION_STRING>"
|
||||
COLLECTION_NAME="<YOUR_CACHE_COLLECTION_NAME>"
|
||||
DATABASE_NAME="<YOUR_DATABASE_NAME>"
|
||||
|
||||
set_llm_cache(MongoDBAtlasSemanticCache(
|
||||
embedding=FakeEmbeddings(),
|
||||
connection_string=mongodb_atlas_uri,
|
||||
collection_name=COLLECTION_NAME,
|
||||
database_name=DATABASE_NAME,
|
||||
))
|
||||
```
|
||||
``
|
Reference in New Issue
Block a user