diff --git a/docs/docs/concepts.mdx b/docs/docs/concepts.mdx index da868750856..4a0c4fb6aad 100644 --- a/docs/docs/concepts.mdx +++ b/docs/docs/concepts.mdx @@ -498,6 +498,29 @@ Retrievers accept a string query as input and return a list of Document's as out For specifics on how to use retrievers, see the [relevant how-to guides here](/docs/how_to/#retrievers). +### Key-value stores + +For some techniques, such as [indexing and retrieval with multiple vectors per document](/docs/how_to/multi_vector/), having some sort of key-value (KV) storage is helpful. + +LangChain includes a [`BaseStore`](https://api.python.langchain.com/en/latest/stores/langchain_core.stores.BaseStore.html) interface, +which allows for storage of arbitrary data. However, LangChain components that require KV-storage accept a +more specific `BaseStore[str, bytes]` instance that stores binary data (referred to as a `ByteStore`), and internally take care of +encoding and decoding data for their specific needs. + +This means that as a user, you only need to think about one type of store rather than different ones for different types of data. + +#### Interface + +All [`BaseStores`](https://api.python.langchain.com/en/latest/stores/langchain_core.stores.BaseStore.html) support the following interface. Note that the interface allows +for modifying **multiple** key-value pairs at once: + +- `mget(key: Sequence[str]) -> List[Optional[bytes]]`: get the contents of multiple keys, returning `None` if the key does not exist +- `mset(key_value_pairs: Sequence[Tuple[str, bytes]]) -> None`: set the contents of multiple keys +- `mdelete(key: Sequence[str]) -> None`: delete multiple keys +- `yield_keys(prefix: Optional[str] = None) -> Iterator[str]`: yield all keys in the store, optionally filtering by a prefix + +For key-value store implementations, see [this section](/docs/integrations/stores/). + ### Tools diff --git a/docs/docs/integrations/stores/index.mdx b/docs/docs/integrations/stores/index.mdx index 5aa9abf1f3a..aa8706d43a3 100644 --- a/docs/docs/integrations/stores/index.mdx +++ b/docs/docs/integrations/stores/index.mdx @@ -3,27 +3,10 @@ sidebar_position: 1 sidebar_class_name: hidden --- -# Stores +# Key-value stores -In many different applications, having some sort of key-value storage is helpful. -In this section, we will look at a few different ways to store key-value pairs -using implementations of the `ByteStore` interface. +[Key-value stores](/docs/concepts/#key-value-stores) are used by other LangChain components to store and retrieve data. -## Features (natively supported) +import DocCardList from "@theme/DocCardList"; -All `ByteStore`s support the following functions, which are used for modifying -**m**ultiple key-value pairs at once: - -- `mget(key: Sequence[str]) -> List[Optional[bytes]]`: get the contents of multiple keys, returning `None` if the key does not exist -- `mset(key_value_pairs: Sequence[Tuple[str, bytes]]) -> None`: set the contents of multiple keys -- `mdelete(key: Sequence[str]) -> None`: delete multiple keys -- `yield_keys(prefix: Optional[str] = None) -> Iterator[str]`: yield all keys in the store, optionally filtering by a prefix - -## How to pick one - -`ByteStore`s are designed to be interchangeable. By default, most dependent integrations -use the `InMemoryByteStore`, which is a simple in-memory key-value store. - -However, if you start having other requirements, like massive scalability or persistence, -you can swap out the `ByteStore` implementation with one of the other ones documented -in this section. + diff --git a/docs/sidebars.js b/docs/sidebars.js index 7b170dc72ae..c7604f198b1 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -339,7 +339,7 @@ module.exports = { }, { type: "category", - label: "Stores", + label: "Key-value stores", collapsed: true, items: [ {