langchain/docs/modules/indexes/vectorstores/examples
Hao Chen a4c9053d40
Integrate Clickhouse as Vector Store (#5650)
<!--
Thank you for contributing to LangChain! Your PR will appear in our
release under the title you set. Please make sure it highlights your
valuable contribution.

Replace this with a description of the change, the issue it fixes (if
applicable), and relevant context. List any dependencies required for
this change.

After you're done, someone will review your PR. They may suggest
improvements. If no one reviews your PR within a few days, feel free to
@-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd
like us to shout you out on Twitter, please also include your handle!
-->

#### Description

This PR is mainly to integrate open source version of ClickHouse as
Vector Store as it is easy for both local development and adoption of
LangChain for enterprises who already have large scale clickhouse
deployment.

ClickHouse is a open source real-time OLAP database with full SQL
support and a wide range of functions to assist users in writing
analytical queries. Some of these functions and data structures perform
distance operations between vectors, [enabling ClickHouse to be used as
a vector
database](https://clickhouse.com/blog/vector-search-clickhouse-p1).
Recently added ClickHouse capabilities like [Approximate Nearest
Neighbour (ANN)
indices](https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/annindexes)
support faster approximate matching of vectors and provide a promising
development aimed to further enhance the vector matching capabilities of
ClickHouse.

In LangChain, some ClickHouse based commercial variant vector stores
like
[Chroma](https://github.com/hwchase17/langchain/blob/master/langchain/vectorstores/chroma.py)
and
[MyScale](https://github.com/hwchase17/langchain/blob/master/langchain/vectorstores/myscale.py),
etc are already integrated, but for some enterprises with large scale
Clickhouse clusters deployment, it will be more straightforward to
upgrade existing clickhouse infra instead of moving to another similar
vector store solution, so we believe it's a valid requirement to
integrate open source version of ClickHouse as vector store.

As `clickhouse-connect` is already included by other integrations, this
PR won't include any new dependencies.

#### Before submitting

<!-- If you're adding a new integration, please include:

1. Added a test for the integration:
https://github.com/haoch/langchain/blob/clickhouse/tests/integration_tests/vectorstores/test_clickhouse.py
2. Added an example notebook and document showing its use: 
* Notebook:
https://github.com/haoch/langchain/blob/clickhouse/docs/modules/indexes/vectorstores/examples/clickhouse.ipynb
* Doc:
https://github.com/haoch/langchain/blob/clickhouse/docs/integrations/clickhouse.md

See contribution guidelines for more information on how to write tests,
lint
etc:


https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

1. Added a test for the integration:
https://github.com/haoch/langchain/blob/clickhouse/tests/integration_tests/vectorstores/test_clickhouse.py
2. Added an example notebook and document showing its use: 
* Notebook:
https://github.com/haoch/langchain/blob/clickhouse/docs/modules/indexes/vectorstores/examples/clickhouse.ipynb
* Doc:
https://github.com/haoch/langchain/blob/clickhouse/docs/integrations/clickhouse.md


#### Who can review?

Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
 
@hwchase17 @dev2049 Could you please help review?

---------

Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
2023-06-05 13:32:04 -07:00
..
analyticdb.ipynb docs: improved vectorstore notebooks (#3724) 2023-04-28 19:26:50 -07:00
annoy.ipynb docs: vectorstores, different updates and fixes (#4939) 2023-05-18 15:35:47 -07:00
atlas.ipynb docs: vectorstores, different updates and fixes (#4939) 2023-05-18 15:35:47 -07:00
chroma.ipynb Fix update_document function, add test and documentation. (#5359) 2023-05-29 06:39:25 -07:00
clickhouse.ipynb Integrate Clickhouse as Vector Store (#5650) 2023-06-05 13:32:04 -07:00
deeplake.ipynb docs: vectorstores, different updates and fixes (#4939) 2023-05-18 15:35:47 -07:00
docarray_hnsw.ipynb docs: vectorstores, different updates and fixes (#4939) 2023-05-18 15:35:47 -07:00
docarray_in_memory.ipynb docs: vectorstores, different updates and fixes (#4939) 2023-05-18 15:35:47 -07:00
elasticsearch.ipynb Es knn index search 5346 (#5569) 2023-06-02 08:40:35 -07:00
faiss.ipynb Faiss no avx2 (#4895) 2023-05-17 19:18:57 -07:00
lancedb.ipynb Rename and fix typo in lancedb (#5425) 2023-05-30 00:24:17 -07:00
matchingengine.ipynb Add matching engine vectorstore (#3350) 2023-05-31 02:28:02 -07:00
milvus.ipynb Update the milvus example (#4706) 2023-05-15 16:16:57 -07:00
mongodb_atlas_vector_search.ipynb removing client+namespace in favor of collection (#5610) 2023-06-03 16:27:31 -07:00
myscale.ipynb docs: improved vectorstore notebooks (#3724) 2023-04-28 19:26:50 -07:00
opensearch.ipynb OpenSearch top k parameter fix (#5216) 2023-05-25 09:51:23 -07:00
pgvector.ipynb Created fix for 5475 (#5659) 2023-06-03 16:47:52 -07:00
pinecone.ipynb docs retriever improvements (#4430) 2023-05-17 15:29:22 -07:00
qdrant.ipynb Qdrant self query (#5567) 2023-06-01 08:40:31 -07:00
redis.ipynb docs: vectorstores, different updates and fixes (#4939) 2023-05-18 15:35:47 -07:00
sklearn.ipynb Add SKLearnVectorStore (#5305) 2023-05-28 08:17:42 -07:00
supabase.ipynb docs supabase update (#4935) 2023-05-18 10:42:08 -07:00
tair.ipynb docs: vectorstores, different updates and fixes (#4939) 2023-05-18 15:35:47 -07:00
typesense.ipynb Add Typesense vector store (#1674) 2023-05-23 23:20:45 -07:00
vectara.ipynb Vectara (#5069) 2023-05-24 01:24:58 -07:00
weaviate.ipynb Weaviate: Add QnA with sources example (#5247) 2023-05-25 09:58:33 -04:00
zilliz.ipynb Update the milvus example (#4706) 2023-05-15 16:16:57 -07:00