Commit Graph

3860 Commits

Author SHA1 Message Date
Bagatur
b9ca5cc5ea update guide import (#9279) 2023-08-15 17:01:06 -07:00
Bagatur
afba2be3dc update openai functions docs (#9278) 2023-08-15 17:00:56 -07:00
Bagatur
9abf60acb6 Bagatur/vectara regression (#9276)
Co-authored-by: Ofer Mendelevitch <ofer@vectara.com>
Co-authored-by: Ofer Mendelevitch <ofermend@gmail.com>
2023-08-15 16:19:46 -07:00
Xiaoyu Xee
b30f449dae Add dashvector vectorstore (#9163)
## Description
Add `Dashvector` vectorstore for langchain

- [dashvector quick
start](https://help.aliyun.com/document_detail/2510223.html)
- [dashvector package description](https://pypi.org/project/dashvector/)

## How to use
```python
from langchain.vectorstores.dashvector import DashVector

dashvector = DashVector.from_documents(docs, embeddings)
```

---------

Co-authored-by: smallrain.xuxy <smallrain.xuxy@alibaba-inc.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-15 16:19:30 -07:00
Bagatur
bfbb97b74c Bagatur/deeplake docs fixes (#9275)
Co-authored-by: adilkhan <adilkhan.sarsen@nu.edu.kz>
2023-08-15 15:56:36 -07:00
Kunj-2206
1b3942ba74 Added BittensorLLM (#9250)
Description: Adding NIBittensorLLM via Validator Endpoint to langchain
llms
Tag maintainer: @Kunj-2206

Maintainer responsibilities:
    Models / Prompts: @hwchase17, @baskaryan

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-15 15:40:52 -07:00
Toshish Jawale
852722ea45 Improvements in Nebula LLM (#9226)
- Description: Added improvements in Nebula LLM to perform auto-retry;
more generation parameters supported. Conversation is no longer required
to be passed in the LLM object. Examples are updated.
  - Issue: N/A
  - Dependencies: N/A
  - Tag maintainer: @baskaryan 
  - Twitter handle: symbldotai

---------

Co-authored-by: toshishjawale <toshish@symbl.ai>
2023-08-15 15:33:07 -07:00
Bagatur
358562769a Bagatur/refac faiss (#9076)
Code cleanup and bug fix in deletion
2023-08-15 15:19:00 -07:00
Bagatur
3eccd72382 pin pydantic (#9274)
don't want default to be v2 yet
2023-08-15 15:02:28 -07:00
Erick Friis
76d09b4ed0 hub push/pull (#9225)
Description: Adds push/pull functions to interact with the hub
Issue: n/a
Dependencies: `langchainhub`

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-15 14:11:43 -07:00
Bagatur
1aae77f26f fix context nb (#9267) 2023-08-15 12:53:37 -07:00
Alex Gamble
cf17c58b47 Update documentation for the Context integration with new URL and features (#9259)
Update documentation and URLs for the Langchain Context integration.

We've moved from getcontext.ai to context.ai \o/

Thanks in advance for the review!
2023-08-15 11:38:34 -07:00
Eugene Yurtsev
a091b4bf4c Update testing workflow to test with both pydantic versions (#9206)
* PR updates test.yml to test with both pydantic versions
* Code should be refactored to make it easier to do testing in matrix
format w/ packages
* Added steps to assert that pydantic version in the environment is as
expected
2023-08-15 13:21:11 -04:00
Bagatur
e0162baa3b add oai sched tests (#9257) 2023-08-15 09:40:33 -07:00
Joseph McElroy
5e9687a196 Elasticsearch self-query retriever (#9248)
Now with ElasticsearchStore VectorStore merged, i've added support for
the self-query retriever.

I've added a notebook also to demonstrate capability. I've also added
unit tests.

**Credit**
@elastic and @phoey1 on twitter.
2023-08-15 10:53:43 -04:00
Anthony Mahanna
0a04e63811 docs: Update ArangoDB Links (#9251)
ready for review 

- mdx link update
- colab link update
2023-08-15 07:43:47 -07:00
Eugene Yurtsev
0470198fb5 Remove packages for pydantic compatibility (#9217)
# Poetry updates

This PR updates LangChains poetry file to remove
any dependencies that aren't pydantic v2 compatible yet.

All packages remain usable under pydantic v1, and can be installed
separately. 

## Bumping the following packages:

* langsmith

## Removing the following packages

not used in extended unit-tests:

* zep-python, anthropic, jina, spacy, steamship, betabageldb

not used at all:

* octoai-sdk

Cleaning up extras w/ for removed packages.

## Snapshots updated

Some snapshots had to be updated due to a change in the data model in
langsmith. RunType used to be Union of Enum and string and was changed
to be string only.
2023-08-15 10:41:25 -04:00
Bagatur
e986afa13a bump 265 (#9253) v0.0.265 2023-08-15 07:21:32 -07:00
Hech
4b505060bd fix: max_marginal_relevance_search and docs in Dingo (#9244) 2023-08-15 01:06:06 -07:00
axiangcoding
664ff28cba feat(llms): support ernie chat (#9114)
Description: support ernie (文心一言) chat model
Related issue: #7990
Dependencies: None
Tag maintainer: @baskaryan
2023-08-15 01:05:46 -07:00
Bharat Ramanathan
08a8363fc6 feat(integration): Add support to serialize protobufs in WandbTracer (#8914)
This PR adds serialization support for protocol bufferes in
`WandbTracer`. This allows code generation chains to be visualized.
Additionally, it also fixes a minor bug where the settings are not
honored when a run is initialized before using the `WandbTracer`

@agola11

---------

Co-authored-by: Bharat Ramanathan <ramanathan.parameshwaran@gohuddl.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-15 01:05:12 -07:00
fanyou-wbd
5e43768f61 docs: update LlamaCpp max_tokens args (#9238)
This PR updates documentations only, `max_length` should be `max_tokens`
according to latest LlamaCpp API doc:
https://api.python.langchain.com/en/latest/llms/langchain.llms.llamacpp.LlamaCpp.html
2023-08-15 00:50:20 -07:00
Bagatur
a8aa1aba1c nit (#9243) 2023-08-15 00:49:12 -07:00
Bagatur
68d8f73698 consolidate redirects (#9242) 2023-08-15 00:48:23 -07:00
Joshua Sundance Bailey
ef0664728e ArcGISLoader update (#9240)
Small bug fixes and added metadata based on user feedback. This PR is
from the author of https://github.com/langchain-ai/langchain/pull/8873 .
2023-08-14 23:44:29 -07:00
Joseph McElroy
eac4ddb4bb Elasticsearch Store Improvements (#8636)
Todo:
- [x] Connection options (cloud, localhost url, es_connection) support
- [x] Logging support
- [x] Customisable field support
- [x] Distance Similarity support 
- [x] Metadata support
  - [x] Metadata Filter support 
- [x] Retrieval Strategies
  - [x] Approx
  - [x] Approx with Hybrid
  - [x] Exact
  - [x] Custom 
  - [x] ELSER (excluding hybrid as we are working on RRF support)
- [x] integration tests 
- [x] Documentation

👋 this is a contribution to improve Elasticsearch integration with
Langchain. Its based loosely on the changes that are in master but with
some notable changes:

## Package name & design improvements
The import name is now `ElasticsearchStore`, to aid discoverability of
the VectorStore.

```py
## Before
from langchain.vectorstores.elastic_vector_search import ElasticVectorSearch, ElasticKnnSearch

## Now
from langchain.vectorstores.elasticsearch import ElasticsearchStore
```

## Retrieval Strategy support
Before we had a number of classes, depending on the strategy you wanted.
`ElasticKnnSearch` for approx, `ElasticVectorSearch` for exact / brute
force.

With `ElasticsearchStore` we have retrieval strategies:

### Approx Example
Default strategy for the vast majority of developers who use
Elasticsearch will be inferring the embeddings from outside of
Elasticsearch. Uses KNN functionality of _search.

```py
        texts = ["foo", "bar", "baz"]
       docsearch = ElasticsearchStore.from_texts(
            texts,
            FakeEmbeddings(),
            es_url="http://localhost:9200",
            index_name="sample-index"
        )
        output = docsearch.similarity_search("foo", k=1)
```

### Approx, with hybrid
Developers who want to search, using both the embedding and the text
bm25 match. Its simple to enable.

```py
 texts = ["foo", "bar", "baz"]
       docsearch = ElasticsearchStore.from_texts(
            texts,
            FakeEmbeddings(),
            es_url="http://localhost:9200",
            index_name="sample-index",
            strategy=ElasticsearchStore.ApproxRetrievalStrategy(hybrid=True)
        )
        output = docsearch.similarity_search("foo", k=1)
```

### Approx, with `query_model_id`
Developers who want to infer within Elasticsearch, using the model
loaded in the ml node.

This relies on the developer to setup the pipeline and index if they
wish to embed the text in Elasticsearch. Example of this in the test.

```py
 texts = ["foo", "bar", "baz"]
       docsearch = ElasticsearchStore.from_texts(
            texts,
            FakeEmbeddings(),
            es_url="http://localhost:9200",
            index_name="sample-index",
            strategy=ElasticsearchStore.ApproxRetrievalStrategy(
                query_model_id="sentence-transformers__all-minilm-l6-v2"
            ),
        )
        output = docsearch.similarity_search("foo", k=1)
```

### I want to provide my own custom Elasticsearch Query
You might want to have more control over the query, to perform
multi-phase retrieval such as LTR, linearly boosting on document
parameters like recently updated or geo-distance. You can do this with
`custom_query_fn`

```py
        def my_custom_query(query_body: dict, query: str) -> dict:
            return {"query": {"match": {"text": {"query": "bar"}}}}

        texts = ["foo", "bar", "baz"]
        docsearch = ElasticsearchStore.from_texts(
            texts, FakeEmbeddings(), **elasticsearch_connection, index_name=index_name
        )
        docsearch.similarity_search("foo", k=1, custom_query=my_custom_query)

```

### Exact Example
Developers who have a small dataset in Elasticsearch, dont want the cost
of indexing the dims vs tradeoff on cost at query time. Uses
script_score.

```py
        texts = ["foo", "bar", "baz"]
       docsearch = ElasticsearchStore.from_texts(
            texts,
            FakeEmbeddings(),
            es_url="http://localhost:9200",
            index_name="sample-index",
            strategy=ElasticsearchStore.ExactRetrievalStrategy(),
        )
        output = docsearch.similarity_search("foo", k=1)
```

### ELSER Example
Elastic provides its own sparse vector model called ELSER. With these
changes, its really easy to use. The vector store creates a pipeline and
index thats setup for ELSER. All the developer needs to do is configure,
ingest and query via langchain tooling.

```py
texts = ["foo", "bar", "baz"]
       docsearch = ElasticsearchStore.from_texts(
            texts,
            FakeEmbeddings(),
            es_url="http://localhost:9200",
            index_name="sample-index",
            strategy=ElasticsearchStore.SparseVectorStrategy(),
        )
        output = docsearch.similarity_search("foo", k=1)

```

## Architecture
In future, we can introduce new strategies and allow us to not break bwc
as we evolve the index / query strategy.

## Credit
On release, could you credit @elastic and @phoey1 please? Thank you!

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-14 23:42:35 -07:00
Harrison Chase
71d5b7c9bf Harrison/fallbacks (#9233)
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-14 18:27:38 -07:00
Lance Martin
41279a3ae1 Move self-check use case to "more" section (#9137)
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-14 18:27:28 -07:00
Lance Martin
22858d99b5 Move code-writing use case to "more" section (#9134)
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-14 18:27:19 -07:00
Bagatur
249d7d06a2 adapter doc nit (#9234) 2023-08-14 18:26:37 -07:00
Divyansh Garg
9529483c2a Improve MultiOn client toolkit prompts (#9222)
- Updated prompts for the MultiOn toolkit for better functionality
- Non-blocking but good to have it merged to improve the overall
performance for the toolkit
 
@hinthornw @hwchase17

---------

Co-authored-by: Naman Garg <ngarg3@binghamton.edu>
2023-08-14 17:39:51 -07:00
Lance Martin
969e1683de Move graph use case to "more" section (#8997)
Clean `use_cases` by moving the `GraphDB` to `integrations`.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-14 17:20:38 -07:00
William FH
c478fc208e Default On Retry (#9230)
Base callbacks don't have a default on retry event

Fix #8542

---------

Co-authored-by: landonsilla <landon.silla@stepstone.com>
2023-08-14 16:45:17 -07:00
Lance Martin
d0a0d560ad Minor formatting on Web Research Use Case (#9221) 2023-08-14 16:29:36 -07:00
Leonid Ganeline
93dd499997 docstrings: document_loaders consistency 3 (#9216)
Updated docstrings into the consistent format (probably, the last update
for the `document_loaders`.
2023-08-14 16:28:39 -07:00
Kshitij Wadhwa
a69cb95850 track langchain usage for Rockset (#9229)
Add ability to track langchain usage for Rockset. Rockset's new python
client allows setting this. To prevent old clients from failing, it
ignore if setting throws exception (we can't track old versions)

Tested locally with old and new Rockset python client

cc @baskaryan
2023-08-14 16:27:34 -07:00
Leonid Ganeline
7810ea5812 docstrings: chat_models consistency (#9227)
Updated docstrings into the consistent format.
2023-08-14 16:15:56 -07:00
William FH
b0896210c7 Return feedback with failed response if there's an error (#9223)
In Evals
2023-08-14 15:59:16 -07:00
William FH
7124f2ebfa Parent Doc Retriever (#9214)
2 things:
- Implement the private method rather than the public one so callbacks
are handled properly
- Add search_kwargs (Open to not adding this if we are trying to
deprecate this UX but seems like as a user i'd assume similar args to
the vector store retriever. In fact some may assume this implements the
same interface but I'm not dealing with that here)
-
2023-08-14 15:41:53 -07:00
Lance Martin
17ae2998e7 Update Ollama docs (#9220)
Based on discussion w/ team.
2023-08-14 13:56:16 -07:00
Harrison Chase
3f601b5809 add async method in (#9204) 2023-08-14 11:04:31 -07:00
Clark
03ea0762a1 fix(jinachat): related to #9197 (#9200)
related to: https://github.com/langchain-ai/langchain/issues/9197

---------

Co-authored-by: qianjun.wqj <qianjun.wqj@alibaba-inc.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-14 11:04:20 -07:00
Eugene Yurtsev
4f1feaca83 Wrap OpenAPI features in conditionals for pydantic v2 compatibility (#9205)
Wrap OpenAPI in conditionals for pydantic v2 compatibility.
2023-08-14 13:40:58 -04:00
Glauco Custódio
89be10f6b4 add ttl to RedisCache (#9068)
Add `ttl` (time to live) to `RedisCache`
2023-08-14 12:59:18 -04:00
Eugene Yurtsev
04bc5f3b18 Conditionally add pydantic v1 to namespace (#9202)
Conditionally add pydantic_v1 to namespace.
2023-08-14 11:26:45 -04:00
shibuiwilliam
feec422bf7 fix logging to logger (#9192)
# What
- fix logging to logger
2023-08-14 08:21:09 -07:00
Bagatur
5935767056 bump lc 246, lce 9 (#9207) v0.0.264 2023-08-14 08:14:37 -07:00
Bagatur
b5a57acf6c lite llm lint (#9208) 2023-08-14 11:03:06 -04:00
Krish Dholakia
49f1d8477c Adding ChatLiteLLM model (#9020)
Description: Adding a langchain integration for the LiteLLM library 
Tag maintainer: @hwchase17, @baskaryan
Twitter handle: @krrish_dh / @Berri_AI

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-14 07:43:40 -07:00
Emmanuel Gautier
f11e5442d6 docs: update LlamaCpp input args (#9173)
This PR only updates the LlamaCpp args documentation. The input arg has
been flattened.
2023-08-14 07:42:03 -07:00