Commit Graph

6456 Commits

Author SHA1 Message Date
yangdihang
ff5024634e fix: openapi controller prompt, when bot is unable to resolve an api … (#7525)
…call, it needs retry

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
- Twitter handle: we announce bigger features on Twitter. If your PR
gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the
same people again.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->

Co-authored-by: yangdihang <yangdihang@bytedance.com>
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-07-31 17:56:43 -07:00
Kenny
1e8fca5518 Add ConcurrentLoader (#7512)
Works just like the GenericLoader but concurrently for those who choose
to optimize their workflow.

@rlancemartin @eyurtsev

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-07-31 17:56:31 -07:00
Kevin Buckley
8061994c61 AzureSearch Vector Store: Moving the usage of additional_fields into context of it's definition (bug fix from python error) (#8551)
Description: Using Azure Cognitive Search as a VectorStore. Calling the
`add_texts` method throws an error if there is no metadata property
specified. The `additional_fields` field is set in an `if` statement and
then is used later outside the if statement. This PR just moves the
declaration of `additional_fields` below and puts the usage of it in
context.

Issue: https://github.com/langchain-ai/langchain/issues/8544

Tagging @rlancemartin, @eyurtsev as this is related to Vector stores.

`make format`, `make lint`, `make spellcheck`, and `make test` have been
run
2023-07-31 17:25:57 -07:00
Pranay Chandekar
7e70cd2a28 Bug Fix - #8415 (#8417)
- Issue: #8415

Signed-off-by: Pranay Chandekar <pranayc6@gmail.com>
2023-07-31 17:08:46 -07:00
shibuiwilliam
de61ebd9e0 add tests to redis vectorstore (#8116)
# What
- Add function to get similarity with score with threshold in Redis
vector store.
- Add tests to Redis vector store.
2023-07-31 17:07:09 -07:00
Bruno Bornsztein
5a490a79f4 fix issue #8357 by making json backtick regex greedy (#8528)
- Description: Markdown code blocks in json response should not break
the parser
  - Issue: #8357

@baskaryan @hinthornw
2023-07-31 16:36:57 -07:00
Gordon Clark
64d0a0fcc0 Updating docstings in utilities (#8411)
Updating docstrings on utility packages
 @baskaryan
2023-07-31 16:34:53 -07:00
Mohammad Mohtashim
144b4c0c78 SQL Query Prompt update + added _execute method for SQLDatabase (#8100)
- Description: This pull request (PR) includes two minor changes:

1. Updated the default prompt for SQL Query Checker: The current prompt
does not clearly specify the final response that the LLM (Language
Model) should provide when checking for the query if `use_query_checker`
is enabled in SQLDatabase Chain. As a result, the LLM adds extra words
like "Here is your updated query" to the response. However, this causes
a syntax error when executing the SQL command in SQLDatabaseChain, as
these additional words are also included in the SQL query.

2. Moved the query's execution part into a separate method for
SQLDatabase: The purpose of this change is to provide users with more
flexibility when obtaining the result of an SQL query in the original
form returned by sqlalchemy. In the previous implementation, the run
method returned the results as a string. By creating a distinct method
for execution, users can now receive the results in original format,
which proves helpful in various scenarios. For example, during the
development of a tool, I found it advantageous to obtain results in
original format rather than a string, as currently done by the run
method.

- Tag maintainer: @hinthornw

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-07-31 16:28:08 -07:00
Matthew DeGuzman
844eca98d5 Add LLaMa Formatter and AzureML Chat Endpoint (#8382)
## Description

Microsoft and Meta recently [announced their
collaboration](https://blogs.microsoft.com/blog/2023/07/18/microsoft-and-meta-expand-their-ai-partnership-with-llama-2-on-azure-and-windows/)
on LLaMa2. This PR extends the current LLM wrapper and introduces a new
Chat Model wrapper for AzureML to support LLaMa2.

## Dependencies

No dependencies added :)

## Twitter Handles

[@matthew_d13](https://twitter.com/matthew_d13)
[@prakhar_in](https://twitter.com/prakhar_in)

maintainers - @hwchase17, @baskaryan
2023-07-31 16:26:25 -07:00
Harrison Chase
15de57b848 fix web loader (#8538) 2023-07-31 12:47:33 -07:00
Nuno Campos
4780156955 Rely less on positional arg order in subclasses of vector store when calling async methods (#8534) 2023-07-31 20:13:11 +01:00
Harrison Chase
5e3b968078 router runnable (#8496)
Co-authored-by: Nuno Campos <nuno@boringbits.io>
2023-07-31 11:07:10 -07:00
Anubhav Bindlish
913a156cff Minor improvements to rockset vectorstore (#8416)
This PR makes minor improvements to our python notebook, and adds
support for `Rockset` workspaces in our vectorstore client.

@rlancemartin, @eyurtsev

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-07-31 09:54:59 -07:00
Bagatur
a8be207ea3 bump 248 (#8518) 2023-07-31 07:14:45 -07:00
Harrison Chase
6556a8fcfd add initial anthropic agent (#8468)
Co-authored-by: Nuno Campos <nuno@boringbits.io>
2023-07-30 21:30:49 -07:00
os1ma
a795c3d860 Fix GitLoader to handle repeated load calls (#8412)
**Description: a description of the change**

In this pull request, GitLoader has been updated to handle multiple load
calls, provided the same repository is being cloned. Previously, calling
`load` multiple times would raise an error if a clone URL was provided.

Additionally, a check has been added to raise a ValueError when
attempting to clone a different repository into an existing path.

New tests have also been introduced to verify the correct behavior of
the GitLoader class when `load` is called multiple times.

Lastly, the GitPython package, a dependency for the GitLoader class, has
been added to the project dependencies (pyproject.toml and poetry.lock).

**Issue: the issue # it fixes (if applicable)**

None

**Dependencies: any dependencies required for this change**

GitPython

**Tag maintainer: for a quicker response, tag the relevant maintainer
(see below)**

- DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
2023-07-30 21:27:20 -07:00
Piyush Jain
b2f8a5bae9 Fixed exports for NeptuneOpenCypherQAChain (#8439)
## Description
The imports for `NeptuneOpenCypherQAChain` are failing. This PR adds the
chain class to the `__init__.py` file to fix this issue.

## Maintainers
@dev2049 
@krlawrence
2023-07-30 20:36:22 -07:00
Eugene Yurtsev
e98e2b2b81 ChatPromptTemplate: clean up doc-string (#8473)
Minor doc-string clean up

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-07-30 20:11:04 -07:00
Eugene Yurtsev
529cb2e30c Update doc-string in few shot template (#8474)
Partial update of doc-string, need to update other instances in
documentation
2023-07-30 19:39:14 -07:00
Muneeb Ahmad
4923cf029a Added Proper Documentation for faiss-gpu Installation (#8492)
### Description
In the LangChain Documentation and Comments, I've Noticed that `pip
install faiss` was mentioned, instead of `pip install faiss-gpu`, since
installing `pip install faiss` results in an error. I've gone ahead and
updated the Documentation, and `faiss.ipynb`. This Change will ensure
ease of use for the end user, trying to install `faiss-gpu`.

### Issue: 
Documentation / Comments Related.

### Dependencies:
No Dependencies we're changed only updated the files with the wrong
reference.

### Tag maintainer:
 @rlancemartin, @eyurtsev (Thank You for your contributions 😄 )
2023-07-30 13:24:30 -07:00
shibuiwilliam
549720ae51 add test to ensure values in time weighted retriever are updated (#8479)
# What
- add test to ensure values in time weighted retriever are updated

<!-- Thank you for contributing to LangChain!

Replace this comment with:
- Description: add test to ensure values in time weighted retriever are
updated
  - Issue: None
  - Dependencies: None
  - Tag maintainer: @rlancemartin, @eyurtsev
  - Twitter handle: @MlopsJ


Please make sure you're PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the
same people again.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
2023-07-30 11:42:25 -07:00
Harrison Chase
18a2452121 prompt cleanup (#8470) 2023-07-30 10:47:31 -07:00
Harrison Chase
4d526c49ed bump experimental to 008 (#8490) 2023-07-30 07:28:18 -07:00
Harrison Chase
8f14ddefdf add anthropic functions wrapper (#8475)
a cheeky wrapper around claude that adds in function calling support
(kind of, hence it going in experimental)
2023-07-30 07:23:46 -07:00
Nuno Campos
b65a9414bb runnable.bind().bind() should combine kwargs, instead of nesting wrappers (#8467)
<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
- Twitter handle: we announce bigger features on Twitter. If your PR
gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the
same people again.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
2023-07-29 15:48:30 -07:00
Nuno Campos
872abb4198 Implement Runnable for Tools (#8460)
- Make _arun optional
- Pass run_manager to inner chains in tools that have them

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
- Twitter handle: we announce bigger features on Twitter. If your PR
gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the
same people again.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
2023-07-29 10:01:18 -07:00
William FH
b7c0eb9ecb Wfh/ref links (#8454) 2023-07-29 08:44:32 -07:00
Harrison Chase
13b4f465e2 log output parser (#8446) 2023-07-29 07:53:45 +01:00
William FH
d935573362 Partial formatting for chat messages (#8450) 2023-07-28 23:08:33 -07:00
William FH
3314f54383 Update supabase docstrings (#8443) 2023-07-28 23:08:14 -07:00
Harrison Chase
2448043b84 bump and fix (#8441) 2023-07-28 17:16:51 -07:00
Amélie
8ee56b9a5b Feature: Add support for meilisearch vectorstore (#7649)
**Description:**

Add support for Meilisearch vector store.
Resolve #7603 

- No external dependencies added
- A notebook has been added

@rlancemartin

https://twitter.com/meilisearch

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-07-28 17:06:54 -07:00
Bearnardd
b7d6e1909c fix empty ids when metadatas is provided (#8127)
Fixes https://github.com/hwchase17/langchain/issues/7865 and
https://github.com/hwchase17/langchain/issues/8061

- [x] fixes returning empty ids when metadatas argument is provided

@baskaryan

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-07-28 16:17:31 -07:00
lvisdd
abe4c361f9 update get_num_tokens_from_messages model (#8431)
(#8430)

Co-authored-by: Kano Kunihiko <kkano@heroz.co.jp>
2023-07-28 15:07:03 -07:00
Jeffrey Wang
e0de62f6da Add RoPE Scaling params from llamacpp (#8422)
Description:
Just adding parameters from `llama-python-cpp` that support RoPE
scaling.
@hwchase17, @baskaryan

sources:
papers and explanation:
https://kaiokendev.github.io/context
llamacpp conversation:
https://github.com/ggerganov/llama.cpp/discussions/1965 
Supports models like:
https://huggingface.co/conceptofmind/LLongMA-2-13b
2023-07-28 14:42:41 -07:00
Harrison Chase
fab24457bc remove code (#8425) 2023-07-28 13:19:44 -07:00
Harrison Chase
3a78450883 update experimental (#8402)
some changes were made to experimental, porting them over
2023-07-28 13:01:36 -07:00
Harrison Chase
af7e70d4af expose function for converting messages to messages (#8426) 2023-07-28 13:00:54 -07:00
Eugene Yurtsev
06bdbe06fe PromptTemplate update documentation and expand kwarg (#8423)
# PromptTemplate

* Update documentation to highlight the classmethod for instantiating a
prompt template.
* Expand kwargs in the classmethod to make parameters easier to discover

This PR got reverted here:
https://github.com/langchain-ai/langchain/pull/8395/files
2023-07-28 14:11:49 -04:00
Eugene Yurtsev
e62a1686e2 ChatPromptTemplate: minor fix in doc string (#8424)
Minor fix in doc-string to use `ai` rather than `assistant`
2023-07-28 13:01:13 -04:00
Eugene Yurtsev
760c278fe0 ChatPromptTemplate: Expand support for message formats and documentation (#8244)
* Expands support for a variety of message formats in the
`from_messages` classmethod. Ideally, we could deprecate the other
on-ramps to reduce the amount of classmethods users need to know about.
* Expand documentation with code examples.
2023-07-28 12:48:08 -04:00
Bagatur
61dd92f821 bump 246 (#8410) 2023-07-28 01:18:37 -07:00
Harrison Chase
394b67ab92 add kwargs to llm runnables (#8388) 2023-07-28 09:13:11 +01:00
HeTaoPKU
d5884017a9 Add Minimax llm model to langchain (#7645)
- Description: Minimax is a great AI startup from China, recently they
released their latest model and chat API, and the API is widely-spread
in China. As a result, I'd like to add the Minimax llm model to
Langchain.
- Tag maintainer: @hwchase17, @baskaryan

---------

Co-authored-by: the <tao.he@hulu.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-07-27 22:53:23 -07:00
James Campbell
0ad2d5f27a [nit] Add default value for ChatOpenAI client (#7939)
Micro convenience PR to avoid warning regarding missing `client`
parameter. It is always set during initialization.

@baskaryan

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-07-27 22:38:32 -07:00
Harrison Chase
82df923f37 Merge branch 'master' of github.com:hwchase17/langchain 2023-07-27 22:01:20 -07:00
Harrison Chase
1b0bfa54cf cr 2023-07-27 22:00:52 -07:00
Jeff Vestal
c7ff5f19a8 ElasticKnnSearch rewrite - bug fix - return Document (#8180)
Fixes: 
https://github.com/hwchase17/langchain/issues/7117
https://github.com/hwchase17/langchain/issues/5760

Adding back `create_index` , `add_texts`, `from_texts` to
ElasticKnnSearch

`from_texts` matches standard `from_texts` methods as quick start up
method

`knn_search` and `hybrid_result` return a list of [`Document()`,
`score`,]

# Test `from_texts` for quick start
```
# create new index using from_text

from langchain.vectorstores.elastic_vector_search import ElasticKnnSearch
from langchain.embeddings import ElasticsearchEmbeddings

model_id = "sentence-transformers__all-distilroberta-v1" 
dims = 768
es_cloud_id = ""
es_user = ""
es_password = ""
test_index = "knn_test_index_305"

embeddings = ElasticsearchEmbeddings.from_credentials(
    model_id,
    #input_field=input_field,
    es_cloud_id=es_cloud_id,
    es_user=es_user,
    es_password=es_password,
)

# add texts and create class instance
texts = ["This is a test document", "This is another test document"]
knnvectorsearch = ElasticKnnSearch.from_texts(
    texts=texts,
    embedding=embeddings,
    index_name= test_index,
    vector_query_field='vector',
    query_field='text',
    model_id=model_id,
    dims=dims,
	es_cloud_id=es_cloud_id, 
	es_user=es_user, 
	es_password=es_password
)

# Test `add_texts` method
texts2 = ["Hello, world!", "Machine learning is fun.", "I love Python."]
knnvectorsearch.add_texts(texts2)

query = "Hello"
knn_result = knnvectorsearch.knn_search(query = query, model_id= model_id, k=2)

hybrid_result = knnvectorsearch.knn_hybrid_search(query = query, model_id= model_id, k=2)

```

The  mapping is as follows:
```
{
  "knn_test_index_012": {
    "mappings": {
      "properties": {
        "text": {
          "type": "text"
        },
        "vector": {
          "type": "dense_vector",
          "dims": 768,
          "index": true,
          "similarity": "dot_product"
        }
      }
    }
  }
}
```

# Check response type
```
>>> hybrid_result
[(Document(page_content='Hello, world!', metadata={}), 0.94232327), (Document(page_content='I love Python.', metadata={}), 0.5321523)]

>>> hybrid_result[0]
(Document(page_content='Hello, world!', metadata={}), 0.94232327)

>>> hybrid_result[0][0]
Document(page_content='Hello, world!', metadata={})

>>> type(hybrid_result[0][0])
<class 'langchain.schema.document.Document'>
```

# Test with existing Index
```
from langchain.vectorstores.elastic_vector_search import ElasticKnnSearch
from langchain.embeddings import ElasticsearchEmbeddings

## Initialize ElasticsearchEmbeddings
model_id = "sentence-transformers__all-distilroberta-v1" 
dims = 768
es_cloud_id = 
es_user = ""
es_password = ""
test_index = "knn_test_index_012"

embeddings = ElasticsearchEmbeddings.from_credentials(
    model_id,
    es_cloud_id=es_cloud_id,
    es_user=es_user,
    es_password=es_password,
)

## Initialize ElasticKnnSearch
knn_search = ElasticKnnSearch(
	es_cloud_id=es_cloud_id, 
	es_user=es_user, 
	es_password=es_password, 
	index_name= test_index, 
	embedding= embeddings
)


## Test adding vectors

### Test `add_texts` method when index created
texts = ["Hello, world!", "Machine learning is fun.", "I love Python."]
knn_search.add_texts(texts)

```

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-07-27 22:00:18 -07:00
Harrison Chase
a221a9ced0 Harrison/sql query (#8370)
Co-authored-by: Nuno Campos <nuno@boringbits.io>
2023-07-27 21:55:17 -07:00
Bagatur
a1a650c743 Bagatur/from texts bug fix (#8394)
---------

Co-authored-by: Davit Buniatyan <davit@loqsh.com>
Co-authored-by: Davit Buniatyan <d@activeloop.ai>
Co-authored-by: adilkhan <adilkhan.sarsen@nu.edu.kz>
Co-authored-by: Ivo Stranic <istranic@gmail.com>
2023-07-27 21:52:38 -07:00