Commit Graph

4071 Commits

Author SHA1 Message Date
Lance Martin
95b0ba2956 Fmt notebook 2023-08-28 09:50:02 -07:00
eryk-dsai
006b5446c8 update llama-cpp list example 2023-08-25 13:58:11 +02:00
eryk-dsai
6550ae48cd grammar for python list supports multi word strings now 2023-08-25 13:44:48 +02:00
eryk-dsai
c1091a0bed correct grammar for python list of single word strings 2023-08-25 13:37:20 +02:00
eryk-dsai
f893ea3a34 fixing small typos 2023-08-25 13:22:16 +02:00
Lance Martin
3ca8feb9b1 More testing 2023-08-24 21:29:57 -07:00
Bagatur
b4e77e5a1d cr 2023-08-24 19:22:36 -07:00
Lance Martin
75d4722760 Further testing 2023-08-24 16:25:26 -07:00
Lance Martin
7fa40c369d Enable grammars w LLaMA 2023-08-24 15:58:26 -07:00
Lance Martin
74de068f1f Add grammar 2023-08-24 12:38:46 -07:00
eryk-dsai
65c6ae7a92 Merge branch 'langchain-ai:master' into llama-cpp-grammar 2023-08-24 18:31:15 +02:00
eryk-dsai
108098c61a removing redundant error handling 2023-08-24 18:29:30 +02:00
eryk-dsai
df7ba746d7 more verbose description of grammar_path, catching ImportError 2023-08-24 18:09:44 +02:00
Leonid Ganeline
cf792891f1 📖 docs: compact api reference (#8651)
Updated design of the "API Reference" text
Here is an example of the current format:

![image](https://github.com/langchain-ai/langchain/assets/2256422/8727f2ba-1b69-497f-aa07-07f939b6da3b)

It changed to
`langchain.retrievers.ElasticSearchBM25Retriever` format. The same
format as it is in the API Reference Toc.

It also resembles code: 
`from langchain.retrievers import ElasticSearchBM25Retriever` (namespace
THEN class_name)

Current format is
`ElasticSearchBM25Retriever from langchain.retrievers` (class_name THEN
namespace)

This change is in line with other formats and improves readability.

 @baskaryan
2023-08-24 09:01:52 -07:00
Bagatur
f5ea725796 bump 272 (#9704) v0.0.272 2023-08-24 07:46:15 -07:00
Patrick Loeber
6bedfdf25a Fix docs for AssemblyAIAudioTranscriptLoader (shorter import path) (#9687)
Uses the shorter import path

`from langchain.document_loaders import` instead of the full path
`from langchain.document_loaders.assemblyai`

Applies those changes to the docs and the unit test.

See #9667 that adds this new loader.
2023-08-24 07:24:53 -07:00
了空
7cf5c582d2 Added a link to the dependencies document (#9703) 2023-08-24 07:23:48 -07:00
Nuno Campos
9666e752b1 Do not share executors between parent and child tasks (#9701)
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
- Twitter handle: we announce bigger features on Twitter. If your PR
gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. These live is docs/extras
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
2023-08-24 16:17:07 +02:00
Nuno Campos
78ffcdd9a9 Lint 2023-08-24 16:09:38 +02:00
Nuno Campos
20d2c0571c Do not share executors between parent and child tasks 2023-08-24 16:05:10 +02:00
eryk-dsai
0c96b00467 grammar based sampling in llama-cpp 2023-08-24 15:56:58 +02:00
Harrison Chase
9963b32e59 Harrison/multi vector (#9700) 2023-08-24 06:42:42 -07:00
Leonid Ganeline
b048236c1a 📖 docs: integrations/agent_toolkits (#9333)
Note: There are no changes in the file names!

- The group name on the main navbar changed: `Agent toolkits` -> `Agents
& Toolkits`. Examples here are the mix of the Agent and Toolkit examples
because Agents and Toolkits in examples are always used together.
- Titles changed: removed "Agent" and "Toolkit" suffixes. The reason is
the same.
- Formatting: mostly cleaning the header structure, so it could be
better on the right-side navbar.

Main navbar is looking much cleaner now.
2023-08-23 23:17:47 -07:00
Leonid Ganeline
c19888c12c docstrings: vectorstores consistency (#9349)
 
- updated the top-level descriptions to a consistent format;
- changed several `ValueError` to `ImportError` in the import cases;
- changed the format of several internal functions from "name" to
"_name". So, these functions are not shown in the Top-level API
Reference page (with lists of classes/functions)
2023-08-23 23:17:05 -07:00
Kim Minjong
d0ff0db698 Update ChatOpenAI._stream to respect finish_reason (#9672)
Currently, ChatOpenAI._stream does not reflect finish_reason to
generation_info. Change it to reflect that.

Same patch as https://github.com/langchain-ai/langchain/pull/9431 , but
also applies to _stream.
2023-08-23 22:58:14 -07:00
Patrick Loeber
5990651070 Add new document_loader: AssemblyAIAudioTranscriptLoader (#9667)
This PR adds a new document loader `AssemblyAIAudioTranscriptLoader`
that allows to transcribe audio files with the [AssemblyAI
API](https://www.assemblyai.com) and loads the transcribed text into
documents.

- Add new document_loader with class `AssemblyAIAudioTranscriptLoader`
- Add optional dependency `assemblyai`
- Add unit tests (using a Mock client)
- Add docs notebook

This is the equivalent to the JS integration already available in
LangChain.js. See the [LangChain JS docs AssemblyAI
page](https://js.langchain.com/docs/modules/data_connection/document_loaders/integrations/web_loaders/assemblyai_audio_transcription).

At its simplest, you can use the loader to get a transcript back from an
audio file like this:

```python
from langchain.document_loaders.assemblyai import AssemblyAIAudioTranscriptLoader

loader =  AssemblyAIAudioTranscriptLoader(file_path="./testfile.mp3")
docs = loader.load()
```

To use it, it needs the `assemblyai` python package installed, and the
environment variable `ASSEMBLYAI_API_KEY` set with your API key.
Alternatively, the API key can also be passed as an argument.

Twitter handles to shout out if so kindly 🙇
[@AssemblyAI](https://twitter.com/AssemblyAI) and
[@patloeber](https://twitter.com/patloeber)

---------

Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2023-08-23 22:51:19 -07:00
seamusp
25f2c82ae8 docs:misc fixes (#9671)
Improve internal consistency in LangChain documentation
- Change occurrences of eg and eg. to e.g.
- Fix headers containing unnecessary capital letters.
- Change instances of "few shot" to "few-shot".
- Add periods to end of sentences where missing.
- Minor spelling and grammar fixes.
2023-08-23 22:36:54 -07:00
Nuno Campos
6283f3b63c Resolve circular imports in runnables (#9675)
These are about to cause circular imports.
2023-08-24 06:05:51 +01:00
Eugene Yurtsev
9e1dbd4b49 x 2023-08-23 22:51:49 -04:00
Eugene Yurtsev
b88dfcb42a Add indexing support (#9614)
This PR introduces a persistence layer to help with indexing workflows
into
vectostores.

The indexing code helps users to:

1. Avoid writing duplicated content into the vectostore
2. Avoid over-writing content if it's unchanged

Importantly, this keeps on working even if the content being written is
derived
via a set of transformations from some source content (e.g., indexing
children
documents that were derived from parent documents by chunking.)

The two main components are:

1. Persistence layer that keeps track of which keys were updated and
when.
Keeping track of the timestamp of updates, allows to clean up old
content
   safely, and with minimal complexity.
2. HashedDocument which is used to hash the contents (including
metadata) of
   the documents. We rely on the hashes for identifying duplicates.


The indexing code works with **ANY** document loader. To add
transformations
to the documents, users for now can add a custom document loader
that composes an existing loader together with document transformers.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-23 21:41:38 -04:00
刘 方瑞
c215481531 Update default index type and metric type for MyScale vector store (#9353)
We update the default index type from `IVFFLAT` to `MSTG`, a new vector
type developed by MyScale.
2023-08-23 18:26:29 -07:00
Joshua Sundance Bailey
a9c86774da Anthropic: Allow the use of kwargs consistent with ChatOpenAI. (#9515)
- Description: ~~Creates a new root_validator in `_AnthropicCommon` that
allows the use of `model_name` and `max_tokens` keyword arguments.~~
Adds pydantic field aliases to support `model_name` and `max_tokens` as
keyword arguments. Ultimately, this makes `ChatAnthropic` more
consistent with `ChatOpenAI`, making the two classes more
interchangeable for the developer.
  - Issue: https://github.com/langchain-ai/langchain/issues/9510

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-23 18:23:21 -07:00
Lakshay Kansal
a8c916955f Updates to Nomic Atlas and GPT4All documentation (#9414)
Description: Updates for Nomic AI Atlas and GPT4All integrations
documentation.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-23 17:49:44 -07:00
Bagatur
342087bdfa fix integration test imports (#9669) 2023-08-23 16:47:01 -07:00
Keras Conv3d
cbaea8d63b tair fix distance_type error, and add hybrid search (#9531)
- fix: distance_type error, 
- feature: Tair add hybrid search

---------

Co-authored-by: thw <hanwen.thw@alibaba-inc.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-23 16:38:31 -07:00
Eugene Yurtsev
cd81e8a8f2 Add exclude to GenericLoader.from_file_system (#9539)
support exclude param in GenericLoader.from_filesystem

---------

Co-authored-by: Kyle Pancamo <50267605+KylePancamo@users.noreply.github.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-23 16:09:10 -07:00
Jacob Lee
278ef0bdcf Adds ChatOllama (#9628)
@rlancemartin

---------

Co-authored-by: Adilkhan Sarsen <54854336+adolkhan@users.noreply.github.com>
Co-authored-by: Kim Minjong <make.dirty.code@gmail.com>
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
Co-authored-by: Lance Martin <lance@langchain.dev>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-23 13:02:26 -07:00
Nuno Campos
fa05e18278 Nc/runnable lambda recurse (#9390)
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
- Twitter handle: we announce bigger features on Twitter. If your PR
gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. These live is docs/extras
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
2023-08-23 20:07:08 +01:00
Nuno Campos
20ce283fa7 Format 2023-08-23 20:03:35 +01:00
Nuno Campos
6424b3cde0 Add another test 2023-08-23 20:02:35 +01:00
William FH
da18e177f1 Update libs/langchain/langchain/schema/runnable/base.py
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2023-08-23 20:00:16 +01:00
Nuno Campos
c326751085 Lint 2023-08-23 20:00:16 +01:00
Nuno Campos
6d19709b65 RunnableLambda, if func returns a Runnable, run it 2023-08-23 20:00:16 +01:00
Nuno Campos
677da6a0fd Add support for async funcs in RunnableSequence 2023-08-23 19:54:48 +01:00
Nuno Campos
64a958c85d Runnables: Add .map() method (#9445)
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
- Twitter handle: we announce bigger features on Twitter. If your PR
gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

See contribution guidelines for more information on how to write/run
tests, lint, etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. These live is docs/extras
directory.

If no one reviews your PR within a few days, please @-mention one of
@baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
2023-08-23 19:54:12 +01:00
Nuno Campos
1751fe114d Add one more test 2023-08-23 19:52:13 +01:00
Nuno Campos
882b97cfd2 Lint 2023-08-23 19:50:20 +01:00
Nuno Campos
3ddabe8b2c Code review 2023-08-23 19:48:33 +01:00
Nuno Campos
fdcd50aab4 Extend test 2023-08-23 19:48:33 +01:00
Nuno Campos
9777c2801d Update method and docstring 2023-08-23 19:48:33 +01:00