Commit Graph

29 Commits

Author SHA1 Message Date
Bagatur
ca9dcee940
standard-tests[patch]: test ToolMessage.status="error" (#25210) 2024-08-09 13:00:14 -07:00
Gram Liu
88a9a6a758
core[patch]: Add pydantic metadata to subset model (#25032)
- **Description:** This includes Pydantic field metadata in
`_create_subset_model_v2` so that it gets included in the final
serialized form that get sent out.
- **Issue:** #25031 
- **Dependencies:** n/a
- **Twitter handle:** @gramliu

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>
2024-08-05 17:57:39 -07:00
Bagatur
e572521f2a
core[patch]: exclude special pydantic init params (#25084) 2024-08-05 23:32:51 +00:00
ccurme
c04d95b962
standard-tests: set integration test parameters independent of unit test (#24979)
This ends up getting set in integration tests.
2024-08-02 10:40:11 -07:00
Bagatur
a6d1fb4275
core[patch]: introduce ToolMessage.status (#24628)
Anthropic models (including via Bedrock and other cloud platforms)
accept a status/is_error attribute on tool messages/results
(specifically in `tool_result` content blocks for Anthropic API). Adding
a ToolMessage.status attribute so that users can set this attribute when
using those models
2024-07-29 14:01:53 -07:00
ccurme
e264ccf484
standard-tests[patch]: update groq and structured output test (#24781)
- Mixtral with Groq has started consistently failing tool calling tests.
Here we restrict testing to llama 3.1.
- `.schema` is deprecated in pydantic proper in favor of
`.model_json_schema`.
2024-07-29 11:10:01 -04:00
Eugene Yurtsev
f47b4edcc2
standard-test: Fix typo in skipif for chat model integration tests (#24553) 2024-07-23 10:11:01 -04:00
Eugene Yurtsev
20b72a044c
standard-tests: Add BaseModel variations tests to with_structured_output (#24527)
After this standard tests will test with the following combinations:

1. pydantic.BaseModel
2. pydantic.v1.BaseModel

If ran within a matrix, it'll covert both pydantic.BaseModel originating
from
pydantic 1 and the one defined in pydantic 2.
2024-07-23 09:01:26 -04:00
ccurme
6fc7610b1c
standard-tests[patch]: update test_bind_runnables_as_tools (#24241)
Reduce number of tool arguments from two to one.
2024-07-15 08:35:07 -04:00
Bagatur
bdc03997c9
standard-tests[patch]: check for ToolCall["type"] (#24209) 2024-07-12 16:17:34 -07:00
Bagatur
12950cc602
standard-tests[patch]: improve runnable tool description (#24210) 2024-07-12 21:33:56 +00:00
ccurme
cb95198398
standard-tests[patch]: add tests for runnables as tools and streaming usage metadata (#24153) 2024-07-11 18:30:05 -04:00
ccurme
74c7198906
core, anthropic[patch]: support streaming tool calls when function has no arguments (#23915)
resolves https://github.com/langchain-ai/langchain/issues/23911

When an AIMessageChunk is instantiated, we attempt to parse tool calls
off of the tool_call_chunks.

Here we add a special-case to this parsing, where `""` will be parsed as
`{}`.

This is a reaction to how Anthropic streams tool calls in the case where
a function has no arguments:
```
{'id': 'toolu_01J8CgKcuUVrMqfTQWPYh64r', 'input': {}, 'name': 'magic_function', 'type': 'tool_use', 'index': 1}
{'partial_json': '', 'type': 'tool_use', 'index': 1}
```
The `partial_json` does not accumulate to a valid json string-- most
other providers tend to emit `"{}"` in this case.
2024-07-05 18:57:41 +00:00
ccurme
46cbf0e4aa
anthropic[patch]: use core output parsers for structured output (#23776)
Also add to standard tests for structured output.
2024-07-02 16:15:26 -04:00
Bagatur
389a568f9a
standard-tests[patch]: add anthropic format integration test (#23717) 2024-07-01 11:06:04 -04:00
ccurme
390ee8d971
standard-tests: add test for structured output (#23631)
- add test for structured output
- fix bug with structured output for Azure
- better testing on Groq (break out Mixtral + Llama3 and add xfails
where needed)
2024-06-28 15:01:40 -04:00
ccurme
a7b4175091
standard tests: add test for tool calling (#23234)
Including streaming
2024-06-20 17:20:11 -04:00
Bagatur
90559fde70
openai[patch], standard-tests[patch]: don't pass in falsey stop vals (#23153)
adds an image input test to standard-tests as well
2024-06-18 18:13:13 -07:00
Bagatur
d96f67b06f
standard-tests[patch]: Update chat model standard tests (#22378)
- Refactor standard test classes to make them easier to configure
- Update openai to support stop_sequences init param
- Update groq to support stop_sequences init param
- Update fireworks to support max_retries init param
- Update ChatModel.bind_tools to type tool_choice
- Update groq to handle tool_choice="any". **this may be controversial**

---------

Co-authored-by: Chester Curme <chester.curme@gmail.com>
2024-06-17 13:37:41 -07:00
Bagatur
9ff249a38d
standard-tests[patch]: don't require str chunk contents (#22965) 2024-06-17 08:52:24 -04:00
ccurme
3999761201
multiple: add stop attribute (#22573) 2024-06-06 12:11:52 -04:00
ccurme
fbfed65fb1
core, partners: add token usage attribute to AIMessage (#21944)
```python
class UsageMetadata(TypedDict):
    """Usage metadata for a message, such as token counts.

    Attributes:
        input_tokens: (int) count of input (or prompt) tokens
        output_tokens: (int) count of output (or completion) tokens
        total_tokens: (int) total token count
    """

    input_tokens: int
    output_tokens: int
    total_tokens: int
```
```python
class AIMessage(BaseMessage):
    ...
    usage_metadata: Optional[UsageMetadata] = None
    """If provided, token usage information associated with the message."""
    ...
```
2024-05-23 14:21:58 -04:00
ccurme
01a3228d8e
standard tests: add test for few-shot examples (#21019) 2024-05-13 10:06:12 -04:00
ccurme
2d77e5e3a1
(standard tests): add test for basic conversation sequence (#21213) 2024-05-02 13:47:10 -04:00
Erick Friis
ddc2274aea
standard-tests: split tool calling test (#20803)
just making it a bit easier to grok
2024-04-23 20:59:45 +00:00
ccurme
2238490069
mistral, openai: allow anthropic-style messages in message histories (#20565) 2024-04-17 15:55:45 -04:00
ccurme
a892f985d3
standardized-tests[patch]: test tool call messages (#20519)
Co-authored-by: Bagatur <baskaryan@gmail.com>
2024-04-16 23:25:50 +00:00
Erick Friis
77eba10f47
standard-tests: fix default fixtures (#20520) 2024-04-16 16:12:36 +00:00
Erick Friis
855ba46f80
standard-tests: a standard unit and integration test set (#20182)
just chat models for now
2024-04-09 12:43:00 -07:00