nvidia-ai-endpoints[patch]: model arguments (e.g. temperature) on construction bug (#17290)

- **Issue:** Issue with model argument support (been there for a while
actually):
- Non-specially-handled arguments like temperature don't work when
passed through constructor.
- Such arguments DO work quite well with `bind`, but also do not abide
by field requirements.
- Since initial push, server-side error messages have gotten better and
v0.0.2 raises better exceptions. So maybe it's better to let server-side
handle such issues?
- **Description:**
- Removed ChatNVIDIA's argument fields in favor of
`model_kwargs`/`model_kws` arguments which aggregates constructor kwargs
(from constructor pathway) and merges them with call kwargs (bind
pathway).
- Shuffled a few functions from `_NVIDIAClient` to `ChatNVIDIA` to
streamline construction for future integrations.
- Minor/Optional: Old services didn't have stop support, so client-side
stopping was implemented. Now do both.
- **Any Breaking Changes:** Minor breaking changes if you strongly rely
on chat_model.temperature, etc. This is captured by
chat_model.model_kwargs.

PR passes tests and example notebooks and example testing. Still gonna
chat with some people, so leaving as draft for now.

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
This commit is contained in:
Vadim Kudlay
2024-02-09 15:46:02 -06:00
committed by GitHub
parent 932c52c333
commit 5f9ac6986e
6 changed files with 313 additions and 242 deletions

File diff suppressed because one or more lines are too long

View File

@@ -28,9 +28,17 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Note: you may need to restart the kernel to use updated packages.\n"
]
}
],
"source": [
"%pip install --upgrade --quiet langchain-nvidia-ai-endpoints"
]
@@ -56,7 +64,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 2,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -64,7 +72,15 @@
"id": "hoF41-tNczS3",
"outputId": "7f2833dc-191c-4d73-b823-7b2745a93a2f"
},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Valid NVIDIA_API_KEY already in environment. Delete to reset\n"
]
}
],
"source": [
"import getpass\n",
"import os\n",
@@ -105,7 +121,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 3,
"metadata": {
"id": "hbXmJssPdIPX"
},
@@ -180,7 +196,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 4,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -194,15 +210,15 @@
"output_type": "stream",
"text": [
"Single Query Embedding: \n",
"\u001b[1mExecuted in 1.39 seconds.\u001b[0m\n",
"\u001b[1mExecuted in 2.19 seconds.\u001b[0m\n",
"Shape: (1024,)\n",
"\n",
"Sequential Embedding: \n",
"\u001b[1mExecuted in 3.20 seconds.\u001b[0m\n",
"\u001b[1mExecuted in 3.16 seconds.\u001b[0m\n",
"Shape: (5, 1024)\n",
"\n",
"Batch Query Embedding: \n",
"\u001b[1mExecuted in 1.52 seconds.\u001b[0m\n",
"\u001b[1mExecuted in 1.23 seconds.\u001b[0m\n",
"Shape: (5, 1024)\n"
]
}
@@ -260,7 +276,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 5,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -274,11 +290,11 @@
"output_type": "stream",
"text": [
"Single Document Embedding: \n",
"\u001b[1mExecuted in 0.76 seconds.\u001b[0m\n",
"\u001b[1mExecuted in 0.52 seconds.\u001b[0m\n",
"Shape: (1024,)\n",
"\n",
"Batch Document Embedding: \n",
"\u001b[1mExecuted in 0.86 seconds.\u001b[0m\n",
"\u001b[1mExecuted in 0.89 seconds.\u001b[0m\n",
"Shape: (5, 1024)\n"
]
}
@@ -324,7 +340,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 6,
"metadata": {},
"outputs": [
{
@@ -341,7 +357,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 7,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",