github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-09-06 21:43:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

GradientLLM Docs update and model_id renaming. (#10963)

Browse Source 
Related to #10800 

- Errors in the Docstring of GradientLLM / Gradient.ai LLM
- Renamed the `model_id` to `model` and adapting this in all tests.
Reason to so is to be in Sync with `GradientEmbeddings` and other LLM's.
- inmproving tests so they check the headers in the sent request.
- making the aiosession a private attribute in the docs, as in the
future `pip install gradientai` will be replacing aiosession.
- adding a example how to fine-tune on the Prompt Template as suggested
in #10800
This commit is contained in:
Michael Feil
2023-10-13 22:57:58 +02:00
committed by GitHub
parent 6876b02c87
commit 233a904f2e
6 changed files with 329 additions and 64 deletions
Download Patch File Download Diff File Expand all files Collapse all files

178
docs/docs/integrations/llms/gradient.ipynb
View File

@@ -24,8 +24,6 @@
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import requests\n",
"from langchain.llms import GradientLLM\n",
"from langchain.prompts import PromptTemplate\n",
"from langchain.chains import LLMChain"
@@ -46,7 +44,7 @@
"outputs": [],
"source": [
"from getpass import getpass\n",
"\n",
"import os\n",
"\n",
"if not os.environ.get(\"GRADIENT_ACCESS_TOKEN\",None):\n",
" # Access token under https://auth.gradient.ai/select-workspace\n",
@@ -61,7 +59,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Optional: Validate your Environment variables ```GRADIENT_ACCESS_TOKEN``` and ```GRADIENT_WORKSPACE_ID``` to get currently deployed models."
"Optional: Validate your Enviroment variables ```GRADIENT_ACCESS_TOKEN``` and ```GRADIENT_WORKSPACE_ID``` to get currently deployed models. Using the `gradientai` Python package."
]
},
{
@@ -73,25 +71,64 @@
"name": "stdout",
"output_type": "stream",
"text": [
"Credentials valid.\n",
"Possible values for `model_id` are:\n",
" {'models': [{'id': '99148c6d-c2a0-4fbe-a4a7-e7c05bdb8a09_base_ml_model', 'name': 'bloom-560m', 'slug': 'bloom-560m', 'type': 'baseModel'}, {'id': 'f0b97d96-51a8-4040-8b22-7940ee1fa24e_base_ml_model', 'name': 'llama2-7b-chat', 'slug': 'llama2-7b-chat', 'type': 'baseModel'}, {'id': 'cc2dafce-9e6e-4a23-a918-cad6ba89e42e_base_ml_model', 'name': 'nous-hermes2', 'slug': 'nous-hermes2', 'type': 'baseModel'}, {'baseModelId': 'f0b97d96-51a8-4040-8b22-7940ee1fa24e_base_ml_model', 'id': 'bb7b9865-0ce3-41a8-8e2b-5cbcbe1262eb_model_adapter', 'name': 'optical-transmitting-sensor', 'type': 'modelAdapter'}]}\n"
"Requirement already satisfied: gradientai in /home/michi/.venv/lib/python3.10/site-packages (1.0.0)\n",
"Requirement already satisfied: aenum>=3.1.11 in /home/michi/.venv/lib/python3.10/site-packages (from gradientai) (3.1.15)\n",
"Requirement already satisfied: pydantic<2.0.0,>=1.10.5 in /home/michi/.venv/lib/python3.10/site-packages (from gradientai) (1.10.12)\n",
"Requirement already satisfied: python-dateutil>=2.8.2 in /home/michi/.venv/lib/python3.10/site-packages (from gradientai) (2.8.2)\n",
"Requirement already satisfied: urllib3>=1.25.3 in /home/michi/.venv/lib/python3.10/site-packages (from gradientai) (1.26.16)\n",
"Requirement already satisfied: typing-extensions>=4.2.0 in /home/michi/.venv/lib/python3.10/site-packages (from pydantic<2.0.0,>=1.10.5->gradientai) (4.5.0)\n",
"Requirement already satisfied: six>=1.5 in /home/michi/.venv/lib/python3.10/site-packages (from python-dateutil>=2.8.2->gradientai) (1.16.0)\n"
]
}
],
"source": [
"import requests\n",
"!pip install gradientai"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"99148c6d-c2a0-4fbe-a4a7-e7c05bdb8a09_base_ml_model\n",
"f0b97d96-51a8-4040-8b22-7940ee1fa24e_base_ml_model\n",
"cc2dafce-9e6e-4a23-a918-cad6ba89e42e_base_ml_model\n"
]
}
],
"source": [
"import gradientai\n",
"\n",
"resp = requests.get(f'https://api.gradient.ai/api/models', headers={\n",
" \"authorization\": f\"Bearer {os.environ['GRADIENT_ACCESS_TOKEN']}\",\n",
" \"x-gradient-workspace-id\": f\"{os.environ['GRADIENT_WORKSPACE_ID']}\",\n",
" },\n",
" )\n",
"if resp.status_code == 200:\n",
" models = resp.json()\n",
" print(\"Credentials valid.\\nPossible values for `model_id` are:\\n\", models)\n",
"else:\n",
" print(\"Error when listing models. Are your credentials valid?\", resp.text)"
"client = gradientai.Gradient()\n",
"\n",
"models = client.list_models(only_base=True)\n",
"for model in models:\n",
" print(model.id)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"('674119b5-f19e-4856-add2-767ae7f7d7ef_model_adapter', 'my_model_adapter')"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"new_model = models[-1].create_model_adapter(name=\"my_model_adapter\")\n",
"new_model.id, new_model.name"
]
},
{
@@ -99,21 +136,24 @@
"metadata": {},
"source": [
"## Create the Gradient instance\n",
"You can specify different parameters such as the model name, max tokens generated, temperature, etc."
"You can specify different parameters such as the model, max_tokens generated, temperature, etc.\n",
"\n",
"As we later want to fine-tune out model, we select the model_adapter with the id `674119b5-f19e-4856-add2-767ae7f7d7ef_model_adapter`, but you can use any base or fine-tunable model."
]
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"llm = GradientLLM(\n",
" # `ID` listed in `$ gradient model list`\n",
" model_id=\"99148c6d-c2a0-4fbe-a4a7-e7c05bdb8a09_base_ml_model\",\n",
" model=\"674119b5-f19e-4856-add2-767ae7f7d7ef_model_adapter\",\n",
" # # optional: set new credentials, they default to environment variables\n",
" # gradient_workspace_id=os.environ[\"GRADIENT_WORKSPACE_ID\"],\n",
" # gradient_access_token=os.environ[\"GRADIENT_ACCESS_TOKEN\"],\n",
" model_kwargs=dict(max_generated_token_count=128)\n",
")"
]
},
@@ -127,13 +167,13 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"template = \"\"\"Question: {question}\n",
"\n",
"Answer: Let's think step by step.\"\"\"\n",
"Answer: \"\"\"\n",
"\n",
"prompt = PromptTemplate(template=template, input_variables=[\"question\"])"
]
@@ -147,7 +187,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
@@ -164,16 +204,16 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"' The first team to win the Super Bowl was the New England Patriots. The Patriots won the'"
"'\\nThe San Francisco 49ers won the Super Bowl in 1994.'"
]
},
"execution_count": 7,
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
@@ -185,6 +225,88 @@
" question=question\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Improve the results by fine-tuning (optional)\n",
"Well - that is wrong - the San Francisco 49ers did not win.\n",
"The correct answer to the question would be `The Dallas Cowboys!`.\n",
"\n",
"Let's increase the odds for the correct answer, by fine-tuning on the correct answer using the PromptTemplate."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'inputs': 'Question: What NFL team won the Super Bowl in 1994?\\n\\nAnswer: The Dallas Cowboys!'}]"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dataset = [{\"inputs\": template.format(question=\"What NFL team won the Super Bowl in 1994?\") + \" The Dallas Cowboys!\"}]\n",
"dataset"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"FineTuneResponse(number_of_trainable_tokens=27, sum_loss=78.17996)"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"new_model.fine_tune(\n",
" samples=dataset\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'The Dallas Cowboys'"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# we can keep the llm_chain, as the registered model just got refreshed on the gradient.ai servers.\n",
"llm_chain.run(\n",
" question=question\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
@@ -203,7 +325,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.13"
"version": "3.10.6"
},
"vscode": {
"interpreter": {

27
docs/docs/integrations/providers/gradient.mdx Normal file
View File

@@ -0,0 +1,27 @@
# Gradient
>[Gradient](https://gradient.ai/) allows to fine tune and get completions on LLMs with a simple web API.
## Installation and Setup
- Install the Python SDK :
```bash
pip install gradientai
```
Get a [Gradient access token and workspace](https://gradient.ai/) and set it as an environment variable (`Gradient_ACCESS_TOKEN`) and (`GRADIENT_WORKSPACE_ID`)
## LLM
There exists an Gradient LLM wrapper, which you can access with
See a [usage example](/docs/integrations/llms/gradient).
```python
from langchain.llms import GradientLLM
```
## Text Embedding Model
There exists an Gradient Embedding model, which you can access with
```python
from langchain.embeddings import GradientEmbeddings
```
For a more detailed walkthrough of this, see [this notebook](/docs/integrations/text_embedding/gradient.html)