mirror of
https://github.com/hwchase17/langchain.git
synced 2025-09-05 13:06:03 +00:00
Beam (#4996)
# Beam Calls the Beam API wrapper to deploy and make subsequent calls to an instance of the gpt2 LLM in a cloud deployment. Requires installation of the Beam library and registration of Beam Client ID and Client Secret. Additional calls can then be made through the instance of the large language model in your code or by calling the Beam API. --------- Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
This commit is contained in:
Nolan Tremelling
GitHub
92
docs/integrations/beam.md
Normal file
92
docs/integrations/beam.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# Beam
|
||||
|
||||
This page covers how to use Beam within LangChain.
|
||||
It is broken into two parts: installation and setup, and then references to specific Beam wrappers.
|
||||
|
||||
## Installation and Setup
|
||||
|
||||
- [Create an account](https://www.beam.cloud/)
|
||||
- Install the Beam CLI with `curl https://raw.githubusercontent.com/slai-labs/get-beam/main/get-beam.sh -sSfL | sh`
|
||||
- Register API keys with `beam configure`
|
||||
- Set environment variables (`BEAM_CLIENT_ID`) and (`BEAM_CLIENT_SECRET`)
|
||||
- Install the Beam SDK `pip install beam-sdk`
|
||||
|
||||
## Wrappers
|
||||
|
||||
### LLM
|
||||
|
||||
There exists a Beam LLM wrapper, which you can access with
|
||||
|
||||
```python
|
||||
from langchain.llms.beam import Beam
|
||||
```
|
||||
|
||||
## Define your Beam app.
|
||||
|
||||
This is the environment you’ll be developing against once you start the app.
|
||||
It's also used to define the maximum response length from the model.
|
||||
```python
|
||||
llm = Beam(model_name="gpt2",
|
||||
name="langchain-gpt2-test",
|
||||
cpu=8,
|
||||
memory="32Gi",
|
||||
gpu="A10G",
|
||||
python_version="python3.8",
|
||||
python_packages=[
|
||||
"diffusers[torch]>=0.10",
|
||||
"transformers",
|
||||
"torch",
|
||||
"pillow",
|
||||
"accelerate",
|
||||
"safetensors",
|
||||
"xformers",],
|
||||
max_length="50",
|
||||
verbose=False)
|
||||
```
|
||||
|
||||
## Deploy your Beam app
|
||||
|
||||
Once defined, you can deploy your Beam app by calling your model's `_deploy()` method.
|
||||
|
||||
```python
|
||||
llm._deploy()
|
||||
```
|
||||
|
||||
## Call your Beam app
|
||||
|
||||
Once a beam model is deployed, it can be called by callying your model's `_call()` method.
|
||||
This returns the GPT2 text response to your prompt.
|
||||
|
||||
```python
|
||||
response = llm._call("Running machine learning on a remote GPU")
|
||||
```
|
||||
|
||||
An example script which deploys the model and calls it would be:
|
||||
|
||||
```python
|
||||
from langchain.llms.beam import Beam
|
||||
import time
|
||||
|
||||
llm = Beam(model_name="gpt2",
|
||||
name="langchain-gpt2-test",
|
||||
cpu=8,
|
||||
memory="32Gi",
|
||||
gpu="A10G",
|
||||
python_version="python3.8",
|
||||
python_packages=[
|
||||
"diffusers[torch]>=0.10",
|
||||
"transformers",
|
||||
"torch",
|
||||
"pillow",
|
||||
"accelerate",
|
||||
"safetensors",
|
||||
"xformers",],
|
||||
max_length="50",
|
||||
verbose=False)
|
||||
|
||||
llm._deploy()
|
||||
|
||||
response = llm._call("Running machine learning on a remote GPU")
|
||||
|
||||
print(response)
|
||||
```
|
||||
159
docs/modules/models/llms/integrations/beam.ipynb
Normal file
159
docs/modules/models/llms/integrations/beam.ipynb
Normal file
@@ -0,0 +1,159 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "J-yvaDTmTTza"
|
||||
},
|
||||
"source": [
|
||||
"# Beam integration for langchain\n",
|
||||
"\n",
|
||||
"Calls the Beam API wrapper to deploy and make subsequent calls to an instance of the gpt2 LLM in a cloud deployment. Requires installation of the Beam library and registration of Beam Client ID and Client Secret. By calling the wrapper an instance of the model is created and run, with returned text relating to the prompt. Additional calls can then be made by directly calling the Beam API.\n",
|
||||
"\n",
|
||||
"[Create an account](https://www.beam.cloud/), if you don't have one already. Grab your API keys from the [dashboard](https://www.beam.cloud/dashboard/settings/api-keys)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "CfTmesWtTfTS"
|
||||
},
|
||||
"source": [
|
||||
"Install the Beam CLI"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "G_tCCurqR7Ik"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!curl https://raw.githubusercontent.com/slai-labs/get-beam/main/get-beam.sh -sSfL | sh"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "jJkcNqOdThQ7"
|
||||
},
|
||||
"source": [
|
||||
"Register API Keys and set your beam client id and secret environment variables:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "7gQd6fszSEaH"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"import subprocess\n",
|
||||
"\n",
|
||||
"beam_client_id = \"<Your beam client id>\"\n",
|
||||
"beam_client_secret = \"<Your beam client secret>\"\n",
|
||||
"\n",
|
||||
"# Set the environment variables\n",
|
||||
"os.environ['BEAM_CLIENT_ID'] = beam_client_id\n",
|
||||
"os.environ['BEAM_CLIENT_SECRET'] = beam_client_secret\n",
|
||||
"\n",
|
||||
"# Run the beam configure command\n",
|
||||
"!beam configure --clientId={beam_client_id} --clientSecret={beam_client_secret}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "c20rkK18TrK2"
|
||||
},
|
||||
"source": [
|
||||
"Install the Beam SDK:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "CH2Vop6ISNIf"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install beam-sdk"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "XflOsp3bTwl1"
|
||||
},
|
||||
"source": [
|
||||
"**Deploy and call Beam directly from langchain!**\n",
|
||||
"\n",
|
||||
"Note that a cold start might take a couple of minutes to return the response, but subsequent calls will be faster!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "KmaHxUqbSVnh"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langchain.llms.beam import Beam\n",
|
||||
"\n",
|
||||
"llm = Beam(model_name=\"gpt2\",\n",
|
||||
" name=\"langchain-gpt2-test\",\n",
|
||||
" cpu=8,\n",
|
||||
" memory=\"32Gi\",\n",
|
||||
" gpu=\"A10G\",\n",
|
||||
" python_version=\"python3.8\",\n",
|
||||
" python_packages=[\n",
|
||||
" \"diffusers[torch]>=0.10\",\n",
|
||||
" \"transformers\",\n",
|
||||
" \"torch\",\n",
|
||||
" \"pillow\",\n",
|
||||
" \"accelerate\",\n",
|
||||
" \"safetensors\",\n",
|
||||
" \"xformers\",],\n",
|
||||
" max_length=\"50\",\n",
|
||||
" verbose=False)\n",
|
||||
"\n",
|
||||
"llm._deploy()\n",
|
||||
"\n",
|
||||
"response = llm._call(\"Running machine learning on a remote GPU\")\n",
|
||||
"\n",
|
||||
"print(response)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"private_outputs": true,
|
||||
"provenance": []
|
||||
},
|
||||
"gpuClass": "standard",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 1
|
||||
}
|
||||
Reference in New Issue
Block a user