github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-09-03 20:16:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Docs refactor (#480)

Browse Source 
Big docs refactor! Motivation is to make it easier for people to find
resources they are looking for. To accomplish this, there are now three
main sections:

- Getting Started: steps for getting started, walking through most core
functionality
- Modules: these are different modules of functionality that langchain
provides. Each part here has a "getting started", "how to", "key
concepts" and "reference" section (except in a few select cases where it
didnt easily fit).
- Use Cases: this is to separate use cases (like summarization, question
answering, evaluation, etc) from the modules, and provide a different
entry point to the code base.

There is also a full reference section, as well as extra resources
(glossary, gallery, etc)

Co-authored-by: Shreya Rajpal <ShreyaR@users.noreply.github.com>
This commit is contained in:
Harrison Chase
2023-01-02 08:24:09 -08:00
committed by GitHub
parent c5f0af9398
commit 985496f4be
164 changed files with 4326 additions and 2586 deletions
Download Patch File Download Diff File Expand all files Collapse all files

156
docs/modules/llms/examples/custom_llm.ipynb Normal file
View File

@@ -0,0 +1,156 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "9e9b7651",
"metadata": {},
"source": [
"# Custom LLM\n",
"\n",
"This notebook goes over how to create a custom LLM wrapper, in case you want to use your own LLM or a different wrapper than one that is supported in LangChain.\n",
"\n",
"There is only one required thing that a custom LLM needs to implement:\n",
"\n",
"1. A `_call` method that takes in a string, some optional stop words, and returns a string\n",
"\n",
"There is a second optional thing it can implement:\n",
"\n",
"1. An `_identifying_params` property that is used to help with printing of this class. Should return a dictionary.\n",
"\n",
"Let's implement a very simple custom LLM that just returns the first N characters of the input."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "a65696a0",
"metadata": {},
"outputs": [],
"source": [
"from langchain.llms.base import LLM\n",
"from typing import Optional, List, Mapping, Any"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "d5ceff02",
"metadata": {},
"outputs": [],
"source": [
"class CustomLLM(LLM):\n",
" \n",
" n: int\n",
" \n",
" @property\n",
" def _llm_type(self) -> str:\n",
" return \"custom\"\n",
" \n",
" def _call(self, prompt: str, stop: Optional[List[str]] = None) -> str:\n",
" if stop is not None:\n",
" raise ValueError(\"stop kwargs are not permitted.\")\n",
" return prompt[:self.n]\n",
" \n",
" @property\n",
" def _identifying_params(self) -> Mapping[str, Any]:\n",
" \"\"\"Get the identifying parameters.\"\"\"\n",
" return {\"n\": self.n}"
]
},
{
"cell_type": "markdown",
"id": "714dede0",
"metadata": {},
"source": [
"We can now use this as an any other LLM."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "10e5ece6",
"metadata": {},
"outputs": [],
"source": [
"llm = CustomLLM(n=10)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "8cd49199",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'This is a '"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"llm(\"This is a foobar thing\")"
]
},
{
"cell_type": "markdown",
"id": "bbfebea1",
"metadata": {},
"source": [
"We can also print the LLM and see its custom print."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "9c33fa19",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[1mCustomLLM\u001b[0m\n",
"Params: {'n': 10}\n"
]
}
],
"source": [
"print(llm)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6dac3f47",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

12
docs/modules/llms/examples/llm.json Normal file
View File

@@ -0,0 +1,12 @@
{
"model_name": "text-davinci-003",
"temperature": 0.7,
"max_tokens": 256,
"top_p": 1.0,
"frequency_penalty": 0.0,
"presence_penalty": 0.0,
"n": 1,
"best_of": 1,
"request_timeout": null,
"_type": "openai"
}

10
docs/modules/llms/examples/llm.yaml Normal file
View File

@@ -0,0 +1,10 @@
_type: openai
best_of: 1
frequency_penalty: 0.0
max_tokens: 256
model_name: text-davinci-003
n: 1
presence_penalty: 0.0
request_timeout: null
temperature: 0.7
top_p: 1.0

525
docs/modules/llms/examples/llm_caching.ipynb Normal file
View File

@@ -0,0 +1,525 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "f36d938c",
"metadata": {},
"source": [
"# LLM Caching\n",
"This notebook covers how to cache results of individual LLM calls."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "10ad9224",
"metadata": {},
"outputs": [],
"source": [
"from langchain.llms import OpenAI"
]
},
{
"cell_type": "markdown",
"id": "b50f0598",
"metadata": {},
"source": [
"## In Memory Cache"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "426ff912",
"metadata": {},
"outputs": [],
"source": [
"import langchain\n",
"from langchain.cache import InMemoryCache\n",
"langchain.llm_cache = InMemoryCache()"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "f69f6283",
"metadata": {},
"outputs": [],
"source": [
"# To make the caching really obvious, lets use a slower model.\n",
"llm = OpenAI(model_name=\"text-davinci-002\", n=2, best_of=2)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "64005d1f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 30.7 ms, sys: 18.6 ms, total: 49.3 ms\n",
"Wall time: 791 ms\n"
]
},
{
"data": {
"text/plain": [
"\"\\n\\nWhy couldn't the bicycle stand up by itself? Because it was...two tired!\""
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"# The first time, it is not yet in cache, so it should take longer\n",
"llm(\"Tell me a joke\")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "c8a1cb2b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 80 µs, sys: 0 ns, total: 80 µs\n",
"Wall time: 83.9 µs\n"
]
},
{
"data": {
"text/plain": [
"\"\\n\\nWhy couldn't the bicycle stand up by itself? Because it was...two tired!\""
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"# The second time it is, so it goes faster\n",
"llm(\"Tell me a joke\")"
]
},
{
"cell_type": "markdown",
"id": "4bf59c12",
"metadata": {},
"source": [
"## SQLite Cache"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "3ff65b00",
"metadata": {},
"outputs": [],
"source": [
"!rm .langchain.db"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "5f036236",
"metadata": {},
"outputs": [],
"source": [
"# We can do the same thing with a SQLite cache\n",
"from langchain.cache import SQLiteCache\n",
"langchain.llm_cache = SQLiteCache(database_path=\".langchain.db\")"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "fa18e3af",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 17 ms, sys: 9.76 ms, total: 26.7 ms\n",
"Wall time: 825 ms\n"
]
},
{
"data": {
"text/plain": [
"'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side.'"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"# The first time, it is not yet in cache, so it should take longer\n",
"llm(\"Tell me a joke\")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "5bf2f6fd",
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 2.46 ms, sys: 1.23 ms, total: 3.7 ms\n",
"Wall time: 2.67 ms\n"
]
},
{
"data": {
"text/plain": [
"'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side.'"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"# The second time it is, so it goes faster\n",
"llm(\"Tell me a joke\")"
]
},
{
"cell_type": "markdown",
"id": "278ad7ae",
"metadata": {},
"source": [
"## Redis Cache"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "39f6eb0b",
"metadata": {},
"outputs": [],
"source": [
"# We can do the same thing with a Redis cache\n",
"# (make sure your local Redis instance is running first before running this example)\n",
"from redis import Redis\n",
"from langchain.cache import RedisCache\n",
"langchain.llm_cache = RedisCache(redis_=Redis())"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "28920749",
"metadata": {},
"outputs": [],
"source": [
"%%time\n",
"# The first time, it is not yet in cache, so it should take longer\n",
"llm(\"Tell me a joke\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "94bf9415",
"metadata": {},
"outputs": [],
"source": [
"%%time\n",
"# The second time it is, so it goes faster\n",
"llm(\"Tell me a joke\")"
]
},
{
"cell_type": "markdown",
"id": "934943dc",
"metadata": {},
"source": [
"## SQLAlchemy Cache"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "acccff40",
"metadata": {},
"outputs": [],
"source": [
"# You can use SQLAlchemyCache to cache with any SQL database supported by SQLAlchemy.\n",
"\n",
"# from langchain.cache import SQLAlchemyCache\n",
"# from sqlalchemy import create_engine\n",
"\n",
"# engine = create_engine(\"postgresql://postgres:postgres@localhost:5432/postgres\")\n",
"# langchain.llm_cache = SQLAlchemyCache(engine)"
]
},
{
"cell_type": "markdown",
"id": "0c69d84d",
"metadata": {},
"source": [
"## Optional Caching\n",
"You can also turn off caching for specific LLMs should you choose. In the example below, even though global caching is enabled, we turn it off for a specific LLM"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "6af46e2b",
"metadata": {},
"outputs": [],
"source": [
"llm = OpenAI(model_name=\"text-davinci-002\", n=2, best_of=2, cache=False)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "26c4fd8f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 5.8 ms, sys: 2.71 ms, total: 8.51 ms\n",
"Wall time: 745 ms\n"
]
},
{
"data": {
"text/plain": [
"'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side!'"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"llm(\"Tell me a joke\")"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "46846b20",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 4.91 ms, sys: 2.64 ms, total: 7.55 ms\n",
"Wall time: 623 ms\n"
]
},
{
"data": {
"text/plain": [
"'\\n\\nTwo guys stole a calendar. They got six months each.'"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"llm(\"Tell me a joke\")"
]
},
{
"cell_type": "markdown",
"id": "5da41b77",
"metadata": {},
"source": [
"## Optional Caching in Chains\n",
"You can also turn off caching for particular nodes in chains. Note that because of certain interfaces, its often easier to construct the chain first, and then edit the LLM afterwards.\n",
"\n",
"As an example, we will load a summarizer map-reduce chain. We will cache results for the map-step, but then not freeze it for the combine step."
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "9afa3f7a",
"metadata": {},
"outputs": [],
"source": [
"llm = OpenAI(model_name=\"text-davinci-002\")\n",
"no_cache_llm = OpenAI(model_name=\"text-davinci-002\", cache=False)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "98a78e8e",
"metadata": {},
"outputs": [],
"source": [
"from langchain.text_splitter import CharacterTextSplitter\n",
"from langchain.chains.mapreduce import MapReduceChain\n",
"\n",
"text_splitter = CharacterTextSplitter()"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "2bfb099b",
"metadata": {},
"outputs": [],
"source": [
"with open('../../state_of_the_union.txt') as f:\n",
" state_of_the_union = f.read()\n",
"texts = text_splitter.split_text(state_of_the_union)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "f78b7f51",
"metadata": {},
"outputs": [],
"source": [
"from langchain.docstore.document import Document\n",
"docs = [Document(page_content=t) for t in texts[:3]]\n",
"from langchain.chains.summarize import load_summarize_chain"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "a2a30822",
"metadata": {},
"outputs": [],
"source": [
"chain = load_summarize_chain(llm, chain_type=\"map_reduce\", reduce_llm=no_cache_llm)"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "a545b743",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 452 ms, sys: 60.3 ms, total: 512 ms\n",
"Wall time: 5.09 s\n"
]
},
{
"data": {
"text/plain": [
"'\\n\\nPresident Biden is discussing the American Rescue Plan and the Bipartisan Infrastructure Law, which will create jobs and help Americans. He also talks about his vision for America, which includes investing in education and infrastructure. In response to Russian aggression in Ukraine, the United States is joining with European allies to impose sanctions and isolate Russia. American forces are being mobilized to protect NATO countries in the event that Putin decides to keep moving west. The Ukrainians are bravely fighting back, but the next few weeks will be hard for them. Putin will pay a high price for his actions in the long run. Americans should not be alarmed, as the United States is taking action to protect its interests and allies.'"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"chain.run(docs)"
]
},
{
"cell_type": "markdown",
"id": "3ed85e9d",
"metadata": {},
"source": [
"When we run it again, we see that it runs substantially faster but the final answer is different. This is due to caching at the map steps, but not at the reduce step."
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "39cbb282",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 11.5 ms, sys: 4.33 ms, total: 15.8 ms\n",
"Wall time: 1.04 s\n"
]
},
{
"data": {
"text/plain": [
"'\\n\\nPresident Biden is discussing the American Rescue Plan and the Bipartisan Infrastructure Law, which will create jobs and help Americans. He also talks about his vision for America, which includes investing in education and infrastructure.'"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"chain.run(docs)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9df0dab8",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

168
docs/modules/llms/examples/llm_serialization.ipynb Normal file
View File

@@ -0,0 +1,168 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "73f9bf40",
"metadata": {},
"source": [
"# LLM Serialization\n",
"\n",
"This notebook walks how to write and read an LLM Configuration to and from disk. This is useful if you want to save the configuration for a given LLM (eg the provider, the temperature, etc)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "9c9fb6ff",
"metadata": {},
"outputs": [],
"source": [
"from langchain.llms import OpenAI\n",
"from langchain.llms.loading import load_llm"
]
},
{
"cell_type": "markdown",
"id": "88ce018b",
"metadata": {},
"source": [
"## Loading\n",
"First, lets go over loading a LLM from disk. LLMs can be saved on disk in two formats: json or yaml. No matter the extension, they are loaded in the same way."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "f12b28f3",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\r\n",
" \"model_name\": \"text-davinci-003\",\r\n",
" \"temperature\": 0.7,\r\n",
" \"max_tokens\": 256,\r\n",
" \"top_p\": 1.0,\r\n",
" \"frequency_penalty\": 0.0,\r\n",
" \"presence_penalty\": 0.0,\r\n",
" \"n\": 1,\r\n",
" \"best_of\": 1,\r\n",
" \"request_timeout\": null,\r\n",
" \"_type\": \"openai\"\r\n",
"}"
]
}
],
"source": [
"!cat llm.json"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "9ab709fc",
"metadata": {},
"outputs": [],
"source": [
"llm = load_llm(\"llm.json\")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "095b1d56",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"_type: openai\r\n",
"best_of: 1\r\n",
"frequency_penalty: 0.0\r\n",
"max_tokens: 256\r\n",
"model_name: text-davinci-003\r\n",
"n: 1\r\n",
"presence_penalty: 0.0\r\n",
"request_timeout: null\r\n",
"temperature: 0.7\r\n",
"top_p: 1.0\r\n"
]
}
],
"source": [
"!cat llm.yaml"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "8cafaafe",
"metadata": {},
"outputs": [],
"source": [
"llm = load_llm(\"llm.yaml\")"
]
},
{
"cell_type": "markdown",
"id": "ab3e4223",
"metadata": {},
"source": [
"## Saving\n",
"If you want to go from a LLM in memory to a serialized version of it, you can do so easily by calling the `.save` method. Again, this supports both json and yaml."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "b38f685d",
"metadata": {},
"outputs": [],
"source": [
"llm.save(\"llm.json\")"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "b7365503",
"metadata": {},
"outputs": [],
"source": [
"llm.save(\"llm.yaml\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "68e45b1c",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

20
docs/modules/llms/generic_how_to.rst Normal file
View File

@@ -0,0 +1,20 @@
Generic Functionality
=====================
The examples here all address certain "how-to" guides for working with LLMs.
`LLM Serialization <examples/llm_serialization.html>`_: A walkthrough of how to serialize LLMs to and from disk.
`LLM Caching <examples/llm_caching.html>`_: Covers different types of caches, and how to use a cache to save results of LLM calls.
`Custom LLM <examples/custom_llm.html>`_: How to create and use a custom LLM class, in case you have an LLM not from one of the standard providers (including one that you host yourself).
.. toctree::
:maxdepth: 1
:glob:
:caption: Generic Functionality
:name: Generic Functionality
:hidden:
examples/*

246
docs/modules/llms/getting_started.ipynb Normal file
View File

@@ -0,0 +1,246 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "20ac6b98",
"metadata": {},
"source": [
"# Getting Started\n",
"\n",
"This notebook goes over how to use the LLM class in LangChain.\n",
"\n",
"The LLM class is a class designed for interfacing with LLMs. There are lots of LLM providers (OpenAI, Cohere, Hugging Face, etc) - this class is designed to provide a standard interface for all of them. In this part of the documentation, we will focus on generic LLM functionality. For details on working with a specific LLM wrapper, please see the examples in the [How-To section](how_to_guides.rst).\n",
"\n",
"For this notebook, we will work with an OpenAI LLM wrapper, although the functionalities highlighted are generic for all LLM types."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "df924055",
"metadata": {},
"outputs": [],
"source": [
"from langchain.llms import OpenAI"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "182b484c",
"metadata": {},
"outputs": [],
"source": [
"llm = OpenAI(model_name=\"text-ada-001\", n=2, best_of=2)"
]
},
{
"cell_type": "markdown",
"id": "9695ccfc",
"metadata": {},
"source": [
"**Generate Text:** The most basic functionality an LLM has is just the ability to call it, passing in a string and getting back a string."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "9d12ac26",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side.'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"llm(\"Tell me a joke\")"
]
},
{
"cell_type": "markdown",
"id": "e7d4d42d",
"metadata": {},
"source": [
"**Generate:** More broadly, you can call it with a list of inputs, getting back a more complete response than just the text. This complete response includes things like multiple top responses, as well as LLM provider specific information"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "f4dc241a",
"metadata": {},
"outputs": [],
"source": [
"llm_result = llm.generate([\"Tell me a joke\", \"Tell me a poem\"]*15)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "740392f6",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"30"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(llm_result.generations)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "ab6cdcf1",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Generation(text='\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side!'),\n",
" Generation(text='\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side.')]"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"llm_result.generations[0]"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "4946a778",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Generation(text=\"\\n\\nWhat if love neverspeech\\n\\nWhat if love never ended\\n\\nWhat if love was only a feeling\\n\\nI'll never know this love\\n\\nIt's not a feeling\\n\\nBut it's what we have for each other\\n\\nWe just know that love is something strong\\n\\nAnd we can't help but be happy\\n\\nWe just feel what love is for us\\n\\nAnd we love each other with all our heart\\n\\nWe just don't know how\\n\\nHow it will go\\n\\nBut we know that love is something strong\\n\\nAnd we'll always have each other\\n\\nIn our lives.\"),\n",
" Generation(text='\\n\\nOnce upon a time\\n\\nThere was a love so pure and true\\n\\nIt lasted for centuries\\n\\nAnd never became stale or dry\\n\\nIt was moving and alive\\n\\nAnd the heart of the love-ick\\n\\nIs still beating strong and true.')]"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"llm_result.generations[-1]"
]
},
{
"cell_type": "markdown",
"id": "9efae834",
"metadata": {},
"source": [
"You can also access provider specific information that is returned. This information is NOT standardized across providers."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "242e4527",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'token_usage': {'completion_tokens': 3903,\n",
" 'total_tokens': 4023,\n",
" 'prompt_tokens': 120}}"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"llm_result.llm_output"
]
},
{
"cell_type": "markdown",
"id": "bde8e04f",
"metadata": {},
"source": [
"**Number of Tokens:** You can also estimate how many tokens a piece of text will be in that model. This is useful because models have a context length (and cost more for more tokens), which means you need to be aware of how long the text you are passing in is.\n",
"\n",
"Notice that by default the tokens are estimated using a HuggingFace tokenizer."
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "b623c774",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"3"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"llm.get_num_tokens(\"what a joke\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b004ffdd",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
},
"vscode": {
"interpreter": {
"hash": "1235b9b19e8e9828b5c1fdb2cd89fe8d3de0fcde5ef5f3db36e4b671adb8660f"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}

17
docs/modules/llms/how_to_guides.rst Normal file
View File

@@ -0,0 +1,17 @@
How-To Guides
=============
The examples here all address certain "how-to" guides for working with LLMs.
They are split into two categories:
1. `Generic Functionality <generic_how_to.html>`_: Covering generic functionality all LLMs should have.
2. `Integrations <integrations.html>`_: Covering integrations with various LLM providers.
.. toctree::
:maxdepth: 1
:glob:
:hidden:
generic_how_to.rst
integrations.rst

20
docs/modules/llms/integrations.rst Normal file
View File

@@ -0,0 +1,20 @@
Integrations
=============
The examples here are all "how-to" guides for how to integrate with various LLM providers.
`Huggingface Hub <examples/huggingface_hub.html>`_: Covers how to connect to LLMs hosted on HuggingFace Hub.
`Azure OpenAI <examples/azure_openai_example.html>`_: Covers how to connect to Azure-hosted OpenAI Models.
`Manifest <examples/manifest.html>`_: Covers how to utilize the Manifest wrapper.
.. toctree::
:maxdepth: 1
:glob:
:caption: Specific LLM Integrations
:name: Specific LLM Integrations
:hidden:
integrations/*

159
docs/modules/llms/integrations/azure_openai_example.ipynb Normal file
View File

@@ -0,0 +1,159 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "9e9b7651",
"metadata": {},
"source": [
"# Azure OpenAI LLM Example\n",
"\n",
"This notebook goes over how to use Langchain with [Azure OpenAI](https://aka.ms/azure-openai).\n",
"\n",
"The Azure OpenAI API is compatible with OpenAI's API. The `openai` Python package makes it easy to use both OpenAI and Azure OpenAI. You can call Azure OpenAI the same way you call OpenAI with the exceptions noted below.\n",
"\n",
"## API configuration\n",
"You can configure the `openai` package to use Azure OpenAI using environment variables. The following is for `bash`:\n",
"\n",
"```bash\n",
"# Set this to `azure`\n",
"export OPENAI_API_TYPE=azure\n",
"# The API version you want to use: set this to `2022-12-01` for the released version.\n",
"export OPENAI_API_VERSION=2022-12-01\n",
"# The base URL for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource.\n",
"export OPENAI_API_BASE_URL=https://your-resource-name.openai.azure.com\n",
"# The API key for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource.\n",
"export OPENAI_API_KEY=<your Azure OpenAI API key>\n",
"```\n",
"\n",
"Alternatively, you can configure the API right within your running Python environment:\n",
"\n",
"```python\n",
"import os\n",
"os.environ[\"OPENAI_API_TYPE\"] = \"azure\"\n",
"...\n",
"```\n",
"\n",
"## Deployments\n",
"With Azure OpenAI, you set up your own deployments of the common GPT-3 and Codex models. When calling the API, you need to specify the deployment you want to use.\n",
"\n",
"Let's say your deployment name is `text-davinci-002-prod`. In the `openai` Python API, you can specify this deployment with the `engine` parameter. For example:\n",
"\n",
"```python\n",
"import openai\n",
"\n",
"response = openai.Completion.create(\n",
" engine=\"text-davinci-002-prod\",\n",
" prompt=\"This is a test\",\n",
" max_tokens=5\n",
")\n",
"```\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "8fad2a6e",
"metadata": {},
"outputs": [],
"source": [
"# Import Azure OpenAI\n",
"from langchain.llms import AzureOpenAI"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "8c80213a",
"metadata": {},
"outputs": [],
"source": [
"# Create an instance of Azure OpenAI\n",
"# Replace the deployment name with your own\n",
"llm = AzureOpenAI(deployment_name=\"text-davinci-002-prod\", model_name=\"text-davinci-002\")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "592dc404",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side.'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Run the LLM\n",
"llm(\"Tell me a joke\")"
]
},
{
"cell_type": "markdown",
"id": "bbfebea1",
"metadata": {},
"source": [
"We can also print the LLM and see its custom print."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "9c33fa19",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[1mAzureOpenAI\u001b[0m\n",
"Params: {'deployment_name': 'text-davinci-002', 'model_name': 'text-davinci-002', 'temperature': 0.7, 'max_tokens': 256, 'top_p': 1, 'frequency_penalty': 0, 'presence_penalty': 0, 'n': 1, 'best_of': 1}\n"
]
}
],
"source": [
"print(llm)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5a8b5917",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
},
"vscode": {
"interpreter": {
"hash": "3bae61d45a4f4d73ecea8149862d4bfbae7d4d4a2f71b6e609a1be8f6c8d4298"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}

71
docs/modules/llms/integrations/huggingface_hub.ipynb Normal file
View File

@@ -0,0 +1,71 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "959300d4",
"metadata": {},
"source": [
"# HuggingFace Hub\n",
"\n",
"This example showcases how to connect to the HuggingFace Hub."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "3acf0069",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The Seattle Seahawks won the Super Bowl in 2010. Justin Beiber was born in 2010. The\n"
]
}
],
"source": [
"from langchain import PromptTemplate, HuggingFaceHub, LLMChain\n",
"\n",
"template = \"\"\"Question: {question}\n",
"\n",
"Answer: Let's think step by step.\"\"\"\n",
"prompt = PromptTemplate(template=template, input_variables=[\"question\"])\n",
"llm_chain = LLMChain(prompt=prompt, llm=HuggingFaceHub(repo_id=\"google/flan-t5-xl\", model_kwargs={\"temperature\":1e-10}))\n",
"\n",
"question = \"What NFL team won the Super Bowl in the year Justin Beiber was born?\"\n",
"\n",
"print(llm_chain.run(question))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ae4559c7",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

215
docs/modules/llms/integrations/manifest.ipynb Normal file
View File

@@ -0,0 +1,215 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "b4462a94",
"metadata": {},
"source": [
"# Manifest\n",
"\n",
"This notebook goes over how to use Manifest and LangChain."
]
},
{
"cell_type": "markdown",
"id": "59fcaebc",
"metadata": {},
"source": [
"For more detailed information on `manifest`, and how to use it with local hugginface models like in this example, see https://github.com/HazyResearch/manifest"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "04a0170a",
"metadata": {},
"outputs": [],
"source": [
"from manifest import Manifest\n",
"from langchain.llms.manifest import ManifestWrapper"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "de250a6a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'model_name': 'bigscience/T0_3B', 'model_path': 'bigscience/T0_3B'}\n"
]
}
],
"source": [
"manifest = Manifest(\n",
" client_name = \"huggingface\",\n",
" client_connection = \"http://127.0.0.1:5000\"\n",
")\n",
"print(manifest.client.get_model_params())"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "67b719d6",
"metadata": {},
"outputs": [],
"source": [
"llm = ManifestWrapper(client=manifest, llm_kwargs={\"temperature\": 0.001, \"max_tokens\": 256})"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "5af505a8",
"metadata": {},
"outputs": [],
"source": [
"# Map reduce example\n",
"from langchain import PromptTemplate\n",
"from langchain.text_splitter import CharacterTextSplitter\n",
"from langchain.chains.mapreduce import MapReduceChain\n",
"\n",
"\n",
"_prompt = \"\"\"Write a concise summary of the following:\n",
"\n",
"\n",
"{text}\n",
"\n",
"\n",
"CONCISE SUMMARY:\"\"\"\n",
"prompt = PromptTemplate(template=_prompt, input_variables=[\"text\"])\n",
"\n",
"text_splitter = CharacterTextSplitter()\n",
"\n",
"mp_chain = MapReduceChain.from_params(llm, prompt, text_splitter)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "485b3ec3",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'President Obama delivered his annual State of the Union address on Tuesday night, laying out his priorities for the coming year. Obama said the government will provide free flu vaccines to all Americans, ending the government shutdown and allowing businesses to reopen. The president also said that the government will continue to send vaccines to 112 countries, more than any other nation. \"We have lost so much to COVID-19,\" Trump said. \"Time with one another. And worst of all, so much loss of life.\" He said the CDC is working on a vaccine for kids under 5, and that the government will be ready with plenty of vaccines when they are available. Obama says the new guidelines are a \"great step forward\" and that the virus is no longer a threat. He says the government is launching a \"Test to Treat\" initiative that will allow people to get tested at a pharmacy and get antiviral pills on the spot at no cost. Obama says the new guidelines are a \"great step forward\" and that the virus is no longer a threat. He says the government will continue to send vaccines to 112 countries, more than any other nation. \"We are coming for your'"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"with open('../state_of_the_union.txt') as f:\n",
" state_of_the_union = f.read()\n",
"mp_chain.run(state_of_the_union)"
]
},
{
"cell_type": "markdown",
"id": "6e9d45a8",
"metadata": {},
"source": [
"## Compare HF Models"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "33407ab3",
"metadata": {},
"outputs": [],
"source": [
"from langchain.model_laboratory import ModelLaboratory\n",
"\n",
"manifest1 = ManifestWrapper(\n",
" client=Manifest(\n",
" client_name=\"huggingface\",\n",
" client_connection=\"http://127.0.0.1:5000\"\n",
" ),\n",
" llm_kwargs={\"temperature\": 0.01}\n",
")\n",
"manifest2 = ManifestWrapper(\n",
" client=Manifest(\n",
" client_name=\"huggingface\",\n",
" client_connection=\"http://127.0.0.1:5001\"\n",
" ),\n",
" llm_kwargs={\"temperature\": 0.01}\n",
")\n",
"manifest3 = ManifestWrapper(\n",
" client=Manifest(\n",
" client_name=\"huggingface\",\n",
" client_connection=\"http://127.0.0.1:5002\"\n",
" ),\n",
" llm_kwargs={\"temperature\": 0.01}\n",
")\n",
"llms = [manifest1, manifest2, manifest3]\n",
"model_lab = ModelLaboratory(llms)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "448935c3",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[1mInput:\u001b[0m\n",
"What color is a flamingo?\n",
"\n",
"\u001b[1mManifestWrapper\u001b[0m\n",
"Params: {'model_name': 'bigscience/T0_3B', 'model_path': 'bigscience/T0_3B', 'temperature': 0.01}\n",
"\u001b[104mpink\u001b[0m\n",
"\n",
"\u001b[1mManifestWrapper\u001b[0m\n",
"Params: {'model_name': 'EleutherAI/gpt-neo-125M', 'model_path': 'EleutherAI/gpt-neo-125M', 'temperature': 0.01}\n",
"\u001b[103mA flamingo is a small, round\u001b[0m\n",
"\n",
"\u001b[1mManifestWrapper\u001b[0m\n",
"Params: {'model_name': 'google/flan-t5-xl', 'model_path': 'google/flan-t5-xl', 'temperature': 0.01}\n",
"\u001b[101mpink\u001b[0m\n",
"\n"
]
}
],
"source": [
"model_lab.compare(\"What color is a flamingo?\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.7"
},
"vscode": {
"interpreter": {
"hash": "51b9b5b89a4976ad21c8b4273a6c78d700e2954ce7d7452948b7774eb33bbce4"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}

19
docs/modules/llms/key_concepts.md Normal file
View File

@@ -0,0 +1,19 @@
# Key Concepts
## LLMs
Wrappers around Large Language Models (in particular, the "generate" ability of large language models) are at the core of LangChain functionality.
The core method that these classes expose is a `generate` method, which takes in a list of strings and returns an LLMResult (which contains outputs for all input strings).
Read more about LLMResult. This interface operates over a list of strings because often the lists of strings can be batched to the LLM provider,
providing speed and efficiency gains.
For convenience, this class also exposes a simpler, more user friendly interface (via `__call__`).
The interface for this takes in a single string, and returns a single string.
## Generation
The output of a single generation. Currently in LangChain this is just the generated text, although could be extended in the future
to contain log probs or the like.
## LLMResult
The full output of a call to the `generate` method of the LLM class.
Since the `generate` method takes as input a list of strings, this returns a list of results.
Each result consists of a list of generations (since you can request N generations per input string).
This also contains a `llm_output` attribute which contains provider-specific information about the call.