Compare commits

...

1 Commits

Author SHA1 Message Date
Erick Friis
d811e969c0 docs[patch]: fix star bullets 2023-12-11 11:04:50 -08:00
56 changed files with 542 additions and 340 deletions

View File

@@ -10,14 +10,14 @@
"The `RunnableWithMessageHistory` let's us add message history to certain types of chains.\n",
"\n",
"Specifically, it can be used for any Runnable that takes as input one of\n",
"* a sequence of `BaseMessage`\n",
"* a dict with a key that takes a sequence of `BaseMessage`\n",
"* a dict with a key that takes the latest message(s) as a string or sequence of `BaseMessage`, and a separate key that takes historical messages\n",
"- a sequence of `BaseMessage`\n",
"- a dict with a key that takes a sequence of `BaseMessage`\n",
"- a dict with a key that takes the latest message(s) as a string or sequence of `BaseMessage`, and a separate key that takes historical messages\n",
"\n",
"And returns as output one of\n",
"* a string that can be treated as the contents of an `AIMessage`\n",
"* a sequence of `BaseMessage`\n",
"* a dict with a key that contains a sequence of `BaseMessage`\n",
"- a string that can be treated as the contents of an `AIMessage`\n",
"- a sequence of `BaseMessage`\n",
"- a dict with a key that contains a sequence of `BaseMessage`\n",
"\n",
"Let's take a look at some examples to see how it works."
]

View File

@@ -71,9 +71,9 @@
" \n",
"The instructions [here](docs/integrations/llms/ollama) provide details, which we summarize:\n",
" \n",
"* [Download and run](https://ollama.ai/download) the app\n",
"* From command line, fetch a model from this [list of options](https://github.com/jmorganca/ollama): e.g., `ollama pull llama2`\n",
"* When the app is running, all models are automatically served on `localhost:11434`\n"
"- [Download and run](https://ollama.ai/download) the app\n",
"- From command line, fetch a model from this [list of options](https://github.com/jmorganca/ollama): e.g., `ollama pull llama2`\n",
"- When the app is running, all models are automatically served on `localhost:11434`\n"
]
},
{
@@ -199,9 +199,9 @@
"\n",
"With [Ollama](docs/integrations/llms/ollama), fetch a model via `ollama pull <model family>:<tag>`:\n",
"\n",
"* E.g., for Llama-7b: `ollama pull llama2` will download the most basic version of the model (e.g., smallest # parameters and 4 bit quantization)\n",
"* We can also specify a particular version from the [model list](https://github.com/jmorganca/ollama), e.g., `ollama pull llama2:13b`\n",
"* See the full set of parameters on the [API reference page](https://api.python.langchain.com/en/latest/llms/langchain.llms.ollama.Ollama.html)"
"- E.g., for Llama-7b: `ollama pull llama2` will download the most basic version of the model (e.g., smallest # parameters and 4 bit quantization)\n",
"- We can also specify a particular version from the [model list](https://github.com/jmorganca/ollama), e.g., `ollama pull llama2:13b`\n",
"- See the full set of parameters on the [API reference page](https://api.python.langchain.com/en/latest/llms/langchain.llms.ollama.Ollama.html)"
]
},
{
@@ -245,20 +245,20 @@
"\n",
"`n_gpu_layers`: number of layers to be loaded into GPU memory\n",
"\n",
"* Value: 1\n",
"* Meaning: Only one layer of the model will be loaded into GPU memory (1 is often sufficient).\n",
"- Value: 1\n",
"- Meaning: Only one layer of the model will be loaded into GPU memory (1 is often sufficient).\n",
"\n",
"`n_batch`: number of tokens the model should process in parallel \n",
"* Value: n_batch\n",
"* Meaning: It's recommended to choose a value between 1 and n_ctx (which in this case is set to 2048)\n",
"- Value: n_batch\n",
"- Meaning: It's recommended to choose a value between 1 and n_ctx (which in this case is set to 2048)\n",
"\n",
"`n_ctx`: Token context window .\n",
"* Value: 2048\n",
"* Meaning: The model will consider a window of 2048 tokens at a time\n",
"- Value: 2048\n",
"- Meaning: The model will consider a window of 2048 tokens at a time\n",
"\n",
"`f16_kv`: whether the model should use half-precision for the key/value cache\n",
"* Value: True\n",
"* Meaning: The model will use half-precision, which can be more memory efficient; Metal only supports True."
"- Value: True\n",
"- Meaning: The model will use half-precision, which can be more memory efficient; Metal only supports True."
]
},
{
@@ -585,8 +585,8 @@
"\n",
"In general, use cases for local LLMs can be driven by at least two factors:\n",
"\n",
"* `Privacy`: private data (e.g., journals, etc) that a user does not want to share \n",
"* `Cost`: text preprocessing (extraction/tagging), summarization, and agent simulations are token-use-intensive tasks\n",
"- `Privacy`: private data (e.g., journals, etc) that a user does not want to share \n",
"- `Cost`: text preprocessing (extraction/tagging), summarization, and agent simulations are token-use-intensive tasks\n",
"\n",
"In addition, [here](https://blog.langchain.dev/using-langsmith-to-support-fine-tuning-of-open-source-llms/) is an overview on fine-tuning, which can utilize open-source LLMs."
]

View File

@@ -11,11 +11,11 @@
"\n",
"This example shows how one can track the following while calling OpenAI and ChatOpenAI models via `LangChain` and [Infino](https://github.com/infinohq/infino):\n",
"\n",
"* prompt input\n",
"* response from `ChatGPT` or any other `LangChain` model\n",
"* latency\n",
"* errors\n",
"* number of tokens consumed"
"- prompt input\n",
"- response from `ChatGPT` or any other `LangChain` model\n",
"- latency\n",
"- errors\n",
"- number of tokens consumed"
]
},
{

View File

@@ -12,9 +12,9 @@
">[Amazon SageMaker Experiments](https://docs.aws.amazon.com/sagemaker/latest/dg/experiments.html) is a capability of `Amazon SageMaker` that lets you organize, track, compare and evaluate ML experiments and model versions.\n",
"\n",
"This notebook shows how LangChain Callback can be used to log and track prompts and other LLM hyperparameters into `SageMaker Experiments`. Here, we use different scenarios to showcase the capability:\n",
"* **Scenario 1**: *Single LLM* - A case where a single LLM model is used to generate output based on a given prompt.\n",
"* **Scenario 2**: *Sequential Chain* - A case where a sequential chain of two LLM models is used.\n",
"* **Scenario 3**: *Agent with Tools (Chain of Thought)* - A case where multiple tools (search and math) are used in addition to an LLM.\n",
"- **Scenario 1**: *Single LLM* - A case where a single LLM model is used to generate output based on a given prompt.\n",
"- **Scenario 2**: *Sequential Chain* - A case where a sequential chain of two LLM models is used.\n",
"- **Scenario 3**: *Agent with Tools (Chain of Thought)* - A case where multiple tools (search and math) are used in addition to an LLM.\n",
"\n",
"\n",
"In this notebook, we will create a single experiment to log the prompts from each scenario."
@@ -50,8 +50,8 @@
},
"source": [
"First, setup the required API keys\n",
"* OpenAI: https://platform.openai.com/account/api-keys (For OpenAI LLM model)\n",
"* Google SERP API: https://serpapi.com/manage-api-key (For Google Search Tool)"
"- OpenAI: https://platform.openai.com/account/api-keys (For OpenAI LLM model)\n",
"- Google SERP API: https://serpapi.com/manage-api-key (For Google Search Tool)"
]
},
{

View File

@@ -20,8 +20,8 @@
"\n",
"This notebook demonstrates the use of `langchain.chat_models.ChatAnyscale` for [Anyscale Endpoints](https://endpoints.anyscale.com/).\n",
"\n",
"* Set `ANYSCALE_API_KEY` environment variable\n",
"* or use the `anyscale_api_key` keyword argument"
"- Set `ANYSCALE_API_KEY` environment variable\n",
"- or use the `anyscale_api_key` keyword argument"
]
},
{

View File

@@ -39,8 +39,8 @@
"\n",
"To use the wrapper, you must [deploy a model on AzureML](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-foundation-models?view=azureml-api-2#deploying-foundation-models-to-endpoints-for-inferencing) and obtain the following parameters:\n",
"\n",
"* `endpoint_api_key`: The API key provided by the endpoint\n",
"* `endpoint_url`: The REST endpoint url provided by the endpoint"
"- `endpoint_api_key`: The API key provided by the endpoint\n",
"- `endpoint_url`: The REST endpoint url provided by the endpoint"
]
},
{
@@ -51,7 +51,7 @@
"\n",
"The `content_formatter` parameter is a handler class for transforming the request and response of an AzureML endpoint to match with required schema. Since there are a wide range of models in the model catalog, each of which may process data differently from one another, a `ContentFormatterBase` class is provided to allow users to transform data to their liking. The following content formatters are provided:\n",
"\n",
"* `LLamaContentFormatter`: Formats request and response data for LLaMa2-chat"
"- `LLamaContentFormatter`: Formats request and response data for LLaMa2-chat"
]
},
{

View File

@@ -21,8 +21,8 @@
"\n",
"This notebook demonstrates the use of `langchain.chat_models.ChatEverlyAI` for [EverlyAI Hosted Endpoints](https://everlyai.xyz/).\n",
"\n",
"* Set `EVERLYAI_API_KEY` environment variable\n",
"* or use the `everlyai_api_key` keyword argument"
"- Set `EVERLYAI_API_KEY` environment variable\n",
"- or use the `everlyai_api_key` keyword argument"
]
},
{

View File

@@ -27,30 +27,30 @@
"\n",
"First, follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance:\n",
"\n",
"* [Download](https://ollama.ai/download)\n",
"* Fetch a model via `ollama pull <model family>`\n",
"* e.g., for `Llama-7b`: `ollama pull llama2`\n",
"* This will download the most basic version of the model (e.g., minimum # parameters and 4-bit quantization)\n",
"* On Mac, it will download to:\n",
"- [Download](https://ollama.ai/download)\n",
"- Fetch a model via `ollama pull <model family>`\n",
"- e.g., for `Llama-7b`: `ollama pull llama2`\n",
"- This will download the most basic version of the model (e.g., minimum # parameters and 4-bit quantization)\n",
"- On Mac, it will download to:\n",
"\n",
"`~/.ollama/models/manifests/registry.ollama.ai/library/<model family>/latest`\n",
"\n",
"* And we can specify a particular version, e.g., for `ollama pull vicuna:13b-v1.5-16k-q4_0`\n",
"* The file is here with the model version in place of `latest`\n",
"- And we can specify a particular version, e.g., for `ollama pull vicuna:13b-v1.5-16k-q4_0`\n",
"- The file is here with the model version in place of `latest`\n",
"\n",
"`~/.ollama/models/manifests/registry.ollama.ai/library/vicuna/13b-v1.5-16k-q4_0`\n",
"\n",
"You can easily access models in a few ways:\n",
"\n",
"1/ if the app is running:\n",
"* All of your local models are automatically served on `localhost:11434`\n",
"* Select your model when setting `llm = Ollama(..., model=\"<model family>:<version>\")`\n",
"* If you set `llm = Ollama(..., model=\"<model family\")` withoout a version it will simply look for `latest`\n",
"- All of your local models are automatically served on `localhost:11434`\n",
"- Select your model when setting `llm = Ollama(..., model=\"<model family>:<version>\")`\n",
"- If you set `llm = Ollama(..., model=\"<model family\")` withoout a version it will simply look for `latest`\n",
"\n",
"2/ if building from source or just running the binary: \n",
"* Then you must run `ollama serve`\n",
"* All of your local models are automatically served on `localhost:11434`\n",
"* Then, select as shown above\n",
"- Then you must run `ollama serve`\n",
"- All of your local models are automatically served on `localhost:11434`\n",
"- Then, select as shown above\n",
"\n",
"\n",
"## Usage\n",
@@ -170,8 +170,8 @@
" Sure! Here's a JSON response with the colors of the sky at different times of the day:\n",
" Begriffe und Abkürzungen:\n",
"\n",
"* `time`: The time of day (in 24-hour format)\n",
"* `sky_color`: The color of the sky at that time (as a hex code)\n",
"- `time`: The time of day (in 24-hour format)\n",
"- `sky_color`: The color of the sky at that time (as a hex code)\n",
"\n",
"Here are the colors of the sky at different times of the day:\n",
"```json\n",

View File

@@ -86,8 +86,8 @@
"metadata": {},
"source": [
"Now you can create an `AirbyteCDKLoader` based on the imported source. It takes a `config` object that's passed to the connector. You also have to pick the stream you want to retrieve records from by name (`stream_name`). Check the connectors documentation page and spec definition for more information on the config object and available streams. For the Github connectors these are:\n",
"* [https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-github/source_github/spec.json](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-github/source_github/spec.json).\n",
"* [https://docs.airbyte.com/integrations/sources/github/](https://docs.airbyte.com/integrations/sources/github/)"
"- [https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-github/source_github/spec.json](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-github/source_github/spec.json).\n",
"- [https://docs.airbyte.com/integrations/sources/github/](https://docs.airbyte.com/integrations/sources/github/)"
]
},
{

View File

@@ -33,9 +33,9 @@
"id": "637e1205",
"metadata": {},
"source": [
"* Get your API key [here](https://support.airtable.com/docs/creating-and-using-api-keys-and-access-tokens).\n",
"* Get ID of your base [here](https://airtable.com/developers/web/api/introduction).\n",
"* Get your table ID from the table url as shown [here](https://www.highviewapps.com/kb/where-can-i-find-the-airtable-base-id-and-table-id/#:~:text=Both%20the%20Airtable%20Base%20ID,URL%20that%20begins%20with%20tbl)."
"- Get your API key [here](https://support.airtable.com/docs/creating-and-using-api-keys-and-access-tokens).\n",
"- Get ID of your base [here](https://airtable.com/developers/web/api/introduction).\n",
"- Get your table ID from the table url as shown [here](https://www.highviewapps.com/kb/where-can-i-find-the-airtable-base-id-and-table-id/#:~:text=Both%20the%20Airtable%20Base%20ID,URL%20that%20begins%20with%20tbl)."
]
},
{

View File

@@ -29,17 +29,17 @@
"\n",
"Initially this Loader supports:\n",
"\n",
"* Loading NFTs as Documents from NFT Smart Contracts (ERC721 and ERC1155)\n",
"* Ethereum Mainnnet, Ethereum Testnet, Polygon Mainnet, Polygon Testnet (default is eth-mainnet)\n",
"* Alchemy's getNFTsForCollection API\n",
"- Loading NFTs as Documents from NFT Smart Contracts (ERC721 and ERC1155)\n",
"- Ethereum Mainnnet, Ethereum Testnet, Polygon Mainnet, Polygon Testnet (default is eth-mainnet)\n",
"- Alchemy's getNFTsForCollection API\n",
"\n",
"It can be extended if the community finds value in this loader. Specifically:\n",
"\n",
"* Additional APIs can be added (e.g. Tranction-related APIs)\n",
"- Additional APIs can be added (e.g. Tranction-related APIs)\n",
"\n",
"This Document Loader Requires:\n",
"\n",
"* A free [Alchemy API Key](https://www.alchemy.com/)\n",
"- A free [Alchemy API Key](https://www.alchemy.com/)\n",
"\n",
"The output takes the following format:\n",
"\n",

View File

@@ -18,33 +18,33 @@
"You will need a `Etherscan api key` to proceed. The free api key has 5 calls per seconds quota.\n",
"\n",
"The loader supports the following six functionalities:\n",
"* Retrieve normal transactions under specific account on Ethereum Mainet\n",
"* Retrieve internal transactions under specific account on Ethereum Mainet\n",
"* Retrieve erc20 transactions under specific account on Ethereum Mainet\n",
"* Retrieve erc721 transactions under specific account on Ethereum Mainet\n",
"* Retrieve erc1155 transactions under specific account on Ethereum Mainet\n",
"* Retrieve ethereum balance in wei under specific account on Ethereum Mainet\n",
"- Retrieve normal transactions under specific account on Ethereum Mainet\n",
"- Retrieve internal transactions under specific account on Ethereum Mainet\n",
"- Retrieve erc20 transactions under specific account on Ethereum Mainet\n",
"- Retrieve erc721 transactions under specific account on Ethereum Mainet\n",
"- Retrieve erc1155 transactions under specific account on Ethereum Mainet\n",
"- Retrieve ethereum balance in wei under specific account on Ethereum Mainet\n",
"\n",
"\n",
"If the account does not have corresponding transactions, the loader will a list with one document. The content of document is ''.\n",
"\n",
"You can pass different filters to loader to access different functionalities we mentioned above:\n",
"* \"normal_transaction\"\n",
"* \"internal_transaction\"\n",
"* \"erc20_transaction\"\n",
"* \"eth_balance\"\n",
"* \"erc721_transaction\"\n",
"* \"erc1155_transaction\"\n",
"- \"normal_transaction\"\n",
"- \"internal_transaction\"\n",
"- \"erc20_transaction\"\n",
"- \"eth_balance\"\n",
"- \"erc721_transaction\"\n",
"- \"erc1155_transaction\"\n",
"The filter is default to normal_transaction\n",
"\n",
"If you have any questions, you can access [Etherscan API Doc](https://etherscan.io/tx/0x0ffa32c787b1398f44303f731cb06678e086e4f82ce07cebf75e99bb7c079c77) or contact me via i@inevitable.tech.\n",
"\n",
"All functions related to transactions histories are restricted 1000 histories maximum because of Etherscan limit. You can use the following parameters to find the transaction histories you need:\n",
"* offset: default to 20. Shows 20 transactions for one time\n",
"* page: default to 1. This controls pagination.\n",
"* start_block: Default to 0. The transaction histories starts from 0 block.\n",
"* end_block: Default to 99999999. The transaction histories starts from 99999999 block\n",
"* sort: \"desc\" or \"asc\". Set default to \"desc\" to get latest transactions."
"- offset: default to 20. Shows 20 transactions for one time\n",
"- page: default to 1. This controls pagination.\n",
"- start_block: Default to 0. The transaction histories starts from 0 block.\n",
"- end_block: Default to 99999999. The transaction histories starts from 99999999 block\n",
"- sort: \"desc\" or \"asc\". Set default to \"desc\" to get latest transactions."
]
},
{

View File

@@ -24,8 +24,8 @@
"The first time you use GoogleDriveLoader, you will be displayed with the consent screen in your browser. If this doesn't happen and you get a `RefreshError`, do not use `credentials_path` in your `GoogleDriveLoader` constructor call. Instead, put that path in a `GOOGLE_APPLICATION_CREDENTIALS` environmental variable.\n",
"\n",
"`GoogleDriveLoader` can load from a list of Google Docs document ids or a folder id. You can obtain your folder and document id from the URL:\n",
"* Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",
"* Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is `\"1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw\"`"
"- Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",
"- Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is `\"1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw\"`"
]
},
{

View File

@@ -46,10 +46,10 @@
"\n",
"**Parameters**:\n",
"\n",
"* `include_outputs` (bool): whether to include cell outputs in the resulting document (default is False).\n",
"* `max_output_length` (int): the maximum number of characters to include from each cell output (default is 10).\n",
"* `remove_newline` (bool): whether to remove newline characters from the cell sources and outputs (default is False).\n",
"* `traceback` (bool): whether to include full traceback (default is False)."
"- `include_outputs` (bool): whether to include cell outputs in the resulting document (default is False).\n",
"- `max_output_length` (int): the maximum number of characters to include from each cell output (default is 10).\n",
"- `remove_newline` (bool): whether to remove newline characters from the cell sources and outputs (default is False).\n",
"- `traceback` (bool): whether to include full traceback (default is False)."
]
},
{

View File

@@ -38,10 +38,10 @@
"\n",
"The Loader requires the following parameters:\n",
"\n",
"* MongoDB connection string\n",
"* MongoDB database name\n",
"* MongoDB collection name\n",
"* (Optional) Content Filter dictionary\n",
"- MongoDB connection string\n",
"- MongoDB database name\n",
"- MongoDB collection name\n",
"- (Optional) Content Filter dictionary\n",
"\n",
"The output takes the following format:\n",
"\n",

View File

@@ -14,9 +14,9 @@
"\n",
"Specify a list `folder_ids` and/or `thread_ids` to load in the corresponding docs into Document objects, if both are specified, loader will get all `thread_ids` belong to this folder based on `folder_ids`, combine with passed `thread_ids`, the union of both sets will be returned.\n",
"\n",
"* How to know folder_id ? \n",
"- How to know folder_id ? \n",
" go to quip folder, right click folder and copy link, extract suffix from link as folder_id. Hint: `https://example.quip.com/<folder_id>`\n",
"* How to know thread_id ? \n",
"- How to know thread_id ? \n",
" thread_id is the document id. Go to quip doc, right click doc and copy link, extract suffix from link as thread_id. Hint: `https://exmaple.quip.com/<thread_id>`\n",
" \n",
"You can also set `include_all_folders` as `True` will fetch group_folder_ids and \n",

View File

@@ -58,9 +58,9 @@
"source": [
"You can import various items from RSpace:\n",
"\n",
"* A single RSpace structured or basic document. This will map 1-1 to a Langchain document.\n",
"* A folder or noteook. All documents inside the notebook or folder are imported as Langchain documents. \n",
"* If you have PDF files in the RSpace Gallery, these can be imported individually as well. Under the hood, Langchain's PDF loader will be used and this creates one Langchain document per PDF page. "
"- A single RSpace structured or basic document. This will map 1-1 to a Langchain document.\n",
"- A folder or noteook. All documents inside the notebook or folder are imported as Langchain documents. \n",
"- If you have PDF files in the RSpace Gallery, these can be imported individually as well. Under the hood, Langchain's PDF loader will be used and this creates one Langchain document per PDF page. "
]
},
{

View File

@@ -18,13 +18,13 @@
"\n",
"For example, we can scrape text content within `<p>, <li>, <div>, and <a>` tags from the HTML content:\n",
"\n",
"* `<p>`: The paragraph tag. It defines a paragraph in HTML and is used to group together related sentences and/or phrases.\n",
"- `<p>`: The paragraph tag. It defines a paragraph in HTML and is used to group together related sentences and/or phrases.\n",
" \n",
"* `<li>`: The list item tag. It is used within ordered (`<ol>`) and unordered (`<ul>`) lists to define individual items within the list.\n",
"- `<li>`: The list item tag. It is used within ordered (`<ol>`) and unordered (`<ul>`) lists to define individual items within the list.\n",
" \n",
"* `<div>`: The division tag. It is a block-level element used to group other inline or block-level elements.\n",
"- `<div>`: The division tag. It is a block-level element used to group other inline or block-level elements.\n",
" \n",
"* `<a>`: The anchor tag. It is used to define hyperlinks."
"- `<a>`: The anchor tag. It is used to define hyperlinks."
]
},
{

View File

@@ -9,9 +9,9 @@
"We can extract useful features of documents using the [Doctran](https://github.com/psychic-api/doctran) library, which uses OpenAI's function calling feature to extract specific metadata.\n",
"\n",
"Extracting metadata from documents is helpful for a variety of tasks, including:\n",
"* **Classification:** classifying documents into different categories\n",
"* **Data mining:** Extract structured data that can be used for data analysis\n",
"* **Style transfer:** Change the way text is written to more closely match expected user input, improving vector search results"
"- **Classification:** classifying documents into different categories\n",
"- **Data mining:** Extract structured data that can be used for data analysis\n",
"- **Style transfer:** Change the way text is written to more closely match expected user input, improving vector search results"
]
},
{

View File

@@ -28,9 +28,9 @@
"\n",
"To use the wrapper, you must [deploy a model on AzureML](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-foundation-models?view=azureml-api-2#deploying-foundation-models-to-endpoints-for-inferencing) and obtain the following parameters:\n",
"\n",
"* `endpoint_api_key`: Required - The API key provided by the endpoint\n",
"* `endpoint_url`: Required - The REST endpoint url provided by the endpoint\n",
"* `deployment_name`: Not required - The deployment name of the model using the endpoint"
"- `endpoint_api_key`: Required - The API key provided by the endpoint\n",
"- `endpoint_url`: Required - The REST endpoint url provided by the endpoint\n",
"- `deployment_name`: Not required - The deployment name of the model using the endpoint"
]
},
{
@@ -41,10 +41,10 @@
"\n",
"The `content_formatter` parameter is a handler class for transforming the request and response of an AzureML endpoint to match with required schema. Since there are a wide range of models in the model catalog, each of which may process data differently from one another, a `ContentFormatterBase` class is provided to allow users to transform data to their liking. The following content formatters are provided:\n",
"\n",
"* `GPT2ContentFormatter`: Formats request and response data for GPT2\n",
"* `DollyContentFormatter`: Formats request and response data for the Dolly-v2\n",
"* `HFContentFormatter`: Formats request and response data for text-generation Hugging Face models\n",
"* `LLamaContentFormatter`: Formats request and response data for LLaMa2\n",
"- `GPT2ContentFormatter`: Formats request and response data for GPT2\n",
"- `DollyContentFormatter`: Formats request and response data for the Dolly-v2\n",
"- `HFContentFormatter`: Formats request and response data for text-generation Hugging Face models\n",
"- `LLamaContentFormatter`: Formats request and response data for LLaMa2\n",
"\n",
"*Note: `OSSContentFormatter` is being deprecated and replaced with `GPT2ContentFormatter`. The logic is the same but `GPT2ContentFormatter` is a more suitable name. You can still continue to use `OSSContentFormatter` as the changes are backwards compatible.*\n",
"\n",

View File

@@ -19,8 +19,8 @@
"\n",
"This example notebook shows how to wrap Databricks endpoints as LLMs in LangChain.\n",
"It supports two endpoint types:\n",
"* Serving endpoint, recommended for production and development,\n",
"* Cluster driver proxy app, recommended for interactive development."
"- Serving endpoint, recommended for production and development,\n",
"- Cluster driver proxy app, recommended for interactive development."
]
},
{
@@ -159,8 +159,8 @@
"## Wrapping a serving endpoint: Custom model\n",
"\n",
"Prerequisites:\n",
"* An LLM was registered and deployed to [a Databricks serving endpoint](https://docs.databricks.com/machine-learning/model-serving/index.html).\n",
"* You have [\"Can Query\" permission](https://docs.databricks.com/security/auth-authz/access-control/serving-endpoint-acl.html) to the endpoint.\n",
"- An LLM was registered and deployed to [a Databricks serving endpoint](https://docs.databricks.com/machine-learning/model-serving/index.html).\n",
"- You have [\"Can Query\" permission](https://docs.databricks.com/security/auth-authz/access-control/serving-endpoint-acl.html) to the endpoint.\n",
"\n",
"The expected MLflow model signature is:\n",
" * inputs: `[{\"name\": \"prompt\", \"type\": \"string\"}, {\"name\": \"stop\", \"type\": \"list[string]\"}]`\n",
@@ -381,13 +381,13 @@
"## Wrapping a cluster driver proxy app\n",
"\n",
"Prerequisites:\n",
"* An LLM loaded on a Databricks interactive cluster in \"single user\" or \"no isolation shared\" mode.\n",
"* A local HTTP server running on the driver node to serve the model at `\"/\"` using HTTP POST with JSON input/output.\n",
"* It uses a port number between `[3000, 8000]` and listens to the driver IP address or simply `0.0.0.0` instead of localhost only.\n",
"* You have \"Can Attach To\" permission to the cluster.\n",
"- An LLM loaded on a Databricks interactive cluster in \"single user\" or \"no isolation shared\" mode.\n",
"- A local HTTP server running on the driver node to serve the model at `\"/\"` using HTTP POST with JSON input/output.\n",
"- It uses a port number between `[3000, 8000]` and listens to the driver IP address or simply `0.0.0.0` instead of localhost only.\n",
"- You have \"Can Attach To\" permission to the cluster.\n",
"\n",
"The expected server schema (using JSON schema) is:\n",
"* inputs:\n",
"- inputs:\n",
" ```json\n",
" {\"type\": \"object\",\n",
" \"properties\": {\n",
@@ -395,7 +395,7 @@
" \"stop\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}}},\n",
" \"required\": [\"prompt\"]}\n",
" ```\n",
"* outputs: `{\"type\": \"string\"}`\n",
"- outputs: `{\"type\": \"string\"}`\n",
"\n",
"If the server schema is incompatible or you want to insert extra configs, you can use `transform_input_fn` and `transform_output_fn` accordingly.\n",
"\n",

View File

@@ -87,8 +87,8 @@
" \n",
"The [gpt4all page](https://gpt4all.io/index.html) has a useful `Model Explorer` section:\n",
"\n",
"* Select a model of interest\n",
"* Download using the UI and move the `.bin` to the `local_path` (noted below)\n",
"- Select a model of interest\n",
"- Download using the UI and move the `.bin` to the `local_path` (noted below)\n",
"\n",
"For more info, visit https://github.com/nomic-ai/gpt4all.\n",
"\n",

View File

@@ -90,8 +90,8 @@
" return {\"prompt\": \"hello world\"}\n",
"```\n",
"\n",
"* See Modal's [web endpoints](https://modal.com/docs/guide/webhooks#passing-arguments-to-web-endpoints) guide for the basics of setting up an endpoint that fulfils this interface.\n",
"* See Modal's ['Run Falcon-40B with AutoGPTQ'](https://modal.com/docs/guide/ex/falcon_gptq) open-source LLM example as a starting point for your custom LLM!"
"- See Modal's [web endpoints](https://modal.com/docs/guide/webhooks#passing-arguments-to-web-endpoints) guide for the basics of setting up an endpoint that fulfils this interface.\n",
"- See Modal's ['Run Falcon-40B with AutoGPTQ'](https://modal.com/docs/guide/ex/falcon_gptq) open-source LLM example as a starting point for your custom LLM!"
]
},
{

View File

@@ -18,30 +18,30 @@
"\n",
"First, follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance:\n",
"\n",
"* [Download](https://ollama.ai/download)\n",
"* Fetch a model via `ollama pull <model family>`\n",
"* e.g., for `Llama-7b`: `ollama pull llama2` (see full list [here](https://github.com/jmorganca/ollama))\n",
"* This will download the most basic version of the model typically (e.g., smallest # parameters and `q4_0`)\n",
"* On Mac, it will download to \n",
"- [Download](https://ollama.ai/download)\n",
"- Fetch a model via `ollama pull <model family>`\n",
"- e.g., for `Llama-7b`: `ollama pull llama2` (see full list [here](https://github.com/jmorganca/ollama))\n",
"- This will download the most basic version of the model typically (e.g., smallest # parameters and `q4_0`)\n",
"- On Mac, it will download to \n",
"\n",
"`~/.ollama/models/manifests/registry.ollama.ai/library/<model family>/latest`\n",
"\n",
"* And we specify a particular version, e.g., for `ollama pull vicuna:13b-v1.5-16k-q4_0`\n",
"* The file is here with the model version in place of `latest`\n",
"- And we specify a particular version, e.g., for `ollama pull vicuna:13b-v1.5-16k-q4_0`\n",
"- The file is here with the model version in place of `latest`\n",
"\n",
"`~/.ollama/models/manifests/registry.ollama.ai/library/vicuna/13b-v1.5-16k-q4_0`\n",
"\n",
"You can easily access models in a few ways:\n",
"\n",
"1/ if the app is running:\n",
"* All of your local models are automatically served on `localhost:11434`\n",
"* Select your model when setting `llm = Ollama(..., model=\"<model family>:<version>\")`\n",
"* If you set `llm = Ollama(..., model=\"<model family\")` withoout a version it will simply look for `latest`\n",
"- All of your local models are automatically served on `localhost:11434`\n",
"- Select your model when setting `llm = Ollama(..., model=\"<model family>:<version>\")`\n",
"- If you set `llm = Ollama(..., model=\"<model family\")` withoout a version it will simply look for `latest`\n",
"\n",
"2/ if building from source or just running the binary: \n",
"* Then you must run `ollama serve`\n",
"* All of your local models are automatically served on `localhost:11434`\n",
"* Then, select as shown above\n",
"- Then you must run `ollama serve`\n",
"- All of your local models are automatically served on `localhost:11434`\n",
"- Then, select as shown above\n",
"\n",
"\n",
"## Usage\n",

View File

@@ -76,12 +76,12 @@
"to \"PERSON_998 is good.\" \n",
"\n",
"Here's how to handle these changes:\n",
"* Consider these masked phrases just as placeholders, but still refer to\n",
"- Consider these masked phrases just as placeholders, but still refer to\n",
"them in a relevant way when answering.\n",
"* It's possible that different masked terms might mean the same thing.\n",
"- It's possible that different masked terms might mean the same thing.\n",
"Stick with the given term and don't modify it.\n",
"* All masked terms follow the \"TYPE_ID\" pattern.\n",
"* Please don't invent new masked terms. For instance, if you see \"PERSON_998,\"\n",
"- All masked terms follow the \"TYPE_ID\" pattern.\n",
"- Please don't invent new masked terms. For instance, if you see \"PERSON_998,\"\n",
"don't come up with \"PERSON_997\" or \"PERSON_999\" unless they're already in the question.\n",
"\n",
"Conversation History: ```{history}```\n",

View File

@@ -8,10 +8,10 @@
"# vLLM\n",
"\n",
"[vLLM](https://vllm.readthedocs.io/en/latest/index.html) is a fast and easy-to-use library for LLM inference and serving, offering:\n",
"* State-of-the-art serving throughput \n",
"* Efficient management of attention key and value memory with PagedAttention\n",
"* Continuous batching of incoming requests\n",
"* Optimized CUDA kernels\n",
"- State-of-the-art serving throughput \n",
"- Efficient management of attention key and value memory with PagedAttention\n",
"- Continuous batching of incoming requests\n",
"- Optimized CUDA kernels\n",
"\n",
"This notebooks goes over how to use a LLM with langchain and vLLM.\n",
"\n",

View File

@@ -10,8 +10,8 @@
"\n",
"This notebook covers:\n",
"\n",
"* A simple example showing what `XataChatMessageHistory` does.\n",
"* A more complex example using a REACT agent that answer questions based on a knowledge based or documentation (stored in Xata as a vector store) and also having a long-term searchable history of its past messages (stored in Xata as a memory store)"
"- A simple example showing what `XataChatMessageHistory` does.\n",
"- A more complex example using a REACT agent that answer questions based on a knowledge based or documentation (stored in Xata as a vector store) and also having a long-term searchable history of its past messages (stored in Xata as a memory store)"
]
},
{
@@ -137,8 +137,8 @@
"source": [
"To store the documents that the chatbot will search for answers, add a table named `docs` to your `langchain` database using the Xata UI, and add the following columns:\n",
"\n",
"* `content` of type \"Text\". This is used to store the `Document.pageContent` values.\n",
"* `embedding` of type \"Vector\". Use the dimension used by the model you plan to use. In this notebook we use OpenAI embeddings, which have 1536 dimensions."
"- `content` of type \"Text\". This is used to store the `Document.pageContent` values.\n",
"- `embedding` of type \"Vector\". Use the dimension used by the model you plan to use. In this notebook we use OpenAI embeddings, which have 1536 dimensions."
]
},
{

View File

@@ -10,10 +10,10 @@
">[Rebuff](https://docs.rebuff.ai/) is a self-hardening prompt injection detector.\n",
"It is designed to protect AI applications from prompt injection (PI) attacks through a multi-stage defense.\n",
"\n",
"* [Homepage](https://rebuff.ai)\n",
"* [Playground](https://playground.rebuff.ai)\n",
"* [Docs](https://docs.rebuff.ai)\n",
"* [GitHub Repository](https://github.com/woop/rebuff)"
"- [Homepage](https://rebuff.ai)\n",
"- [Playground](https://playground.rebuff.ai)\n",
"- [Docs](https://docs.rebuff.ai)\n",
"- [GitHub Repository](https://github.com/woop/rebuff)"
]
},
{

View File

@@ -39,9 +39,9 @@
"metadata": {},
"source": [
"Make sure to set the required API keys and config required to send telemetry to WhyLabs:\n",
"* WhyLabs API Key: https://whylabs.ai/whylabs-free-sign-up\n",
"* Org and Dataset [https://docs.whylabs.ai/docs/whylabs-onboarding](https://docs.whylabs.ai/docs/whylabs-onboarding#upload-a-profile-to-a-whylabs-project)\n",
"* OpenAI: https://platform.openai.com/account/api-keys\n",
"- WhyLabs API Key: https://whylabs.ai/whylabs-free-sign-up\n",
"- Org and Dataset [https://docs.whylabs.ai/docs/whylabs-onboarding](https://docs.whylabs.ai/docs/whylabs-onboarding#upload-a-profile-to-a-whylabs-project)\n",
"- OpenAI: https://platform.openai.com/account/api-keys\n",
"\n",
"Then you can set them like this:\n",
"\n",

View File

@@ -45,7 +45,7 @@
"\n",
"`EmbedchainRetriever` has a static `.create()` factory method that takes the following arguments:\n",
"\n",
"* `yaml_path: string` optional -- Path to the YAML configuration file. If not provided, a default configuration is used. You can browse the [docs](https://docs.embedchain.ai/) to explore various customization options."
"- `yaml_path: string` optional -- Path to the YAML configuration file. If not provided, a default configuration is used. You can browse the [docs](https://docs.embedchain.ai/) to explore various customization options."
]
},
{

View File

@@ -32,8 +32,8 @@
"metadata": {},
"source": [
"You can obtain your folder and document id from the URL:\n",
"* Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",
"* Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is `\"1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw\"`\n",
"- Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",
"- Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is `\"1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw\"`\n",
"\n",
"The special value `root` is for your personal home."
]

View File

@@ -25,9 +25,9 @@
"\n",
"`KayAiRetriever` has a static `.create()` factory method that takes the following arguments:\n",
"\n",
"* `dataset_id: string` required -- A Kay dataset id. This is a collection of data about a particular entity such as companies, people, or places. For example, try `\"company\"` \n",
"* `data_type: List[string]` optional -- This is a category within a dataset based on its origin or format, such as SEC Filings, Press Releases, or Reports within the “company” dataset. For example, try [\"10-K\", \"10-Q\", \"PressRelease\"] under the “company” dataset. If left empty, Kay will retrieve the most relevant context across all types.\n",
"* `num_contexts: int` optional, defaults to 6 -- The number of document chunks to retrieve on each call to `get_relevant_documents()`"
"- `dataset_id: string` required -- A Kay dataset id. This is a collection of data about a particular entity such as companies, people, or places. For example, try `\"company\"` \n",
"- `data_type: List[string]` optional -- This is a category within a dataset based on its origin or format, such as SEC Filings, Press Releases, or Reports within the “company” dataset. For example, try [\"10-K\", \"10-Q\", \"PressRelease\"] under the “company” dataset. If left empty, Kay will retrieve the most relevant context across all types.\n",
"- `num_contexts: int` optional, defaults to 6 -- The number of document chunks to retrieve on each call to `get_relevant_documents()`"
]
},
{

View File

@@ -11,7 +11,7 @@
"\n",
"The [embeddings](https://docs.nlpcloud.com/#embeddings) endpoint offers the following model:\n",
"\n",
"* `paraphrase-multilingual-mpnet-base-v2`: Paraphrase Multilingual MPNet Base V2 is a very fast model based on Sentence Transformers that is perfectly suited for embeddings extraction in more than 50 languages (see the full list here)."
"- `paraphrase-multilingual-mpnet-base-v2`: Paraphrase Multilingual MPNet Base V2 is a very fast model based on Sentence Transformers that is perfectly suited for embeddings extraction in more than 50 languages (see the full list here)."
]
},
{

View File

@@ -69,11 +69,11 @@
"### 2. Create a Github App\n",
"\n",
"[Follow the instructions here](https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app) to create and register a Github app. Make sure your app has the following [repository permissions:](https://docs.github.com/en/rest/overview/permissions-required-for-github-apps?apiVersion=2022-11-28)\n",
"* Commit statuses (read only)\n",
"* Contents (read and write)\n",
"* Issues (read and write)\n",
"* Metadata (read only)\n",
"* Pull requests (read and write)\n",
"- Commit statuses (read only)\n",
"- Contents (read and write)\n",
"- Issues (read and write)\n",
"- Metadata (read only)\n",
"- Pull requests (read and write)\n",
"\n",
"\n",
"Once the app has been registered, you must give your app permission to access each of the repositories you whish it to act upon. Use the App settings on [github.com here](https://github.com/settings/installations).\n",
@@ -82,11 +82,11 @@
"\n",
"Before initializing your agent, the following environmental variables need to be set:\n",
"\n",
"* **GITHUB_APP_ID**- A six digit number found in your app's general settings\n",
"* **GITHUB_APP_PRIVATE_KEY**- The location of your app's private key .pem file, or the full text of that file as a string.\n",
"* **GITHUB_REPOSITORY**- The name of the Github repository you want your bot to act upon. Must follow the format {username}/{repo-name}. *Make sure the app has been added to this repository first!*\n",
"* Optional: **GITHUB_BRANCH**- The branch where the bot will make its commits. Defaults to `repo.default_branch`.\n",
"* Optional: **GITHUB_BASE_BRANCH**- The base branch of your repo upon which PRs will based from. Defaults to `repo.default_branch`.\n"
"- **GITHUB_APP_ID**- A six digit number found in your app's general settings\n",
"- **GITHUB_APP_PRIVATE_KEY**- The location of your app's private key .pem file, or the full text of that file as a string.\n",
"- **GITHUB_REPOSITORY**- The name of the Github repository you want your bot to act upon. Must follow the format {username}/{repo-name}. *Make sure the app has been added to this repository first!*\n",
"- Optional: **GITHUB_BRANCH**- The branch where the bot will make its commits. Defaults to `repo.default_branch`.\n",
"- Optional: **GITHUB_BASE_BRANCH**- The base branch of your repo upon which PRs will based from. Defaults to `repo.default_branch`.\n"
]
},
{
@@ -266,9 +266,9 @@
"\n",
"\n",
"### Input data and LangSmith Trace\n",
"* LangSmith trace for this run: https://smith.langchain.com/public/fee6643c-b214-42d0-967b-d24dcdd690fe/r\n",
"* Input issue: https://github.com/KastanDay/ML4Bio/issues/33\n",
"* Final PR created by bot: https://github.com/KastanDay/ML4Bio/pull/40"
"- LangSmith trace for this run: https://smith.langchain.com/public/fee6643c-b214-42d0-967b-d24dcdd690fe/r\n",
"- Input issue: https://github.com/KastanDay/ML4Bio/issues/33\n",
"- Final PR created by bot: https://github.com/KastanDay/ML4Bio/pull/40"
]
},
{

View File

@@ -69,19 +69,19 @@
"### 2. Create a Gitlab personal access token\n",
"\n",
"[Follow the instructions here](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) to create a Gitlab personal access token. Make sure your app has the following repository permissions:\n",
"* read_api\n",
"* read_repository\n",
"* write_repository\n",
"- read_api\n",
"- read_repository\n",
"- write_repository\n",
"\n",
"### 3. Set Environmental Variables\n",
"\n",
"Before initializing your agent, the following environmental variables need to be set:\n",
"\n",
"* **GITLAB_URL** - The URL hosted Gitlab. Defaults to \"https://gitlab.com\". \n",
"* **GITLAB_PERSONAL_ACCESS_TOKEN**- The personal access token you created in the last step\n",
"* **GITLAB_REPOSITORY**- The name of the Gitlab repository you want your bot to act upon. Must follow the format {username}/{repo-name}.\n",
"* **GITLAB_BRANCH**- The branch where the bot will make its commits. Defaults to 'main.'\n",
"* **GITLAB_BASE_BRANCH**- The base branch of your repo, usually either 'main' or 'master.' This is where pull requests will base from. Defaults to 'main.'\n"
"- **GITLAB_URL** - The URL hosted Gitlab. Defaults to \"https://gitlab.com\". \n",
"- **GITLAB_PERSONAL_ACCESS_TOKEN**- The personal access token you created in the last step\n",
"- **GITLAB_REPOSITORY**- The name of the Gitlab repository you want your bot to act upon. Must follow the format {username}/{repo-name}.\n",
"- **GITLAB_BRANCH**- The branch where the bot will make its commits. Defaults to 'main.'\n",
"- **GITLAB_BASE_BRANCH**- The base branch of your repo, usually either 'main' or 'master.' This is where pull requests will base from. Defaults to 'main.'\n"
]
},
{

View File

@@ -38,8 +38,8 @@
"metadata": {},
"source": [
"You can obtain your folder and document id from the URL:\n",
"* Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",
"* Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is `\"1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw\"`\n",
"- Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",
"- Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is `\"1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw\"`\n",
"\n",
"The special value `root` is for your personal home."
]

View File

@@ -38,8 +38,8 @@
"metadata": {},
"source": [
"You can obtain your folder and document id from the URL:\n",
"* Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",
"* Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is `\"1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw\"`\n",
"- Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",
"- Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is `\"1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw\"`\n",
"\n",
"The special value `root` is for your personal home."
]

View File

@@ -19,24 +19,24 @@
"\n",
"Redis uses compressed, inverted indexes for fast indexing with a low memory footprint. It also supports a number of advanced features such as:\n",
"\n",
"* Indexing of multiple fields in Redis hashes and JSON\n",
"* Vector similarity search (with HNSW (ANN) or FLAT (KNN))\n",
"* Vector Range Search (e.g. find all vectors within a radius of a query vector)\n",
"* Incremental indexing without performance loss\n",
"* Document ranking (using [tf-idf](https://en.wikipedia.org/wiki/Tf%E2%80%93idf), with optional user-provided weights)\n",
"* Field weighting\n",
"* Complex boolean queries with AND, OR, and NOT operators\n",
"* Prefix matching, fuzzy matching, and exact-phrase queries\n",
"* Support for [double-metaphone phonetic matching](https://redis.io/docs/stack/search/reference/phonetic_matching/)\n",
"* Auto-complete suggestions (with fuzzy prefix suggestions)\n",
"* Stemming-based query expansion in [many languages](https://redis.io/docs/stack/search/reference/stemming/) (using [Snowball](http://snowballstem.org/))\n",
"* Support for Chinese-language tokenization and querying (using [Friso](https://github.com/lionsoul2014/friso))\n",
"* Numeric filters and ranges\n",
"* Geospatial searches using [Redis geospatial indexing](/commands/georadius)\n",
"* A powerful aggregations engine\n",
"* Supports for all utf-8 encoded text\n",
"* Retrieve full documents, selected fields, or only the document IDs\n",
"* Sorting results (for example, by creation date)\n",
"- Indexing of multiple fields in Redis hashes and JSON\n",
"- Vector similarity search (with HNSW (ANN) or FLAT (KNN))\n",
"- Vector Range Search (e.g. find all vectors within a radius of a query vector)\n",
"- Incremental indexing without performance loss\n",
"- Document ranking (using [tf-idf](https://en.wikipedia.org/wiki/Tf%E2%80%93idf), with optional user-provided weights)\n",
"- Field weighting\n",
"- Complex boolean queries with AND, OR, and NOT operators\n",
"- Prefix matching, fuzzy matching, and exact-phrase queries\n",
"- Support for [double-metaphone phonetic matching](https://redis.io/docs/stack/search/reference/phonetic_matching/)\n",
"- Auto-complete suggestions (with fuzzy prefix suggestions)\n",
"- Stemming-based query expansion in [many languages](https://redis.io/docs/stack/search/reference/stemming/) (using [Snowball](http://snowballstem.org/))\n",
"- Support for Chinese-language tokenization and querying (using [Friso](https://github.com/lionsoul2014/friso))\n",
"- Numeric filters and ranges\n",
"- Geospatial searches using [Redis geospatial indexing](/commands/georadius)\n",
"- A powerful aggregations engine\n",
"- Supports for all utf-8 encoded text\n",
"- Retrieve full documents, selected fields, or only the document IDs\n",
"- Sorting results (for example, by creation date)\n",
"\n",
"\n",
"\n",

View File

@@ -68,9 +68,9 @@
"In this example, we assume that you've created an account and a corpus, and added your VECTARA_CUSTOMER_ID, VECTARA_CORPUS_ID and VECTARA_API_KEY (created with permissions for both indexing and query) as environment variables.\n",
"\n",
"The corpus has 3 fields defined as metadata for filtering:\n",
"* url: a string field containing the source URL of the document (where relevant)\n",
"* speech: a string field containing the name of the speech\n",
"* author: the name of the author\n",
"- url: a string field containing the source URL of the document (where relevant)\n",
"- speech: a string field containing the name of the speech\n",
"- author: the name of the author\n",
"\n",
"Let's start by ingesting 3 documents into the corpus:\n",
"1. The State of the Union speech from 2022, available in the LangChain repository as a text file\n",

View File

@@ -28,10 +28,10 @@
"In the [Xata UI](https://app.xata.io) create a new database. You can name it whatever you want, in this notepad we'll use `langchain`.\n",
"Create a table, again you can name it anything, but we will use `vectors`. Add the following columns via the UI:\n",
"\n",
"* `content` of type \"Text\". This is used to store the `Document.pageContent` values.\n",
"* `embedding` of type \"Vector\". Use the dimension used by the model you plan to use. In this notebook we use OpenAI embeddings, which have 1536 dimensions.\n",
"* `search` of type \"Text\". This is used as a metadata column by this example.\n",
"* any other columns you want to use as metadata. They are populated from the `Document.metadata` object. For example, if in the `Document.metadata` object you have a `title` property, you can create a `title` column in the table and it will be populated.\n"
"- `content` of type \"Text\". This is used to store the `Document.pageContent` values.\n",
"- `embedding` of type \"Vector\". Use the dimension used by the model you plan to use. In this notebook we use OpenAI embeddings, which have 1536 dimensions.\n",
"- `search` of type \"Text\". This is used as a metadata column by this example.\n",
"- any other columns you want to use as metadata. They are populated from the `Document.metadata` object. For example, if in the `Document.metadata` object you have a `title` property, you can create a `title` column in the table and it will be populated.\n"
]
},
{

View File

@@ -30,14 +30,14 @@
"## LCEL\n",
"\n",
"The most visible part of LCEL is that it provides an intuitive and readable syntax for composition. But more importantly, it also provides first-class support for:\n",
"* [streaming](/docs/expression_language/interface#stream),\n",
"* [async calls](/docs/expression_language/interface#async-stream),\n",
"* [batching](/docs/expression_language/interface#batch),\n",
"* [parallelization](/docs/expression_language/interface#parallelism),\n",
"* retries,\n",
"* [fallbacks](/docs/expression_language/how_to/fallbacks),\n",
"* tracing,\n",
"* [and more.](/docs/expression_language/why)\n",
"- [streaming](/docs/expression_language/interface#stream),\n",
"- [async calls](/docs/expression_language/interface#async-stream),\n",
"- [batching](/docs/expression_language/interface#batch),\n",
"- [parallelization](/docs/expression_language/interface#parallelism),\n",
"- retries,\n",
"- [fallbacks](/docs/expression_language/how_to/fallbacks),\n",
"- tracing,\n",
"- [and more.](/docs/expression_language/why)\n",
"\n",
"As a simple and common example, we can see what it's like to combine a prompt, model and output parser:"
]

View File

@@ -11,9 +11,9 @@
"\n",
"The indexing API lets you load and keep in sync documents from any source into a vector store. Specifically, it helps:\n",
"\n",
"* Avoid writing duplicated content into the vector store\n",
"* Avoid re-writing unchanged content\n",
"* Avoid re-computing embeddings over unchanged content\n",
"- Avoid writing duplicated content into the vector store\n",
"- Avoid re-writing unchanged content\n",
"- Avoid re-computing embeddings over unchanged content\n",
"\n",
"All of which should save you time and money, as well as improve your vector search results.\n",
"\n",
@@ -45,13 +45,13 @@
"\n",
"`incremental` and `full` offer the following automated clean up:\n",
"\n",
"* If the content of the source document or derived documents has **changed**, both `incremental` or `full` modes will clean up (delete) previous versions of the content.\n",
"* If the source document has been **deleted** (meaning it is not included in the documents currently being indexed), the `full` cleanup mode will delete it from the vector store correctly, but the `incremental` mode will not.\n",
"- If the content of the source document or derived documents has **changed**, both `incremental` or `full` modes will clean up (delete) previous versions of the content.\n",
"- If the source document has been **deleted** (meaning it is not included in the documents currently being indexed), the `full` cleanup mode will delete it from the vector store correctly, but the `incremental` mode will not.\n",
"\n",
"When content is mutated (e.g., the source PDF file was revised) there will be a period of time during indexing when both the new and old versions may be returned to the user. This happens after the new content was written, but before the old version was deleted.\n",
"\n",
"* `incremental` indexing minimizes this period of time as it is able to do clean up continuously, as it writes.\n",
"* `full` mode does the clean up after all batches have been written.\n",
"- `incremental` indexing minimizes this period of time as it is able to do clean up continuously, as it writes.\n",
"- `full` mode does the clean up after all batches have been written.\n",
"\n",
"## Requirements\n",
"\n",

View File

@@ -9,10 +9,10 @@
"\n",
"Given a query, this retriever will: \n",
"\n",
"* Formulate a set of relate Google searches\n",
"* Search for each \n",
"* Load all the resulting URLs\n",
"* Then embed and perform similarity search with the query on the consolidate page content"
"- Formulate a set of relate Google searches\n",
"- Search for each \n",
"- Load all the resulting URLs\n",
"- Then embed and perform similarity search with the query on the consolidate page content"
]
},
{

View File

@@ -40,7 +40,7 @@
"source": [
"In this example, we will write a custom memory class that uses spaCy to extract entities and save information about them in a simple hash table. Then, during the conversation, we will look at the input text, extract any entities, and put any information about them into the context.\n",
"\n",
"* Please note that this implementation is pretty simple and brittle and probably not useful in a production setting. Its purpose is to showcase that you can add custom memory implementations.\n",
"- Please note that this implementation is pretty simple and brittle and probably not useful in a production setting. Its purpose is to showcase that you can add custom memory implementations.\n",
"\n",
"For this, we will need spaCy."
]

View File

@@ -232,8 +232,8 @@
"source": [
"### Prerequisites\n",
"\n",
"* Tecton Deployment (sign up at [https://tecton.ai](https://tecton.ai))\n",
"* `TECTON_API_KEY` environment variable set to a valid Service Account key"
"- Tecton Deployment (sign up at [https://tecton.ai](https://tecton.ai))\n",
"- `TECTON_API_KEY` environment variable set to a valid Service Account key"
]
},
{
@@ -615,9 +615,9 @@
"source": [
"### Prerequisites\n",
"\n",
"* Create feature store with online materialization using instructions here [Enable online materialization and run online inference](https://github.com/Azure/azureml-examples/blob/featurestore/online/sdk/python/featurestore_sample/notebooks/sdk_only/5.%20Enable%20online%20store%20and%20run%20online%20inference.ipynb).\n",
"- Create feature store with online materialization using instructions here [Enable online materialization and run online inference](https://github.com/Azure/azureml-examples/blob/featurestore/online/sdk/python/featurestore_sample/notebooks/sdk_only/5.%20Enable%20online%20store%20and%20run%20online%20inference.ipynb).\n",
"\n",
"* A successfully created feature store by following the instructions should have an `account` featureset with version as `1`. It will have `accountID` as index column with features `accountAge`, `accountCountry`, `numPaymentRejects1dPerUser`."
"- A successfully created feature store by following the instructions should have an `account` featureset with version as `1`. It will have `accountID` as index column with features `accountAge`, `accountCountry`, `numPaymentRejects1dPerUser`."
]
},
{
@@ -627,9 +627,9 @@
"source": [
"### Prompts\n",
"\n",
"* Here we will set up a custom AzureMLFeatureStorePromptTemplate. This prompt template will take in an `account_id` and optional `query`. It then fetches feature values from feature store and format those features into the output prompt. Note that the required input to this prompt template is just `account_id`, since that is the only user defined piece (all other variables are looked up inside the prompt template).\n",
"- Here we will set up a custom AzureMLFeatureStorePromptTemplate. This prompt template will take in an `account_id` and optional `query`. It then fetches feature values from feature store and format those features into the output prompt. Note that the required input to this prompt template is just `account_id`, since that is the only user defined piece (all other variables are looked up inside the prompt template).\n",
"\n",
"* Also note that this is a bootstrap example to showcase how LLM applications can leverage AzureML managed feature store. Developers are welcome to improve the prompt template further to suit their needs."
"- Also note that this is a bootstrap example to showcase how LLM applications can leverage AzureML managed feature store. Developers are welcome to improve the prompt template further to suit their needs."
]
},
{

View File

@@ -31,8 +31,8 @@
"\n",
"There are two primary ways to interface LLMs with external APIs:\n",
" \n",
"* `Functions`: For example, [OpenAI functions](https://platform.openai.com/docs/guides/gpt/function-calling) is one popular means of doing this.\n",
"* `LLM-generated interface`: Use an LLM with access to API documentation to create an interface.\n",
"- `Functions`: For example, [OpenAI functions](https://platform.openai.com/docs/guides/gpt/function-calling) is one popular means of doing this.\n",
"- `LLM-generated interface`: Use an LLM with access to API documentation to create an interface.\n",
"\n",
"![Image description](../../static/img/api_use_case.png)"
]
@@ -54,8 +54,8 @@
"\n",
"Other options include:\n",
"\n",
"* [Speak](https://api.speak.com/openapi.yaml) for translation\n",
"* [XKCD](https://gist.githubusercontent.com/roaldnefs/053e505b2b7a807290908fe9aa3e1f00/raw/0a212622ebfef501163f91e23803552411ed00e4/openapi.yaml) for comics\n",
"- [Speak](https://api.speak.com/openapi.yaml) for translation\n",
"- [XKCD](https://gist.githubusercontent.com/roaldnefs/053e505b2b7a807290908fe9aa3e1f00/raw/0a212622ebfef501163f91e23803552411ed00e4/openapi.yaml) for comics\n",
"\n",
"We can supply the specification to `get_openapi_chain` directly in order to query the API with OpenAI functions:"
]
@@ -159,20 +159,20 @@
"\n",
"Let's look at the [LangSmith trace](https://smith.langchain.com/public/76a58b85-193f-4eb7-ba40-747f0d5dd56e/r):\n",
"\n",
"* See [here](https://github.com/langchain-ai/langchain/blob/7fc07ba5df99b9fa8bef837b0fafa220bc5c932c/libs/langchain/langchain/chains/openai_functions/openapi.py#L279C9-L279C19) that we call the OpenAI LLM with the provided API spec:\n",
"- See [here](https://github.com/langchain-ai/langchain/blob/7fc07ba5df99b9fa8bef837b0fafa220bc5c932c/libs/langchain/langchain/chains/openai_functions/openapi.py#L279C9-L279C19) that we call the OpenAI LLM with the provided API spec:\n",
"\n",
"```\n",
"https://www.klarna.com/us/shopping/public/openai/v0/api-docs/\n",
"```\n",
"\n",
"* The prompt then tells the LLM to use the API spec with input question:\n",
"- The prompt then tells the LLM to use the API spec with input question:\n",
"\n",
"```\n",
"Use the provided APIs to respond to this user query:\n",
"What are some options for a men's large blue button down shirt\n",
"```\n",
"\n",
"* The LLM returns the parameters for the function call `productsUsingGET`, which is [specified in the provided API spec](https://www.klarna.com/us/shopping/public/openai/v0/api-docs/):\n",
"- The LLM returns the parameters for the function call `productsUsingGET`, which is [specified in the provided API spec](https://www.klarna.com/us/shopping/public/openai/v0/api-docs/):\n",
"```\n",
"function_call:\n",
" name: productsUsingGET\n",
@@ -190,7 +190,7 @@
" \n",
"![Image description](../../static/img/api_function_call.png)\n",
" \n",
"* This `Dict` above split and the [API is called here](https://github.com/langchain-ai/langchain/blob/7fc07ba5df99b9fa8bef837b0fafa220bc5c932c/libs/langchain/langchain/chains/openai_functions/openapi.py#L215)."
"- This `Dict` above split and the [API is called here](https://github.com/langchain-ai/langchain/blob/7fc07ba5df99b9fa8bef837b0fafa220bc5c932c/libs/langchain/langchain/chains/openai_functions/openapi.py#L215)."
]
},
{
@@ -286,17 +286,17 @@
"source": [
"Under the hood, we do two things:\n",
" \n",
"* `api_request_chain`: Generate an API URL based on the input question and the api_docs\n",
"* `api_answer_chain`: generate a final answer based on the API response\n",
"- `api_request_chain`: Generate an API URL based on the input question and the api_docs\n",
"- `api_answer_chain`: generate a final answer based on the API response\n",
"\n",
"We can look at the [LangSmith trace](https://smith.langchain.com/public/1e0d18ca-0d76-444c-97df-a939a6a815a7/r) to inspect this:\n",
"\n",
"* The `api_request_chain` produces the API url from our question and the API documentation:\n",
"- The `api_request_chain` produces the API url from our question and the API documentation:\n",
"\n",
"![Image description](../../static/img/api_chain.png)\n",
"\n",
"* [Here](https://github.com/langchain-ai/langchain/blob/bbd22b9b761389a5e40fc45b0570e1830aabb707/libs/langchain/langchain/chains/api/base.py#L82) we make the API request with the API url.\n",
"* The `api_answer_chain` takes the response from the API and provides us with a natural language response:\n",
"- [Here](https://github.com/langchain-ai/langchain/blob/bbd22b9b761389a5e40fc45b0570e1830aabb707/libs/langchain/langchain/chains/api/base.py#L82) we make the API request with the API url.\n",
"- The `api_answer_chain` takes the response from the API and provides us with a natural language response:\n",
"\n",
"![Image description](../../static/img/api_chain_response.png)"
]

View File

@@ -42,10 +42,10 @@
"\n",
"The chat model interface is based around messages rather than raw text. Several components are important to consider for chat:\n",
"\n",
"* `chat model`: See [here](/docs/integrations/chat) for a list of chat model integrations and [here](/docs/modules/model_io/chat) for documentation on the chat model interface in LangChain. You can use `LLMs` (see [here](/docs/modules/model_io/llms)) for chatbots as well, but chat models have a more conversational tone and natively support a message interface.\n",
"* `prompt template`: Prompt templates make it easy to assemble prompts that combine default messages, user input, chat history, and (optionally) additional retrieved context.\n",
"* `memory`: [See here](/docs/modules/memory/) for in-depth documentation on memory types\n",
"* `retriever` (optional): [See here](/docs/modules/data_connection/retrievers) for in-depth documentation on retrieval systems. These are useful if you want to build a chatbot with domain-specific knowledge.\n",
"- `chat model`: See [here](/docs/integrations/chat) for a list of chat model integrations and [here](/docs/modules/model_io/chat) for documentation on the chat model interface in LangChain. You can use `LLMs` (see [here](/docs/modules/model_io/llms)) for chatbots as well, but chat models have a more conversational tone and natively support a message interface.\n",
"- `prompt template`: Prompt templates make it easy to assemble prompts that combine default messages, user input, chat history, and (optionally) additional retrieved context.\n",
"- `memory`: [See here](/docs/modules/memory/) for in-depth documentation on memory types\n",
"- `retriever` (optional): [See here](/docs/modules/data_connection/retrievers) for in-depth documentation on retrieval systems. These are useful if you want to build a chatbot with domain-specific knowledge.\n",
"\n",
"## Quickstart\n",
"\n",
@@ -203,8 +203,8 @@
"## Memory \n",
"\n",
"As we mentioned above, the core component of chatbots is the memory system. One of the simplest and most commonly used forms of memory is `ConversationBufferMemory`:\n",
"* This memory allows for storing of messages in a `buffer`\n",
"* When called in a chain, it returns all of the messages it has stored\n",
"- This memory allows for storing of messages in a `buffer`\n",
"- When called in a chain, it returns all of the messages it has stored\n",
"\n",
"LangChain comes with many other types of memory, too. [See here](/docs/modules/memory/) for in-depth documentation on memory types.\n",
"\n",
@@ -710,7 +710,7 @@
"\n",
"### Going deeper \n",
"\n",
"* Agents, such as the [conversational retrieval agent](/docs/use_cases/question_answering/conversational_retrieval_agents), can be used for retrieval when necessary while also holding a conversation.\n"
"- Agents, such as the [conversational retrieval agent](/docs/use_cases/question_answering/conversational_retrieval_agents), can be used for retrieval when necessary while also holding a conversation.\n"
]
},
{

View File

@@ -503,8 +503,8 @@
"\n",
"And, we need to do a bit more work:\n",
"\n",
"* Define a class that holds multiple instances of `Person`\n",
"* Explicitly parse the output of the LLM to the Pydantic class\n",
"- Define a class that holds multiple instances of `Person`\n",
"- Explicitly parse the output of the LLM to the Pydantic class\n",
"\n",
"We can see this for other cases, too."
]
@@ -581,11 +581,11 @@
"\n",
"### Going deeper\n",
"\n",
"* The [output parser](/docs/modules/model_io/output_parsers/) documentation includes various parser examples for specific types (e.g., lists, datetime, enum, etc). \n",
"* The experimental [Anthropic function calling](https://python.langchain.com/docs/integrations/chat/anthropic_functions) support provides similar functionality to Anthropic chat models.\n",
"* [LlamaCPP](https://python.langchain.com/docs/integrations/llms/llamacpp#grammars) natively supports constrained decoding using custom grammars, making it easy to output structured content using local LLMs \n",
"* [JSONFormer](/docs/integrations/llms/jsonformer_experimental) offers another way for structured decoding of a subset of the JSON Schema.\n",
"* [Kor](https://eyurtsev.github.io/kor/) is another library for extraction where schema and examples can be provided to the LLM."
"- The [output parser](/docs/modules/model_io/output_parsers/) documentation includes various parser examples for specific types (e.g., lists, datetime, enum, etc). \n",
"- The experimental [Anthropic function calling](https://python.langchain.com/docs/integrations/chat/anthropic_functions) support provides similar functionality to Anthropic chat models.\n",
"- [LlamaCPP](https://python.langchain.com/docs/integrations/llms/llamacpp#grammars) natively supports constrained decoding using custom grammars, making it easy to output structured content using local LLMs \n",
"- [JSONFormer](/docs/integrations/llms/jsonformer_experimental) offers another way for structured decoding of a subset of the JSON Schema.\n",
"- [Kor](https://eyurtsev.github.io/kor/) is another library for extraction where schema and examples can be provided to the LLM."
]
},
{

View File

@@ -19,10 +19,10 @@
"\n",
"This combination allows for use cases such as:\n",
"\n",
"* Building knowledge graphs from textual documents, websites, or social media feeds.\n",
"* Generating recommendations based on semantic relationships in the data.\n",
"* Creating advanced search features that understand the relationships between entities.\n",
"* Building analytics dashboards that allow users to explore the hidden relationships in data.\n",
"- Building knowledge graphs from textual documents, websites, or social media feeds.\n",
"- Generating recommendations based on semantic relationships in the data.\n",
"- Creating advanced search features that understand the relationships between entities.\n",
"- Building analytics dashboards that allow users to explore the hidden relationships in data.\n",
"\n",
"## Overview\n",
"\n",

View File

@@ -69,10 +69,10 @@
" \n",
"Follow [installation steps](https://database.guide/2-sample-databases-sqlite/) to create `Chinook.db` in the same directory as this notebook:\n",
"\n",
"* Save [this file](https://raw.githubusercontent.com/lerocha/chinook-database/master/ChinookDatabase/DataSources/Chinook_Sqlite.sql) to the directory as `Chinook_Sqlite.sql`\n",
"* Run `sqlite3 Chinook.db`\n",
"* Run `.read Chinook_Sqlite.sql`\n",
"* Test `SELECT * FROM Artist LIMIT 10;`\n",
"- Save [this file](https://raw.githubusercontent.com/lerocha/chinook-database/master/ChinookDatabase/DataSources/Chinook_Sqlite.sql) to the directory as `Chinook_Sqlite.sql`\n",
"- Run `sqlite3 Chinook.db`\n",
"- Run `.read Chinook_Sqlite.sql`\n",
"- Test `SELECT * FROM Artist LIMIT 10;`\n",
"\n",
"Now, `Chinhook.db` is in our directory.\n",
"\n",
@@ -236,8 +236,8 @@
"\n",
"[Some papers](https://arxiv.org/pdf/2204.00498.pdf) have reported good performance when prompting with:\n",
" \n",
"* A `CREATE TABLE` description for each table, which include column names, their types, etc\n",
"* Followed by three example rows in a `SELECT` statement\n",
"- A `CREATE TABLE` description for each table, which include column names, their types, etc\n",
"- Followed by three example rows in a `SELECT` statement\n",
"\n",
"`create_sql_query_chain` adopts this the best practice (see more in this [blog](https://blog.langchain.dev/llms-and-sql/)). \n",
"![sql_usecase.png](../../../static/img/create_sql_query_chain.png)\n",
@@ -373,8 +373,8 @@
"\n",
"**Be careful** while using this approach as it is susceptible to `SQL Injection`:\n",
"\n",
"* The chain is executing queries that are created by an LLM, and weren't validated\n",
"* e.g. records may be created, modified or deleted unintentionally_\n",
"- The chain is executing queries that are created by an LLM, and weren't validated\n",
"- e.g. records may be created, modified or deleted unintentionally_\n",
"\n",
"This is why we see the `SQLDatabaseChain` is inside `langchain_experimental`."
]
@@ -389,13 +389,13 @@
"\n",
"We can use the [LangSmith trace](https://smith.langchain.com/public/7f202a0c-1e35-42d6-a84a-6c2a58f697ef/r) to see what is happening under the hood:\n",
"\n",
"* As discussed above, first we create the query:\n",
"- As discussed above, first we create the query:\n",
"\n",
"```\n",
"text: ' SELECT COUNT(*) FROM \"Employee\";'\n",
"```\n",
"\n",
"* Then, it executes the query and passes the results to an LLM for synthesis.\n",
"- Then, it executes the query and passes the results to an LLM for synthesis.\n",
"\n",
"![sql_usecase.png](../../../static/img/sqldbchain_trace.png)\n",
"\n",
@@ -505,10 +505,10 @@
"\n",
"This agent contains the `SQLDatabaseToolkit` which contains tools to: \n",
"\n",
"* Create and execute queries\n",
"* Check query syntax\n",
"* Retrieve table descriptions\n",
"* ... and more"
"- Create and execute queries\n",
"- Check query syntax\n",
"- Retrieve table descriptions\n",
"- ... and more"
]
},
{
@@ -640,9 +640,9 @@
"source": [
"Looking at the [LangSmith trace](https://smith.langchain.com/public/a86dbe17-5782-4020-bce6-2de85343168a/r), we can see:\n",
"\n",
"* The agent is using a ReAct style prompt\n",
"* First, it will look at the tables: `Action: sql_db_list_tables` using tool `sql_db_list_tables`\n",
"* Given the tables as an observation, it `thinks` and then determinates the next `action`:\n",
"- The agent is using a ReAct style prompt\n",
"- First, it will look at the tables: `Action: sql_db_list_tables` using tool `sql_db_list_tables`\n",
"- Given the tables as an observation, it `thinks` and then determinates the next `action`:\n",
"\n",
"```\n",
"Observation: Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\n",
@@ -651,7 +651,7 @@
"Action Input: Invoice, Customer\n",
"```\n",
"\n",
"* It then formulates the query using the schema from tool `sql_db_schema`\n",
"- It then formulates the query using the schema from tool `sql_db_schema`\n",
"\n",
"```\n",
"Thought: I should query the total sales per country.\n",
@@ -659,7 +659,7 @@
"Action Input: SELECT Country, SUM(Total) AS TotalSales FROM Invoice INNER JOIN Customer ON Invoice.CustomerId = Customer.CustomerId GROUP BY Country ORDER BY TotalSales DESC LIMIT 10\n",
"```\n",
"\n",
"* It finally executes the generated query using tool `sql_db_query`\n",
"- It finally executes the generated query using tool `sql_db_query`\n",
"\n",
"![sql_usecase.png](../../../static/img/SQLDatabaseToolkit.png)"
]

View File

@@ -32,9 +32,9 @@
"\n",
"In particular, we can employ a [splitting strategy](https://python.langchain.com/docs/integrations/document_loaders/source_code) that does a few things:\n",
"\n",
"* Keeps each top-level function and class in the code is loaded into separate documents. \n",
"* Puts remaining into a separate document.\n",
"* Retains metadata about where each split comes from\n",
"- Keeps each top-level function and class in the code is loaded into separate documents. \n",
"- Puts remaining into a separate document.\n",
"- Retains metadata about where each split comes from\n",
"\n",
"## Quickstart"
]
@@ -101,9 +101,9 @@
"source": [
"We load the py code using [`LanguageParser`](https://python.langchain.com/docs/integrations/document_loaders/source_code), which will:\n",
"\n",
"* Keep top-level functions and classes together (into a single document)\n",
"* Put remaining code into a separate document\n",
"* Retains metadata about where each split comes from"
"- Keep top-level functions and classes together (into a single document)\n",
"- Put remaining code into a separate document\n",
"- Retains metadata about where each split comes from"
]
},
{
@@ -183,8 +183,8 @@
"\n",
"When setting up the vectorstore retriever:\n",
"\n",
"* We test [max marginal relevance](/docs/docs/use_cases/question_answering) for retrieval\n",
"* And 8 documents returned\n",
"- We test [max marginal relevance](/docs/docs/use_cases/question_answering) for retrieval\n",
"- And 8 documents returned\n",
"\n",
"#### Go deeper\n",
"\n",
@@ -336,8 +336,8 @@
"source": [
"The can look at the [LangSmith trace](https://smith.langchain.com/public/2b23045f-4e49-4d2d-8980-dec85259af36/r) to see what is happening under the hood:\n",
"\n",
"* In particular, the code well structured and kept together in the retrieval output\n",
"* The retrieved code and chat history are passed to the LLM for answer distillation\n",
"- In particular, the code well structured and kept together in the retrieval output\n",
"- The retrieved code and chat history are passed to the LLM for answer distillation\n",
"\n",
"![Image description](../../../static/img/code_retrieval.png)"
]

View File

@@ -181,8 +181,8 @@
"source": [
"We can use `chain_type=\"stuff\"`, especially if using larger context window models such as:\n",
"\n",
"* 16k token OpenAI `gpt-3.5-turbo-1106` \n",
"* 100k token Anthropic [Claude-2](https://www.anthropic.com/index/claude-2)\n",
"- 16k token OpenAI `gpt-3.5-turbo-1106` \n",
"- 100k token Anthropic [Claude-2](https://www.anthropic.com/index/claude-2)\n",
"\n",
"We can also supply `chain_type=\"map_reduce\"` or `chain_type=\"refine\"` (read more [here](/docs/modules/chains/document/refine))."
]
@@ -275,8 +275,8 @@
"\n",
"### Go deeper\n",
"\n",
"* You can easily customize the prompt. \n",
"* You can easily try different LLMs, (e.g., [Claude](/docs/integrations/chat/anthropic)) via the `llm` parameter."
"- You can easily customize the prompt. \n",
"- You can easily try different LLMs, (e.g., [Claude](/docs/integrations/chat/anthropic)) via the `llm` parameter."
]
},
{
@@ -506,13 +506,13 @@
" \n",
"**Customization** \n",
"\n",
"* As shown above, you can customize the LLMs and prompts for map and reduce stages.\n",
"- As shown above, you can customize the LLMs and prompts for map and reduce stages.\n",
"\n",
"**Real-world use-case**\n",
"\n",
"* See [this blog post](https://blog.langchain.dev/llms-to-improve-documentation/) case-study on analyzing user interactions (questions about LangChain documentation)! \n",
"* The blog post and associated [repo](https://github.com/mendableai/QA_clustering) also introduce clustering as a means of summarization.\n",
"* This opens up a third path beyond the `stuff` or `map-reduce` approaches that is worth considering.\n",
"- See [this blog post](https://blog.langchain.dev/llms-to-improve-documentation/) case-study on analyzing user interactions (questions about LangChain documentation)! \n",
"- The blog post and associated [repo](https://github.com/mendableai/QA_clustering) also introduce clustering as a means of summarization.\n",
"- This opens up a third path beyond the `stuff` or `map-reduce` approaches that is worth considering.\n",
"\n",
"![Image description](../../static/img/summarization_use_case_3.png)"
]

View File

@@ -34,8 +34,8 @@
"\n",
"Tagging has a few components:\n",
"\n",
"* `function`: Like [extraction](/docs/use_cases/extraction), tagging uses [functions](https://openai.com/blog/function-calling-and-other-api-updates) to specify how the model should tag a document\n",
"* `schema`: defines how we want to tag the document\n",
"- `function`: Like [extraction](/docs/use_cases/extraction), tagging uses [functions](https://openai.com/blog/function-calling-and-other-api-updates) to specify how the model should tag a document\n",
"- `schema`: defines how we want to tag the document\n",
"\n",
"## Quickstart\n",
"\n",
@@ -290,8 +290,8 @@
"source": [
"The [LangSmith trace](https://smith.langchain.com/public/311e663a-bbe8-4053-843e-5735055c032d/r) lets us peek under the hood:\n",
"\n",
"* As with [extraction](/docs/use_cases/extraction), we call the `information_extraction` function [here](https://github.com/langchain-ai/langchain/blob/269f85b7b7ffd74b38cd422d4164fc033388c3d0/libs/langchain/langchain/chains/openai_functions/extraction.py#L20) on the input string.\n",
"* This OpenAI function extraction information based upon the provided schema.\n",
"- As with [extraction](/docs/use_cases/extraction), we call the `information_extraction` function [here](https://github.com/langchain-ai/langchain/blob/269f85b7b7ffd74b38cd422d4164fc033388c3d0/libs/langchain/langchain/chains/openai_functions/extraction.py#L20) on the input string.\n",
"- This OpenAI function extraction information based upon the provided schema.\n",
"\n",
"![Image description](../../static/img/tagging_trace.png)"
]
@@ -394,8 +394,8 @@
"source": [
"### Going deeper\n",
"\n",
"* You can use the [metadata tagger](https://python.langchain.com/docs/integrations/document_transformers/openai_metadata_tagger) document transformer to extract metadata from a LangChain `Document`. \n",
"* This covers the same basic functionality as the tagging chain, only applied to a LangChain `Document`."
"- You can use the [metadata tagger](https://python.langchain.com/docs/integrations/document_transformers/openai_metadata_tagger) document transformer to extract metadata from a LangChain `Document`. \n",
"- This covers the same basic functionality as the tagging chain, only applied to a LangChain `Document`."
]
}
],

View File

@@ -22,8 +22,8 @@
"\n",
"[Web research](https://blog.langchain.dev/automating-web-research/) is one of the killer LLM applications:\n",
"\n",
"* Users have [highlighted it](https://twitter.com/GregKamradt/status/1679913813297225729?s=20) as one of his top desired AI tools. \n",
"* OSS repos like [gpt-researcher](https://github.com/assafelovic/gpt-researcher) are growing in popularity. \n",
"- Users have [highlighted it](https://twitter.com/GregKamradt/status/1679913813297225729?s=20) as one of his top desired AI tools. \n",
"- OSS repos like [gpt-researcher](https://github.com/assafelovic/gpt-researcher) are growing in popularity. \n",
" \n",
"![Image description](../../static/img/web_scraping.png)\n",
" \n",
@@ -31,9 +31,9 @@
"\n",
"Gathering content from the web has a few components:\n",
"\n",
"* `Search`: Query to url (e.g., using `GoogleSearchAPIWrapper`).\n",
"* `Loading`: Url to HTML (e.g., using `AsyncHtmlLoader`, `AsyncChromiumLoader`, etc).\n",
"* `Transforming`: HTML to formatted text (e.g., using `HTML2Text` or `Beautiful Soup`).\n",
"- `Search`: Query to url (e.g., using `GoogleSearchAPIWrapper`).\n",
"- `Loading`: Url to HTML (e.g., using `AsyncHtmlLoader`, `AsyncChromiumLoader`, etc).\n",
"- `Transforming`: HTML to formatted text (e.g., using `HTML2Text` or `Beautiful Soup`).\n",
"\n",
"## Quickstart"
]
@@ -60,8 +60,8 @@
"source": [
"Scraping HTML content using a headless instance of Chromium.\n",
"\n",
"* The async nature of the scraping process is handled using Python's asyncio library.\n",
"* The actual interaction with the web pages is handled by Playwright."
"- The async nature of the scraping process is handled using Python's asyncio library.\n",
"- The actual interaction with the web pages is handled by Playwright."
]
},
{
@@ -86,15 +86,15 @@
"source": [
"Scrape text content tags such as `<p>, <li>, <div>, and <a>` tags from the HTML content:\n",
"\n",
"* `<p>`: The paragraph tag. It defines a paragraph in HTML and is used to group together related sentences and/or phrases.\n",
"- `<p>`: The paragraph tag. It defines a paragraph in HTML and is used to group together related sentences and/or phrases.\n",
" \n",
"* `<li>`: The list item tag. It is used within ordered (`<ol>`) and unordered (`<ul>`) lists to define individual items within the list.\n",
"- `<li>`: The list item tag. It is used within ordered (`<ol>`) and unordered (`<ul>`) lists to define individual items within the list.\n",
" \n",
"* `<div>`: The division tag. It is a block-level element used to group other inline or block-level elements.\n",
"- `<div>`: The division tag. It is a block-level element used to group other inline or block-level elements.\n",
" \n",
"* `<a>`: The anchor tag. It is used to define hyperlinks.\n",
"- `<a>`: The anchor tag. It is used to define hyperlinks.\n",
"\n",
"* `<span>`: an inline container used to mark up a part of a text, or a part of a document. \n",
"- `<span>`: an inline container used to mark up a part of a text, or a part of a document. \n",
"\n",
"For many news websites (e.g., WSJ, CNN), headlines and summaries are all in `<span>` tags."
]
@@ -447,9 +447,9 @@
"\n",
"Looking at the [LangSmith trace](https://smith.langchain.com/public/c3070198-5b13-419b-87bf-3821cdf34fa6/r), we can see what is going on under the hood:\n",
"\n",
"* It's following what is explained in the [extraction](docs/use_cases/extraction).\n",
"* We call the `information_extraction` function on the input text.\n",
"* It will attempt to populate the provided schema from the url content."
"- It's following what is explained in the [extraction](docs/use_cases/extraction).\n",
"- We call the `information_extraction` function on the input text.\n",
"- It will attempt to populate the provided schema from the url content."
]
},
{
@@ -512,12 +512,12 @@
"source": [
"Initialize retriever with the above tools to:\n",
" \n",
"* Use an LLM to generate multiple relevant search queries (one LLM call)\n",
"* Execute a search for each query\n",
"* Choose the top K links per query (multiple search calls in parallel)\n",
"* Load the information from all chosen links (scrape pages in parallel)\n",
"* Index those documents into a vectorstore\n",
"* Find the most relevant documents for each original generated search query"
"- Use an LLM to generate multiple relevant search queries (one LLM call)\n",
"- Execute a search for each query\n",
"- Choose the top K links per query (multiple search calls in parallel)\n",
"- Load the information from all chosen links (scrape pages in parallel)\n",
"- Index those documents into a vectorstore\n",
"- Find the most relevant documents for each original generated search query"
]
},
{
@@ -591,7 +591,7 @@
"source": [
"### Going deeper \n",
"\n",
"* Here's a [app](https://github.com/langchain-ai/web-explorer/tree/main) that wraps this retriever with a lighweight UI."
"- Here's a [app](https://github.com/langchain-ai/web-explorer/tree/main) that wraps this retriever with a lighweight UI."
]
},
{

211
libs/cli/poetry.lock generated
View File

@@ -143,6 +143,25 @@ doc = ["Sphinx", "packaging", "sphinx-autodoc-typehints (>=1.2.0)", "sphinx-rtd-
test = ["anyio[trio]", "coverage[toml] (>=4.5)", "hypothesis (>=4.0)", "mock (>=4)", "psutil (>=5.9)", "pytest (>=7.0)", "pytest-mock (>=3.6.1)", "trustme", "uvloop (>=0.17)"]
trio = ["trio (<0.22)"]
[[package]]
name = "arrow"
version = "1.3.0"
description = "Better dates & times for Python"
optional = false
python-versions = ">=3.8"
files = [
{file = "arrow-1.3.0-py3-none-any.whl", hash = "sha256:c728b120ebc00eb84e01882a6f5e7927a53960aa990ce7dd2b10f39005a67f80"},
{file = "arrow-1.3.0.tar.gz", hash = "sha256:d4540617648cb5f895730f1ad8c82a65f2dad0166f57b75f3ca54759c4d67a85"},
]
[package.dependencies]
python-dateutil = ">=2.7.0"
types-python-dateutil = ">=2.8.10"
[package.extras]
doc = ["doc8", "sphinx (>=7.0.0)", "sphinx-autobuild", "sphinx-autodoc-typehints", "sphinx_rtd_theme (>=1.3.0)"]
test = ["dateparser (==1.*)", "pre-commit", "pytest", "pytest-cov", "pytest-mock", "pytz (==2021.1)", "simplejson (==3.*)"]
[[package]]
name = "async-timeout"
version = "4.0.3"
@@ -172,6 +191,20 @@ docs = ["furo", "myst-parser", "sphinx", "sphinx-notfound-page", "sphinxcontrib-
tests = ["attrs[tests-no-zope]", "zope-interface"]
tests-no-zope = ["cloudpickle", "hypothesis", "mypy (>=1.1.1)", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins", "pytest-xdist[psutil]"]
[[package]]
name = "binaryornot"
version = "0.4.4"
description = "Ultra-lightweight pure Python package to check if a file is binary or text."
optional = false
python-versions = "*"
files = [
{file = "binaryornot-0.4.4-py2.py3-none-any.whl", hash = "sha256:b8b71173c917bddcd2c16070412e369c3ed7f0528926f70cac18a6c97fd563e4"},
{file = "binaryornot-0.4.4.tar.gz", hash = "sha256:359501dfc9d40632edc9fac890e19542db1a287bbcfa58175b66658392018061"},
]
[package.dependencies]
chardet = ">=3.0.2"
[[package]]
name = "certifi"
version = "2023.7.22"
@@ -183,6 +216,17 @@ files = [
{file = "certifi-2023.7.22.tar.gz", hash = "sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082"},
]
[[package]]
name = "chardet"
version = "5.2.0"
description = "Universal encoding detector for Python 3"
optional = false
python-versions = ">=3.7"
files = [
{file = "chardet-5.2.0-py3-none-any.whl", hash = "sha256:e1cf59446890a00105fe7b7912492ea04b6e6f06d4b742b2c788469e34c82970"},
{file = "chardet-5.2.0.tar.gz", hash = "sha256:1b3b6ff479a8c414bc3fa2c0852995695c4a026dcd6d0633b2dd092ca39c1cf7"},
]
[[package]]
name = "charset-normalizer"
version = "3.3.2"
@@ -307,6 +351,27 @@ files = [
{file = "colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44"},
]
[[package]]
name = "cookiecutter"
version = "2.5.0"
description = "A command-line utility that creates projects from project templates, e.g. creating a Python package project from a Python package project template."
optional = false
python-versions = ">=3.7"
files = [
{file = "cookiecutter-2.5.0-py3-none-any.whl", hash = "sha256:8aa2f12ed11bc05628651e9dc4353a10571dd9908aaaaeec959a2b9ea465a5d2"},
{file = "cookiecutter-2.5.0.tar.gz", hash = "sha256:e61e9034748e3f41b8bd2c11f00d030784b48711c4d5c42363c50989a65331ec"},
]
[package.dependencies]
arrow = "*"
binaryornot = ">=0.4.4"
click = ">=7.0,<9.0.0"
Jinja2 = ">=2.7,<4.0.0"
python-slugify = ">=4.0.0"
pyyaml = ">=5.3.1"
requests = ">=2.23.0"
rich = "*"
[[package]]
name = "dataclasses-json"
version = "0.6.1"
@@ -626,6 +691,23 @@ files = [
{file = "iniconfig-2.0.0.tar.gz", hash = "sha256:2d91e135bf72d31a410b17c16da610a82cb55f6b0477d1a902134b24a455b8b3"},
]
[[package]]
name = "jinja2"
version = "3.1.2"
description = "A very fast and expressive template engine."
optional = false
python-versions = ">=3.7"
files = [
{file = "Jinja2-3.1.2-py3-none-any.whl", hash = "sha256:6088930bfe239f0e6710546ab9c19c9ef35e29792895fed6e6e31a023a182a61"},
{file = "Jinja2-3.1.2.tar.gz", hash = "sha256:31351a702a408a9e7595a8fc6150fc3f43bb6bf7e319770cbc0db9df9437e852"},
]
[package.dependencies]
MarkupSafe = ">=2.0"
[package.extras]
i18n = ["Babel (>=2.7)"]
[[package]]
name = "jsonpatch"
version = "1.33"
@@ -754,6 +836,65 @@ profiling = ["gprof2dot"]
rtd = ["jupyter_sphinx", "mdit-py-plugins", "myst-parser", "pyyaml", "sphinx", "sphinx-copybutton", "sphinx-design", "sphinx_book_theme"]
testing = ["coverage", "pytest", "pytest-cov", "pytest-regressions"]
[[package]]
name = "markupsafe"
version = "2.1.3"
description = "Safely add untrusted strings to HTML/XML markup."
optional = false
python-versions = ">=3.7"
files = [
{file = "MarkupSafe-2.1.3-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:cd0f502fe016460680cd20aaa5a76d241d6f35a1c3350c474bac1273803893fa"},
{file = "MarkupSafe-2.1.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:e09031c87a1e51556fdcb46e5bd4f59dfb743061cf93c4d6831bf894f125eb57"},
{file = "MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:68e78619a61ecf91e76aa3e6e8e33fc4894a2bebe93410754bd28fce0a8a4f9f"},
{file = "MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:65c1a9bcdadc6c28eecee2c119465aebff8f7a584dd719facdd9e825ec61ab52"},
{file = "MarkupSafe-2.1.3-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:525808b8019e36eb524b8c68acdd63a37e75714eac50e988180b169d64480a00"},
{file = "MarkupSafe-2.1.3-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:962f82a3086483f5e5f64dbad880d31038b698494799b097bc59c2edf392fce6"},
{file = "MarkupSafe-2.1.3-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:aa7bd130efab1c280bed0f45501b7c8795f9fdbeb02e965371bbef3523627779"},
{file = "MarkupSafe-2.1.3-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:c9c804664ebe8f83a211cace637506669e7890fec1b4195b505c214e50dd4eb7"},
{file = "MarkupSafe-2.1.3-cp310-cp310-win32.whl", hash = "sha256:10bbfe99883db80bdbaff2dcf681dfc6533a614f700da1287707e8a5d78a8431"},
{file = "MarkupSafe-2.1.3-cp310-cp310-win_amd64.whl", hash = "sha256:1577735524cdad32f9f694208aa75e422adba74f1baee7551620e43a3141f559"},
{file = "MarkupSafe-2.1.3-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:ad9e82fb8f09ade1c3e1b996a6337afac2b8b9e365f926f5a61aacc71adc5b3c"},
{file = "MarkupSafe-2.1.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:3c0fae6c3be832a0a0473ac912810b2877c8cb9d76ca48de1ed31e1c68386575"},
{file = "MarkupSafe-2.1.3-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b076b6226fb84157e3f7c971a47ff3a679d837cf338547532ab866c57930dbee"},
{file = "MarkupSafe-2.1.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bfce63a9e7834b12b87c64d6b155fdd9b3b96191b6bd334bf37db7ff1fe457f2"},
{file = "MarkupSafe-2.1.3-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:338ae27d6b8745585f87218a3f23f1512dbf52c26c28e322dbe54bcede54ccb9"},
{file = "MarkupSafe-2.1.3-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:e4dd52d80b8c83fdce44e12478ad2e85c64ea965e75d66dbeafb0a3e77308fcc"},
{file = "MarkupSafe-2.1.3-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:df0be2b576a7abbf737b1575f048c23fb1d769f267ec4358296f31c2479db8f9"},
{file = "MarkupSafe-2.1.3-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:5bbe06f8eeafd38e5d0a4894ffec89378b6c6a625ff57e3028921f8ff59318ac"},
{file = "MarkupSafe-2.1.3-cp311-cp311-win32.whl", hash = "sha256:dd15ff04ffd7e05ffcb7fe79f1b98041b8ea30ae9234aed2a9168b5797c3effb"},
{file = "MarkupSafe-2.1.3-cp311-cp311-win_amd64.whl", hash = "sha256:134da1eca9ec0ae528110ccc9e48041e0828d79f24121a1a146161103c76e686"},
{file = "MarkupSafe-2.1.3-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:8e254ae696c88d98da6555f5ace2279cf7cd5b3f52be2b5cf97feafe883b58d2"},
{file = "MarkupSafe-2.1.3-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cb0932dc158471523c9637e807d9bfb93e06a95cbf010f1a38b98623b929ef2b"},
{file = "MarkupSafe-2.1.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9402b03f1a1b4dc4c19845e5c749e3ab82d5078d16a2a4c2cd2df62d57bb0707"},
{file = "MarkupSafe-2.1.3-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ca379055a47383d02a5400cb0d110cef0a776fc644cda797db0c5696cfd7e18e"},
{file = "MarkupSafe-2.1.3-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:b7ff0f54cb4ff66dd38bebd335a38e2c22c41a8ee45aa608efc890ac3e3931bc"},
{file = "MarkupSafe-2.1.3-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:c011a4149cfbcf9f03994ec2edffcb8b1dc2d2aede7ca243746df97a5d41ce48"},
{file = "MarkupSafe-2.1.3-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:56d9f2ecac662ca1611d183feb03a3fa4406469dafe241673d521dd5ae92a155"},
{file = "MarkupSafe-2.1.3-cp37-cp37m-win32.whl", hash = "sha256:8758846a7e80910096950b67071243da3e5a20ed2546e6392603c096778d48e0"},
{file = "MarkupSafe-2.1.3-cp37-cp37m-win_amd64.whl", hash = "sha256:787003c0ddb00500e49a10f2844fac87aa6ce977b90b0feaaf9de23c22508b24"},
{file = "MarkupSafe-2.1.3-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:2ef12179d3a291be237280175b542c07a36e7f60718296278d8593d21ca937d4"},
{file = "MarkupSafe-2.1.3-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:2c1b19b3aaacc6e57b7e25710ff571c24d6c3613a45e905b1fde04d691b98ee0"},
{file = "MarkupSafe-2.1.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8afafd99945ead6e075b973fefa56379c5b5c53fd8937dad92c662da5d8fd5ee"},
{file = "MarkupSafe-2.1.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8c41976a29d078bb235fea9b2ecd3da465df42a562910f9022f1a03107bd02be"},
{file = "MarkupSafe-2.1.3-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d080e0a5eb2529460b30190fcfcc4199bd7f827663f858a226a81bc27beaa97e"},
{file = "MarkupSafe-2.1.3-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:69c0f17e9f5a7afdf2cc9fb2d1ce6aabdb3bafb7f38017c0b77862bcec2bbad8"},
{file = "MarkupSafe-2.1.3-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:504b320cd4b7eff6f968eddf81127112db685e81f7e36e75f9f84f0df46041c3"},
{file = "MarkupSafe-2.1.3-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:42de32b22b6b804f42c5d98be4f7e5e977ecdd9ee9b660fda1a3edf03b11792d"},
{file = "MarkupSafe-2.1.3-cp38-cp38-win32.whl", hash = "sha256:ceb01949af7121f9fc39f7d27f91be8546f3fb112c608bc4029aef0bab86a2a5"},
{file = "MarkupSafe-2.1.3-cp38-cp38-win_amd64.whl", hash = "sha256:1b40069d487e7edb2676d3fbdb2b0829ffa2cd63a2ec26c4938b2d34391b4ecc"},
{file = "MarkupSafe-2.1.3-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:8023faf4e01efadfa183e863fefde0046de576c6f14659e8782065bcece22198"},
{file = "MarkupSafe-2.1.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:6b2b56950d93e41f33b4223ead100ea0fe11f8e6ee5f641eb753ce4b77a7042b"},
{file = "MarkupSafe-2.1.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9dcdfd0eaf283af041973bff14a2e143b8bd64e069f4c383416ecd79a81aab58"},
{file = "MarkupSafe-2.1.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:05fb21170423db021895e1ea1e1f3ab3adb85d1c2333cbc2310f2a26bc77272e"},
{file = "MarkupSafe-2.1.3-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:282c2cb35b5b673bbcadb33a585408104df04f14b2d9b01d4c345a3b92861c2c"},
{file = "MarkupSafe-2.1.3-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:ab4a0df41e7c16a1392727727e7998a467472d0ad65f3ad5e6e765015df08636"},
{file = "MarkupSafe-2.1.3-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:7ef3cb2ebbf91e330e3bb937efada0edd9003683db6b57bb108c4001f37a02ea"},
{file = "MarkupSafe-2.1.3-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:0a4e4a1aff6c7ac4cd55792abf96c915634c2b97e3cc1c7129578aa68ebd754e"},
{file = "MarkupSafe-2.1.3-cp39-cp39-win32.whl", hash = "sha256:fec21693218efe39aa7f8599346e90c705afa52c5b31ae019b2e57e8f6542bb2"},
{file = "MarkupSafe-2.1.3-cp39-cp39-win_amd64.whl", hash = "sha256:3fd4abcb888d15a94f32b75d8fd18ee162ca0c064f35b11134be77050296d6ba"},
{file = "MarkupSafe-2.1.3.tar.gz", hash = "sha256:af598ed32d6ae86f1b747b82783958b1a4ab8f617b06fe68795c7f026abbdcad"},
]
[[package]]
name = "marshmallow"
version = "3.20.1"
@@ -1075,6 +1216,37 @@ docopt = ">=0.4.0"
pytest = ">=2.6.4"
watchdog = ">=0.6.0"
[[package]]
name = "python-dateutil"
version = "2.8.2"
description = "Extensions to the standard Python datetime module"
optional = false
python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7"
files = [
{file = "python-dateutil-2.8.2.tar.gz", hash = "sha256:0123cacc1627ae19ddf3c27a5de5bd67ee4586fbdd6440d9748f8abb483d3e86"},
{file = "python_dateutil-2.8.2-py2.py3-none-any.whl", hash = "sha256:961d03dc3453ebbc59dbdea9e4e11c5651520a876d0f4db161e8674aae935da9"},
]
[package.dependencies]
six = ">=1.5"
[[package]]
name = "python-slugify"
version = "8.0.1"
description = "A Python slugify application that also handles Unicode"
optional = false
python-versions = ">=3.7"
files = [
{file = "python-slugify-8.0.1.tar.gz", hash = "sha256:ce0d46ddb668b3be82f4ed5e503dbc33dd815d83e2eb6824211310d3fb172a27"},
{file = "python_slugify-8.0.1-py2.py3-none-any.whl", hash = "sha256:70ca6ea68fe63ecc8fa4fcf00ae651fc8a5d02d93dcd12ae6d4fc7ca46c4d395"},
]
[package.dependencies]
text-unidecode = ">=1.3"
[package.extras]
unidecode = ["Unidecode (>=1.1.1)"]
[[package]]
name = "pyyaml"
version = "6.0.1"
@@ -1201,6 +1373,17 @@ files = [
{file = "shellingham-1.5.4.tar.gz", hash = "sha256:8dbca0739d487e5bd35ab3ca4b36e11c4078f3a234bfce294b0a0291363404de"},
]
[[package]]
name = "six"
version = "1.16.0"
description = "Python 2 and 3 compatibility utilities"
optional = false
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*"
files = [
{file = "six-1.16.0-py2.py3-none-any.whl", hash = "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254"},
{file = "six-1.16.0.tar.gz", hash = "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926"},
]
[[package]]
name = "smmap"
version = "5.0.1"
@@ -1232,9 +1415,7 @@ python-versions = ">=3.7"
files = [
{file = "SQLAlchemy-2.0.23-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:638c2c0b6b4661a4fd264f6fb804eccd392745c5887f9317feb64bb7cb03b3ea"},
{file = "SQLAlchemy-2.0.23-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e3b5036aa326dc2df50cba3c958e29b291a80f604b1afa4c8ce73e78e1c9f01d"},
{file = "SQLAlchemy-2.0.23-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:787af80107fb691934a01889ca8f82a44adedbf5ef3d6ad7d0f0b9ac557e0c34"},
{file = "SQLAlchemy-2.0.23-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c14eba45983d2f48f7546bb32b47937ee2cafae353646295f0e99f35b14286ab"},
{file = "SQLAlchemy-2.0.23-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:0666031df46b9badba9bed00092a1ffa3aa063a5e68fa244acd9f08070e936d3"},
{file = "SQLAlchemy-2.0.23-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:89a01238fcb9a8af118eaad3ffcc5dedaacbd429dc6fdc43fe430d3a941ff965"},
{file = "SQLAlchemy-2.0.23-cp310-cp310-win32.whl", hash = "sha256:cabafc7837b6cec61c0e1e5c6d14ef250b675fa9c3060ed8a7e38653bd732ff8"},
{file = "SQLAlchemy-2.0.23-cp310-cp310-win_amd64.whl", hash = "sha256:87a3d6b53c39cd173990de2f5f4b83431d534a74f0e2f88bd16eabb5667e65c6"},
@@ -1271,9 +1452,7 @@ files = [
{file = "SQLAlchemy-2.0.23-cp38-cp38-win_amd64.whl", hash = "sha256:964971b52daab357d2c0875825e36584d58f536e920f2968df8d581054eada4b"},
{file = "SQLAlchemy-2.0.23-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:616fe7bcff0a05098f64b4478b78ec2dfa03225c23734d83d6c169eb41a93e55"},
{file = "SQLAlchemy-2.0.23-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:0e680527245895aba86afbd5bef6c316831c02aa988d1aad83c47ffe92655e74"},
{file = "SQLAlchemy-2.0.23-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9585b646ffb048c0250acc7dad92536591ffe35dba624bb8fd9b471e25212a35"},
{file = "SQLAlchemy-2.0.23-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4895a63e2c271ffc7a81ea424b94060f7b3b03b4ea0cd58ab5bb676ed02f4221"},
{file = "SQLAlchemy-2.0.23-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:cc1d21576f958c42d9aec68eba5c1a7d715e5fc07825a629015fe8e3b0657fb0"},
{file = "SQLAlchemy-2.0.23-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:967c0b71156f793e6662dd839da54f884631755275ed71f1539c95bbada9aaab"},
{file = "SQLAlchemy-2.0.23-cp39-cp39-win32.whl", hash = "sha256:0a8c6aa506893e25a04233bc721c6b6cf844bafd7250535abb56cb6cc1368884"},
{file = "SQLAlchemy-2.0.23-cp39-cp39-win_amd64.whl", hash = "sha256:f3420d00d2cb42432c1d0e44540ae83185ccbbc67a6054dcc8ab5387add6620b"},
@@ -1356,6 +1535,17 @@ files = [
[package.extras]
doc = ["reno", "sphinx", "tornado (>=4.5)"]
[[package]]
name = "text-unidecode"
version = "1.3"
description = "The most basic Text::Unidecode port"
optional = false
python-versions = "*"
files = [
{file = "text-unidecode-1.3.tar.gz", hash = "sha256:bad6603bb14d279193107714b288be206cac565dfa49aa5b105294dd5c4aab93"},
{file = "text_unidecode-1.3-py2.py3-none-any.whl", hash = "sha256:1311f10e8b895935241623731c2ba64f4c455287888b18189350b67134a822e8"},
]
[[package]]
name = "tomli"
version = "2.0.1"
@@ -1402,6 +1592,17 @@ dev = ["autoflake (>=1.3.1,<2.0.0)", "flake8 (>=3.8.3,<4.0.0)", "pre-commit (>=2
doc = ["cairosvg (>=2.5.2,<3.0.0)", "mdx-include (>=1.4.1,<2.0.0)", "mkdocs (>=1.1.2,<2.0.0)", "mkdocs-material (>=8.1.4,<9.0.0)", "pillow (>=9.3.0,<10.0.0)"]
test = ["black (>=22.3.0,<23.0.0)", "coverage (>=6.2,<7.0)", "isort (>=5.0.6,<6.0.0)", "mypy (==0.910)", "pytest (>=4.4.0,<8.0.0)", "pytest-cov (>=2.10.0,<5.0.0)", "pytest-sugar (>=0.9.4,<0.10.0)", "pytest-xdist (>=1.32.0,<4.0.0)", "rich (>=10.11.0,<14.0.0)", "shellingham (>=1.3.0,<2.0.0)"]
[[package]]
name = "types-python-dateutil"
version = "2.8.19.14"
description = "Typing stubs for python-dateutil"
optional = false
python-versions = "*"
files = [
{file = "types-python-dateutil-2.8.19.14.tar.gz", hash = "sha256:1f4f10ac98bb8b16ade9dbee3518d9ace017821d94b057a425b069f834737f4b"},
{file = "types_python_dateutil-2.8.19.14-py3-none-any.whl", hash = "sha256:f977b8de27787639986b4e28963263fd0e5158942b3ecef91b9335c130cb1ce9"},
]
[[package]]
name = "typing-extensions"
version = "4.8.0"
@@ -1596,4 +1797,4 @@ serve = []
[metadata]
lock-version = "2.0"
python-versions = ">=3.8.1,<4.0"
content-hash = "34222f93856b3eb89b55790d36cdba8d118d24d2e740ad26725f384f4711c860"
content-hash = "45941e067d04c25839e9dd967608d222cf4acb38ae9f2ed627e583f44795aaad"

View File

@@ -13,6 +13,7 @@ gitpython = "^3.1.40"
langserve = {extras = ["all"], version = ">=0.0.16"}
uvicorn = "^0.23.2"
tomlkit = "^0.12.2"
cookiecutter = "^2.5.0"
[tool.poetry.scripts]
langchain = "langchain_cli.cli:app"