github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-06-28 17:38:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs for batch size (#1082)

Browse Source
This commit is contained in:
Harrison Chase 2023-02-15 23:53:56 -08:00 committed by GitHub
parent 5e10e19bfe
commit 971458c5de
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 28 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
docs/modules/chains/combine_docs_examples/qa_with_sources.ipynb
View File

@ -365,6 +365,20 @@
"chain({\"input_documents\": docs, \"question\": query}, return_only_outputs=True)"
]
},
{
"cell_type": "markdown",
"id": "d943c6c1",
"metadata": {},
"source": [
"**Batch Size**\n",
"\n",
"When using the `map_reduce` chain, one thing to keep in mind is the batch size you are using during the map step. If this is too high, it could cause rate limiting errors. You can control this by setting the batch size on the LLM used. Note that this only applies for LLMs with this parameter. Below is an example of doing so:\n",
"\n",
"```python\n",
"llm = OpenAI(batch_size=5, temperature=0)\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "5bf0e1ab",

14
docs/modules/chains/combine_docs_examples/question_answering.ipynb
View File

@ -356,6 +356,20 @@
"chain({\"input_documents\": docs, \"question\": query}, return_only_outputs=True)"
]
},
{
"cell_type": "markdown",
"id": "6391b7ab",
"metadata": {},
"source": [
"**Batch Size**\n",
"\n",
"When using the `map_reduce` chain, one thing to keep in mind is the batch size you are using during the map step. If this is too high, it could cause rate limiting errors. You can control this by setting the batch size on the LLM used. Note that this only applies for LLMs with this parameter. Below is an example of doing so:\n",
"\n",
"```python\n",
"llm = OpenAI(batch_size=5, temperature=0)\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "6ea50ad0",