Taqi Jaffri b7290f01d8 Batching for hf_pipeline (#10795) ... The huggingface pipeline in langchain (used for locally hosted models) does not support batching. If you send in a batch of prompts, it just processes them serially using the base implementation of _generate: https://github.com/docugami/langchain/blob/master/libs/langchain/langchain/llms/base.py#L1004C2-L1004C29 This PR adds support for batching in this pipeline, so that GPUs can be fully saturated. I updated the accompanying notebook to show GPU batch inference. --------- Co-authored-by: Taqi Jaffri <tjaffri@docugami.com>		2023-09-25 18:23:11 +01:00
..
_scripts	llm feat table revision (#10947)	2023-09-22 10:29:12 -07:00
api_reference	add model feat table (#10921)	2023-09-22 01:10:27 -07:00
docs_skeleton	redirect mrkl (#10979)	2023-09-23 16:09:13 -07:00
extras	Batching for hf_pipeline (#10795)	2023-09-25 18:23:11 +01:00
snippets	JSONLoader Documentation Fix (#10505)	2023-09-21 11:37:40 -07:00
.local_build.sh	Update local script for docs build (#8377)	2023-07-27 13:13:59 -07:00
vercel_requirements.txt	Add api cross ref linking (#8275)	2023-07-26 12:38:58 -07:00