Piyush Jain 8eea46ed0e Bedrock embeddings async methods (#9024) ...

## Description
This PR adds the `aembed_query` and `aembed_documents` async methods for
improving the embeddings generation for large documents. The
implementation uses asyncio tasks and gather to achieve concurrency as
there is no bedrock async API in boto3.

### Maintainers
@agola11 
@aarora79  

### Open questions
To avoid throttling from the Bedrock API, should there be an option to
limit the concurrency of the calls?

2023-08-10 14:21:03 -07:00

2.1 KiB

Raw Blame History

View Raw