Use aiobotocore for the ChatBedrock async implementation #27443
fabiopicchi
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Checked
Feature request
To avoid the creation of a ThreadPoolExecutor, we should evaluate using aiobotocore for ChatBedrock. It is reasonably stable and even added support to BedrockConverse.
Motivation
Better control over the application's lifecycle. I have a use case in which I spin up a background thread that schedules langchain tasks to its own event loop (main thread runs synchronous code) and I would like to avoid the creation of another ThreadPoolExecutor from this background thread to help debuggability and tame complexity.
Proposal (If applicable)
My proposal would be to keep the _astream and other asynchronous methods from the interface but reimplement them using aiobotocore.
Beta Was this translation helpful? Give feedback.
All reactions