Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.Net: Add Async Support for ConverseStreamResponse to Avoid Thread Blocking #10159

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

shethaadit
Copy link
Contributor

Motivation and Context

This change replaces the use of response.Stream.AsEnumerable() with await foreach for async streaming in the Amazon Bedrock SDK. The previous method blocked the calling thread, leading to potential thread exhaustion in high-concurrency scenarios. This update aligns with best practices for asynchronous operations in .NET.

Description

  • Replaced blocking foreach with await foreach for streaming ConverseStreamResponse chunks asynchronously.
  • Upgraded Amazon Bedrock runtime and core SDKs to support this functionality.

Contribution Checklist

Fixes #9519

@shethaadit shethaadit requested a review from a team as a code owner January 11, 2025 08:50
@markwallace-microsoft markwallace-microsoft added .NET Issue or Pull requests regarding .NET code kernel Issues or pull requests impacting the core kernel labels Jan 11, 2025
@github-actions github-actions bot changed the title Add Async Support for ConverseStreamResponse to Avoid Thread Blocking .Net: Add Async Support for ConverseStreamResponse to Avoid Thread Blocking Jan 11, 2025
@RogerBarret0
Copy link
Contributor

@shethaadit Do we really need to rely on 4.0.0-preview package for this? Can't we keep the current GA version?

@shethaadit
Copy link
Contributor Author

Hi @RogerBarret0, thia fix just got available yesterday in latest Nuget. We would need to keep the new version to get this fix.

aws/aws-sdk-net#3542 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kernel Issues or pull requests impacting the core kernel .NET Issue or Pull requests regarding .NET code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

.Net: Bug: .NET Bedrock streaming completion is keeping the request thread busy
4 participants