Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BugFix] Fix crash when receive a req with structed output in DP attention mode. #3841

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

hcyz33
Copy link

@hcyz33 hcyz33 commented Feb 25, 2025

Motivation

Fix the issuse:
#3600

Modifications

  1. set sampling_info_done in idle batch.
  2. use attn_tp_cpu_group when enable dp attention

Checklist

@hcyz33 hcyz33 force-pushed the fix_dpatten_structed_out branch from b6d0dde to ce07173 Compare February 25, 2025 08:31
@zhaochenyang20
Copy link
Collaborator

@FrankLeeeee I've run the CI. And wait for your approve.

@zhaochenyang20
Copy link
Collaborator

Please review it, thansk!

Copy link
Collaborator

@FrankLeeeee FrankLeeeee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@zhaochenyang20
Copy link
Collaborator

@hcyz33 Thanks. We will merge it after passing the CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants