Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use get_max_fragment_size for medium_data tests #369

Merged
merged 1 commit into from
Oct 24, 2024

Conversation

sunshowers
Copy link
Contributor

@sunshowers sunshowers commented Oct 18, 2024

Currently, the medium_data tests all hang on illumos because (a) we first send a message and then receive it on the same thread, and (b) the tests fill up the pipe buffer before the send is complete. That's because illumos uses a smaller buffer size than other platforms like Linux.

The goal of these tests is to test situations where fragmentation doesn't happen, so use get_max_fragment_size which shouldn't fill up the pipe buffer under ordinary circumstances.

(There are other tests which check for fragmentation, so that code path is still covered.)

It would also be nice to move all of the receives to other threads in the future, but this appears to work quite well.

See #101.

Copy link
Member

@jdm jdm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a reasonable improvement!

@jdm
Copy link
Member

jdm commented Oct 18, 2024

Oh, lol—for the macOS implementation the max fragment size is usize::MAX, so that's why we're seeing capacity overflow panics in CI.

@sunshowers
Copy link
Contributor Author

Oh, lol—for the macOS implementation the max fragment size is usize::MAX, so that's why we're seeing capacity overflow panics in CI.

Ah I guess that's because the macOS impl uses Mach ports. I'll set a cap on the fragment size used, thanks.

@sunshowers
Copy link
Contributor Author

Updated! Capped it at the previously-used 64KiB value.

@sagudev
Copy link
Member

sagudev commented Oct 18, 2024

Are we sure we need to change these values? I think you need to make sure that fragmentation works on illumos:

// These tests only apply to platforms that need fragmentation.
#[cfg(all(
not(feature = "force-inprocess"),
any(target_os = "linux", target_os = "freebsd", target_os = "windows")
))]
mod fragment_tests {

@sunshowers
Copy link
Contributor Author

sunshowers commented Oct 18, 2024

Are we sure we need to change these values? I think you need to make sure that fragmentation works on illumos:

The huge_data tests do exercise fragmentation, but good point about those specific tests -- will enable that. Note however that ipc-channel doesn't yet compile on illumos, there's another PR I need to do which depends on this one.

@sunshowers
Copy link
Contributor Author

I've updated the PR summary to indicate what's happening.

Some platforms like illumos have smaller buffer sizes, so use
`get_max_fragment_size` which shouldn't block under ordinary circumstances.

It would also be nice to move all of the fetches to other threads, but this
appears to work quite well.

See servo#101.
@sunshowers
Copy link
Contributor Author

Also enabled the fragment tests on illumos.

@sunshowers
Copy link
Contributor Author

Hi — is there any way I can help get this landed? Thanks!

@mrobinson
Copy link
Member

Apologies for the wait and thanks for the fix. This looks good to me!

@mrobinson mrobinson added this pull request to the merge queue Oct 24, 2024
Merged via the queue into servo:main with commit bdfdd0a Oct 24, 2024
17 checks passed
@sunshowers sunshowers deleted the max-frag branch October 24, 2024 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants