Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature split local and remote buffer tracking #272

Draft
wants to merge 5 commits into
base: v0.7-dev
Choose a base branch
from

Conversation

ryan-dozier
Copy link
Contributor

Summary

This feature aims to split how the comm buffer handles local and remote communications separately. This should allow for more fine-tuning in our environment parameters as we can set the node local and node remote target msg size separately.

Description of Changes

  • Split m_send_buffer_bytes and m_send_dest_queue into local and remote counterparts
  • Added a comm function queue_next_send to reduce repeated code since we have two dest queues now
  • Updated several places in the comm to check if the next dest is local and handled appropriately

Unfinished items / Todo

  • More testing, especially on large node scales. I've tested locally, and on a few nodes with my SHM version of this but haven't extensively tested this branch
  • There's likely a cleaner way to implement some if(local) checks.
  • Check comm::check_if_production_halt_required. I have this as the pending isend_bytes being less than the sum of the two buffers. In my SHM version, this is just the remote_send_bytes, as we would expect the majority of communication to be through shared memory.

@ryan-dozier ryan-dozier marked this pull request as draft January 7, 2025 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant