Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Endpoint::close not consuming self #2882

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

iacore
Copy link

@iacore iacore commented Nov 3, 2024

Description

Incorporating the idea from #2741 (comment). endpoint.close() need not consume endpoint.

Breaking Changes

Notes & open questions

I don't see #[must_use] on Endpoint, so I assume this function is optional? As in, it may not need to be called.

Change checklist

  • Self-review.
  • Documentation updates following the style guide, if relevant.
  • Tests if relevant.
  • All breaking changes documented.

Copy link
Contributor

@flub flub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to get the changes in main, and then check if the new doc comment needs tweaking because we need to make it clear how this works. But in general I think this is probably a good change.

iroh-net/src/endpoint.rs Outdated Show resolved Hide resolved
@@ -465,7 +465,6 @@ impl<D: iroh_blobs::store::Store> NodeInner<D> {
// connections: Operations will immediately fail with ConnectionError::LocallyClosed.
// All streams are interrupted, this is not graceful.
self.endpoint
.clone()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol, frankly this is probably sufficient reason to not make this move the endpoint.

@flub flub added the c-iroh-net label Nov 5, 2024
Co-authored-by: Floris Bruynooghe <[email protected]>
@@ -858,9 +858,7 @@ impl Endpoint {
tracing::debug!("Closing connections");
endpoint.close(error_code, reason);
endpoint.wait_idle().await;
// In case this is the last clone of `Endpoint`, dropping the `quinn::Endpoint` will
// make it more likely that the underlying socket is not polled by quinn anymore after this
drop(endpoint);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@divagant-martian does it make sense to store the quinn::Endpoint in an Option so that it can be taken out and dropped? That would also mean that regardless how many clones there are of iroh_net::Endpoint they will all lose the quinn::Endpoint when closed, which can be used as a very good indicator that the endpoint is closed.

Copy link
Author

@iacore iacore Nov 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arc<Mutex<Option<EndPoint>>>?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

having to put that into a mutex would not be great. it already has mutexes inside it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, we should not try and drop the quinn endpoint here.

Copy link
Contributor

@flub flub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM to me, but would love @divagant-martian to lgtm too (or not)

@divagant-martian
Copy link
Contributor

divagant-martian commented Nov 12, 2024

this was in part because of how quinn used to work. Closing the magic socket before dropping the endpoint would basically always emit an error in logs breaking all our cli tests. The changes in the endpoint loop make this less likely to happen in this scenario. This is not to say the previous state was exactly what we needed: As the comment mentions, it's there to make it less likely that the socket is polled. In general closing here is messy and kinda gross. When we wrote this code (not sure now) the socket was the last resource the (quinn) endpoint dropped. This was later that the drop of the endpoint itself given the many cloned Arcs and spawned tasks.

All this long rant is to say: there was a reason this looked this way. To get a best effort imperfect close because both how quinn works and the lack of async drop (and I thought the rant was over) but after we updated quinn I think the error is no longer an issue and we are fine with dropping the endpoint after the socket in most cases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 🏗 In progress
Development

Successfully merging this pull request may close these issues.

4 participants