Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: permit abandoned data transfers to be re-started by another data-plane runtime #4650

Conversation

ndr-brt
Copy link
Member

@ndr-brt ndr-brt commented Nov 28, 2024

What this PR changes/adds

Add the capability to the DataPlaneManager to pick up data transfers (DataFlows) "abandoned" by other runtimes.

BREAKING CHANGE: a new runtime_id column has been added in the edc_data_plane table

Why it does that

Permit data plane to be deployed in a clustered environment.

Further notes

  • changed the runtime-id assignment logic, currently it was always set to a random value, but in fact it should also be assignable with the edc.runtime.id setting.
  • added the isRunning capability to EmbeddedRuntime
  • created a new mockito ArrayContainsMatcher that permits to match arguments when the passed array is contained in the actual one (necessary for unit-testing the flow lease behavior)

Linked Issue(s)

Closes #4440

Please be sure to take a look at the contributing guidelines and our etiquette for pull requests.

@ndr-brt ndr-brt added enhancement New feature or request dpf Feature related to the Data Plane Framework breaking-change Will require manual intervention for version update labels Nov 28, 2024
@ndr-brt ndr-brt requested a review from wolf4ood November 28, 2024 13:30
Copy link
Contributor

@jimmarino jimmarino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a separate PR, can you update the Data Plane user document to describe cluster operations?

@ndr-brt ndr-brt force-pushed the 4440-restart-interrupted-data-flows-cluster branch from 6074f11 to d97a093 Compare December 2, 2024 10:25
@ndr-brt
Copy link
Member Author

ndr-brt commented Dec 2, 2024

To be rebased and merged after #4653 will be merged.

EDIT: rebased

@ndr-brt ndr-brt force-pushed the 4440-restart-interrupted-data-flows-cluster branch 3 times, most recently from 531b9bf to 67281ea Compare December 4, 2024 08:50
@ndr-brt ndr-brt force-pushed the 4440-restart-interrupted-data-flows-cluster branch from 67281ea to 3d3a95c Compare December 4, 2024 08:55
@ndr-brt ndr-brt merged commit 81e2cba into eclipse-edc:main Dec 5, 2024
21 checks passed
@ndr-brt ndr-brt deleted the 4440-restart-interrupted-data-flows-cluster branch December 5, 2024 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change Will require manual intervention for version update dpf Feature related to the Data Plane Framework enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

data-plane: restart interrupted data flows
2 participants