Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Java Client v4] External storage support + Event refactoring #274

Merged
merged 1 commit into from
Oct 3, 2024

Conversation

jmigueprieto
Copy link
Contributor

@jmigueprieto jmigueprieto commented Oct 2, 2024

Pull Request type

  • Bugfix
  • Feature
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • WHOSUSING.md
  • Other (please describe):

Changes in this PR

  1. Adds support for External Payload Storage (as supported in OSS Conductor Client v3). SEE: TaskClient.evaluateAndUploadLargePayload and WorkflowClient.checkAndUploadToExternalStorage
  2. Event refactoring
    2.1 Moved Events out of automator package to com.netflix.conductor.client.events.
    2.2 Extracted logic from TaskRunner to EventDispatcher- Events are not only published by TaskRunner.
    2.3 TaskClient and WorkflowClient publish events related to external storage payload and workflow start.

TODO

  • Use the events from TaskClient and WorkflowClient in PrometheusMetricsCollector.

@jmigueprieto jmigueprieto self-assigned this Oct 2, 2024
@jmigueprieto jmigueprieto force-pushed the fix/client-v4-external-storage-support branch from 2d91152 to bfa68e8 Compare October 2, 2024 22:14
@@ -370,42 +372,6 @@ private <T, R> R retryOperation(Function<T, R> operation, int count, T input, St
throw new RuntimeException("Exhausted retries performing " + opName);
}

private void publish(TaskRunnerEvent event) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic Moved to EventDispatcher

* A default implementation of {@link ConductorClientConfiguration} where external payload barriers
* is disabled. SEE: SEE: <a href="https://conductor-oss.github.io/conductor/documentation/advanced/externalpayloadstorage.html">External Payload Storage</a>
*/
public class DefaultConductorClientConfiguration implements ConductorClientConfiguration {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and the upload/download logic were taken from Client V3 (current OSS client)

@jmigueprieto
Copy link
Contributor Author

Events from TaskClient and WorkflowClient will be consume in PrometheusMetricsCollector in an up-coming PR along with other changes.

@jmigueprieto jmigueprieto merged commit ba18cb6 into main Oct 3, 2024
6 checks passed
@jmigueprieto jmigueprieto deleted the fix/client-v4-external-storage-support branch October 3, 2024 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants