server: support for customizable data providers (outputs) via data provider factories #80

bhufmann · 2024-09-23T13:57:49Z

It uses the new IDataProviderFactory.getAdapter() interface to get ITmfDataProviderConfigurator implementation to create data providers. It passes an ITmfConfiguration instance with default TSP parameters (name, description, typeId) and the custom parameters in a JSON converted Map<String, Object> on to the configurator which the implementer of ITmfDataProviderConfigurator needs to apply.

It depends on
eclipse-tracecompass/org.eclipse.tracecompass#154
eclipse-tracecompass/org.eclipse.tracecompass#167

~~This PR includes changes of PR #71 and its dependent pull requests, and will be rebased once dependent PRs are merged.~~

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

PatrickTasse · 2024-09-27T18:36:51Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+            @ApiResponse(responseCode = "200", description = "Returns a list of output provider descriptors", content = @Content(array = @ArraySchema(schema = @Schema(implementation = DataProvider.class)))),
+            @ApiResponse(responseCode = "400", description = INVALID_PARAMETERS, content = @Content(schema = @Schema(implementation = String.class))),
+            @ApiResponse(responseCode = "404", description = PROVIDER_NOT_FOUND, content = @Content(schema = @Schema(implementation = String.class))),
+            @ApiResponse(responseCode = "404", description = NO_SUCH_CONFIGURATION, content = @Content(schema = @Schema(implementation = String.class))),


Can this happen? Is it when the typeId inside the OutputConfigurationQueryParameters is not recognized by the data provider (outputId)?

Yes, if configType for a typeId doesn't exists for the given output.

PatrickTasse · 2024-09-27T18:39:53Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+    @Operation(summary = "Delete a configuration instance of a given configuration source type", responses = {
+            @ApiResponse(responseCode = "200", description = "The derived data provider (and it's configuration) was successfully deleted", content = @Content(schema = @Schema(implementation = org.eclipse.tracecompass.incubator.internal.trace.server.jersey.rest.core.model.Configuration.class))),
+            @ApiResponse(responseCode = "404", description = PROVIDER_NOT_FOUND, content = @Content(schema = @Schema(implementation = String.class))),
+            @ApiResponse(responseCode = "404", description = NO_SUCH_CONFIGURATION, content = @Content(schema = @Schema(implementation = String.class))),


I assume PROVIDER_NOT_FOUND is for the outputId and NO_SUCH_CONFIGURATION is for the derivedOutputId?
Should we have a different message for that one?

yes, if derived output doesn't exists then we would return NO_SUCH_CONFIGURATION.

PatrickTasse · 2024-10-15T15:04:33Z

...cubator/internal/trace/server/jersey/rest/core/model/OutputConfigurationQueryParameters.java

+ * Contributes to the model used for TSP swagger-core annotations.
+ */
+@Schema(allOf = ConfigurationQueryParameters.class)
+public interface OutputConfigurationQueryParameters extends ConfigurationQueryParameters {


A suggestion was made to have typeId, name, and description as explicit fields in a parameters map because these are used by the configuration feature itself, so their use can be described in the TSP, and they don't need to be included or described in the JSON Schema.

For the application-specific parameters, it was suggested to use a 'properties' field in the parameters map, where the value is an object that matches and validates against the corresponding 'properties' definition in the JSON Schema of the configuration source type.

It's true, that name and description are valid for all configurations, so is the typeId. They could be part of the TSP definition instead of each configuration extension. The typeId would be required, the name and description can be optional.

Using properties would help us to distinguish the TSP string parameters and the custom parameters. However, if we use parameters, then the whole json would look like a serialized ITmfConfiguration (without the ID which will be assigned). I think both are ok.

So here how it would look like from the TSP point of view (for InAndOut analysis) :

{ "parameters": { "name": "An InAndOut Analysis", "description": "My own InAndOut analysis", "typeId": "org.eclipse.tracecompass.incubator.internal.inandout.core.config", "properties": { "specifiers" : [ { "label":"Latency", "inRegex":"(\\S*)_entry", "outRegex":"(\\S*)_exit", "contextInRegex":"(\\S*)_entry", "contextOutRegex":"(\\S*)_exit", "classifier":"CPU" } ] } }, }

FYI @abhinava-ericsson

As per discussion here #70 (comment), properties will be replaced by parameters to match the Configuration data structure

I came to the realization, that having the name, description and type ID inside the Map<String, Object> and extracting the custom parameters map is not ideal. The items passed are actually the items of an ITmfConfiguration object, where the ID is created in the back-end and doesn't need to be passed.

See eclipse-tracecompass/org.eclipse.tracecompass#167 for an update of the API.

PatrickTasse · 2024-10-22T17:53:13Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+        }
+
+        Map<String, Object> parameters = queryParameters.getParameters();
+        if (!(parameters.get(DataProviderParameterUtils.CONFIGURATION_TYPE_ID) instanceof String)){


This should be queryParameters.getTypeId() ?
edit: But this is only introduced in the next commit.
edit 2: Then you can probably remove the "typeId" constant in DataProviderParameterUtils?

PatrickTasse · 2024-10-22T17:53:56Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

    }

+    private static @Nullable IDataProviderDescriptor getDescriptor(@NonNull ITmfTrace experiment, @NonNull String outputId) {
+        List<IDataProviderDescriptor> list = DataProviderManager.getInstance().getAvailableProviders(experiment);


If method not static this could use 'manager' variable.

PatrickTasse · 2024-10-22T17:54:04Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+        return null;
+    }
+
+


extra blank line

PatrickTasse · 2024-10-22T18:12:24Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

+
+            IDataProviderFactory factory = manager.getFactory(outputId);
+            if (factory == null) {
+                return Response.status(Status.METHOD_NOT_ALLOWED).entity(NO_SUCH_PROVIDER).build();


It seems this status should only be used when the method (e.g. POST, DELETE) is not allowed for this endpoint.

Makes sense. I'll update what is returned and align the API documentation

PatrickTasse · 2024-10-22T19:15:12Z

...cecompass/incubator/internal/trace/server/jersey/rest/core/services/DataProviderService.java

@@ -521,6 +525,9 @@ public Response getStates(
                return Response.status(Status.BAD_REQUEST).entity(errorMessage).build();
            }

+            Object value = params.get(DataProviderParameterUtils.REQUESTED_ITEMS_KEY);
+            System.out.println(value);


Update ConfigurationManagerService and corresponding tests for that. Signed-off-by: Bernd Hufmann <[email protected]>

Update skeleton endpoints in DataProviderService for that. Use ITmfDataProviderConfigurator interface for that. Signed-off-by: Bernd Hufmann <[email protected]>

bhufmann mentioned this pull request Sep 23, 2024

server: support for customizable data providers (outputs) #72

Closed

bhufmann force-pushed the output-conifg-factory-impl branch from 1a087fb to fd96cc5 Compare September 23, 2024 23:45

PatrickTasse requested changes Sep 27, 2024

View reviewed changes

PatrickTasse requested changes Oct 15, 2024

View reviewed changes

bhufmann force-pushed the output-conifg-factory-impl branch from fd96cc5 to 6a35ec9 Compare October 17, 2024 19:38

bhufmann mentioned this pull request Oct 17, 2024

Custom analysis API and InAndOut Implementation #88

Merged

bhufmann requested a review from PatrickTasse October 17, 2024 21:58

bhufmann force-pushed the output-conifg-factory-impl branch from 6a35ec9 to 96a343d Compare October 21, 2024 12:55

PatrickTasse requested changes Oct 22, 2024

View reviewed changes

PatrickTasse reviewed Oct 22, 2024

View reviewed changes

bhufmann force-pushed the output-conifg-factory-impl branch from 96a343d to ea062d8 Compare October 22, 2024 19:03

PatrickTasse requested changes Oct 22, 2024

View reviewed changes

bhufmann added 2 commits October 22, 2024 15:22

server: Use ITmfConfiguration in create interface for configurations

872b4e9

Update ConfigurationManagerService and corresponding tests for that. Signed-off-by: Bernd Hufmann <[email protected]>

server: Implement support for customizable data providers

81e3eb3

Update skeleton endpoints in DataProviderService for that. Use ITmfDataProviderConfigurator interface for that. Signed-off-by: Bernd Hufmann <[email protected]>

bhufmann force-pushed the output-conifg-factory-impl branch from ea062d8 to 81e3eb3 Compare October 22, 2024 19:30

PatrickTasse approved these changes Oct 22, 2024

View reviewed changes

bhufmann merged commit fcbdca6 into eclipse-tracecompass-incubator:master Oct 22, 2024
2 checks passed

bhufmann deleted the output-conifg-factory-impl branch October 22, 2024 20:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

server: support for customizable data providers (outputs) via data provider factories #80

server: support for customizable data providers (outputs) via data provider factories #80

bhufmann commented Sep 23, 2024 •

edited

Loading

PatrickTasse Sep 27, 2024

bhufmann Oct 17, 2024

PatrickTasse Sep 27, 2024

bhufmann Oct 17, 2024

PatrickTasse Oct 15, 2024

bhufmann Oct 15, 2024

bhufmann Oct 17, 2024 •

edited

Loading

bhufmann Oct 21, 2024 •

edited

Loading

PatrickTasse Oct 22, 2024

PatrickTasse Oct 22, 2024

bhufmann Oct 22, 2024

PatrickTasse Oct 22, 2024

bhufmann Oct 22, 2024

PatrickTasse Oct 22, 2024

bhufmann Oct 22, 2024

PatrickTasse Oct 22, 2024

server: support for customizable data providers (outputs) via data provider factories #80

server: support for customizable data providers (outputs) via data provider factories #80

Conversation

bhufmann commented Sep 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bhufmann Oct 17, 2024 • edited Loading

Choose a reason for hiding this comment

bhufmann Oct 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bhufmann commented Sep 23, 2024 •

edited

Loading

bhufmann Oct 17, 2024 •

edited

Loading

bhufmann Oct 21, 2024 •

edited

Loading