Add options to manipulate the Dev UI Card Page of another extension

[Dudeplayz]

Description

We have the following extension structure:
Commons
-> Extension 1
-> Extension 2

But both "end" extension share common logic, also for the Dev-UI. But it seems there isn't a possibility to build card page data, that is then added to both end extensions. Instead the "Commons" extension has to be visible, which isn't preferred and we want to unlist the commons extension.

Fell free to check out the PR in our project: mcollovati/quarkus-hilla#1036

Implementation ideas

Introduce a build step to get the card page for another extension to allow the manipulation of it. E.g. adding another page, remove a page or other modifications.

[quarkus-bot]

/cc @cescoffier (devui), @phillip-kruger (devui)

[Dudeplayz]

Some additional data:
The CardPageBuildItem is in the commons extension defined as:

    @BuildStep(onlyIf = IsDevelopment.class)
    public CardPageBuildItem pages(EndpointBuildItem endpointBuildItems) {
        CardPageBuildItem cardPageBuildItem = new CardPageBuildItem();
        cardPageBuildItem.addBuildTimeData("hillaEndpoints", endpointBuildItems.getEndpoints());
        cardPageBuildItem.addPage(Page.webComponentPageBuilder()
                .title("Endpoints")
                .icon("font-awesome-solid:table-list")
                .componentLink("qwc-quarkus-hilla-endpoints.js")
                .staticLabel(String.valueOf(endpointBuildItems.getEndpoints().stream()
                        .mapToInt(a -> a.children().size())
                        .sum())));
        return cardPageBuildItem;
    }

Which results in two card pages, as both extension must be shown. The commons module can't be unlisted. To bring it in both extension I would have to copy/paste some stuff, which I think isn't a good way to do it.

[phillip-kruger]

I am not sure if this should be the responsibility of the Dev UI Cards to do this. Extensions can already do this by creating their own sharable BuildItems (in an SPI or in your case maybe in common) and share data this way.

[Dudeplayz]

Hi @phillip-kruger .
I tried the approach you mentioned, but I would end still at an copy/paste for the qwc component js file, since this file is also in commons, because functionality is always the same.

Trying to set the namespace for the component to common works, as long common also has the page available and is visible. If I unlist the commons card or remove the page from the commons card, it also isn't available for the other extensions, as they are not copied / made available to the frontend. Whits behavior makes sense, because why should made things available in frontend if not accessed by the extension card itself.

If I missed something here or you have some other approach please let me know.

One thing I also would like to mention here: It would be nice, if the page builders have appropriate JDocs, because I had to dig through the source code to understand what each options does.

[phillip-kruger]

OK, so you are trying to share a component (js) between two extensions ? Are both of these always available ? If so you can add it to one and use it from the other. If not, I don't think this is possible (currently) except if you have a common module (like you mentioned) but that needs to be added to Dev UI. We can add a BuildItem that enables this, if this is what you want ?

[Dudeplayz]

OK, so you are trying to share a component (js) between two extensions ? Are both of these always available ? If so you can add it to one and use it from the other. If not, I don't think this is possible (currently) except if you have a common module (like you mentioned) but that needs to be added to Dev UI. We can add a BuildItem that enables this, if this is what you want ?

In this case, both would be available because the common extension is always included by both:
|- common
|---> quarkus-hilla
|---> quarkus-hilla-react

But the common extension should be marked as "unlisted", so that it is never shown in the dev-ui, because it contains only shared functionality. So should the dev-ui functions, but these should be listed under each extension itself.

So what I want to do:
The js component is in the common module. But this needs to be available to the extending extensions. For the data I was able to share it with BuildItems. But not for the js file. The best thing would be to host the js file under the namespace of the extending extensions. So that the common extension is completely hidden to the user, as it is only for sharing stuff.

So I think a BuildItem to make resources available to the dev-ui would be great. The important thing here is that the namespace is customizable for each extension that uses it.

[phillip-kruger]

Just an update: I started looking at this

[phillip-kruger]

Ok, when I started looking at this, I realized that this might already be possible. So I am going to share how I think this can be done, let me know if this works for you.

You can look at phillip-kruger/quarkus-jokes@73b6416 for my example. I will reference this example below.

Basically, in your common module you will add the web component (example qwc-joke.js). This is a normal component as you would have used in the deployment extension, however, if you want to use JsonRPC, you will not contruct it with this, but rather allow the user (the component that will use this) pass the namespace in an attribure (property). JsonRPC auto detect the namespace from this, but you can also pass it in.

To make this common component available in Dev UI you need to add the following:

public class JokesCommonProcessor {
    private static final GACT UI_JAR = new GACT("io.quarkiverse.jokes", "quarkus-jokes-common", null, "jar");
    private static final String DEVUI = "dev-ui";

    @BuildStep(onlyIf = IsDevelopment.class)
    void createShared(BuildProducer<WebJarBuildItem> webJarBuildProducer,
            BuildProducer<DevUIWebJarBuildItem> devUIWebJarProducer) {
        webJarBuildProducer.produce(WebJarBuildItem.builder()
                .artifactKey(UI_JAR)
                .root(DEVUI + "/")
                .filter(new WebJarResourcesFilter() {
                    @Override
                    public WebJarResourcesFilter.FilterResult apply(String fileName, InputStream file) throws IOException {
                        return new WebJarResourcesFilter.FilterResult(file, true);
                    }
                }).build());
        devUIWebJarProducer.produce(new DevUIWebJarBuildItem(UI_JAR, DEVUI));
    }
}

This will make the js resources in your common jar (the GACT in the code snippet above) available in Dev UI.

Now in my deployment module, I add a dependency to common, and I still create a Page (or Footer or Menu) but the page is now really just importing and displaying that common component:

import { LitElement, html, css} from 'lit';
import './../io.quarkiverse.jokes.quarkus-jokes-common/qwc-joke.js';

/**
 * This component shows how to add to the section menu
 */
export class QwcJokesMenu extends LitElement {
    
    render() {
        return html`<qwc-joke namespace='io.quarkiverse.jokes.quarkus-jokes'></qwc-joke>`;    
    }
}
customElements.define('qwc-jokes-menu', QwcJokesMenu);

The important part is the import of the common component (using the correct namespace, that is made up from the groupId and artifactId) and the passing the namespace of the current component (so that JsonRPC can find the methods)

Let me know if this works for you.

[phillip-kruger]

Also if you are using build-time-data, you either need to implement the filter like here:

quarkus/extensions/vertx-http/deployment/src/main/java/io/quarkus/devui/deployment/DevUIProcessor.java

Line 715 in c872c3b

webJarBuildProducer.produce(WebJarBuildItem.builder()

where we do a replace for devui-data or import with the correct namespace. (Let me know if you need this and I can add this to the example)

[Dudeplayz]

Hi @phillip-kruger ,
thanks for the detailed description. Yes we are using build-time data, so it would be nice if you could add some example for this too. I will try it then tomorrow.
Thanks!

[phillip-kruger]

Ok so you have two options.

Option 1 (same as we use this internally for dev ui):
(see phillip-kruger/quarkus-jokes@86e0254)

Because we want extension developer to not worry about the namespace, you can use this to create your jsonrpc (and we will detect the namespace) and similarly we allow importing from build-time-data, that we will replace with the namespace. We want to use namespaces everywhere so that methods in jsonrpc and fields/keys in build-time-data is scoped to an extension.

First, to add common buildtimedata you can use the BuildTimeConstBuildItem:

        Map<String, Object> buildTimeData = new HashMap<>();
        buildTimeData.put("someKey", "value from common"); // This value can be an object, as long as this can serialize to json

        BuildTimeConstBuildItem item = new BuildTimeConstBuildItem(namespace, buildTimeData);
        buildTimeConstProducer.produce(item);

So now you can do the same in your creation of the common js (same as my example in the previous reply but now with the filter implemented same as we do in in Dev UI)

    @BuildStep(onlyIf = IsDevelopment.class)
    void createShared(BuildProducer<WebJarBuildItem> webJarBuildProducer,
            BuildProducer<DevUIWebJarBuildItem> devUIWebJarProducer,
            BuildProducer<BuildTimeConstBuildItem> buildTimeConstProducer) {

        String namespace = UI_JAR.getGroupId() + "." + UI_JAR.getArtifactId();

        // Build Time data (under the common namespace, so usable in the common
        // This will allow you to access someKey in js
        
        Map<String, Object> buildTimeData = new HashMap<>();
        buildTimeData.put("someKey", "value from common"); // This value can be an object, as long as this can serialize to json

        BuildTimeConstBuildItem item = new BuildTimeConstBuildItem(namespace, buildTimeData);
        buildTimeConstProducer.produce(item);

        String buildTimeDataImport = namespace + "-data";

        webJarBuildProducer.produce(WebJarBuildItem.builder()
                .artifactKey(UI_JAR)
                .root(DEVUI + "/")
                .filter(new WebJarResourcesFilter() {
                    @Override
                    public WebJarResourcesFilter.FilterResult apply(String fileName, InputStream file) throws IOException {

                        // If you want to use import { someKey } from `build-time-data`; you need this below:
                        if (fileName.endsWith(".js")) {
                            String content = new String(file.readAllBytes(), StandardCharsets.UTF_8);
                            content = content.replaceAll("build-time-data", buildTimeDataImport); // This part replace `build-time-data` with `io.quarkiverse.jokes.quarkus-jokes-common-data`
                            return new WebJarResourcesFilter.FilterResult(
                                    new ByteArrayInputStream(content.getBytes(StandardCharsets.UTF_8)), true);
                        }

                        return new WebJarResourcesFilter.FilterResult(file, false);
                    }
                }).build());
        devUIWebJarProducer.produce(new DevUIWebJarBuildItem(UI_JAR, DEVUI));
    }

All that the filter does is replace replace build-time-data with io.quarkiverse.jokes.quarkus-jokes-common-data.

Option 2: Just import the correct namespace

Like I said, the only reason the filter is there is to make life easier for extensions developers so they do not know about the namespace. However this case is fairly unique (the common module). So you can also just keep it simple and import the correct name in the js. So then your Java code stay the same (Just adding the build time data):

public class JokesCommonProcessor {
    private static final GACT UI_JAR = new GACT("io.quarkiverse.jokes", "quarkus-jokes-common", null, "jar");
    private static final String DEVUI = "dev-ui";

    @BuildStep(onlyIf = IsDevelopment.class)
    void createShared(BuildProducer<WebJarBuildItem> webJarBuildProducer,
            BuildProducer<DevUIWebJarBuildItem> devUIWebJarProducer,
            BuildProducer<BuildTimeConstBuildItem> buildTimeConstProducer) {

        String namespace = UI_JAR.getGroupId() + "." + UI_JAR.getArtifactId();

        // Build Time data (under the common namespace, so usable in the common
        // This will allow you to access someKey in js:
        
        Map<String, Object> buildTimeData = new HashMap<>();
        buildTimeData.put("someKey", "value from common"); // This value can be an object, as long as this can serialize to json

        BuildTimeConstBuildItem item = new BuildTimeConstBuildItem(namespace, buildTimeData);
        buildTimeConstProducer.produce(item);

        webJarBuildProducer.produce(WebJarBuildItem.builder()
                .artifactKey(UI_JAR)
                .root(DEVUI + "/")
                .filter(new WebJarResourcesFilter() {
                    @Override
                    public WebJarResourcesFilter.FilterResult apply(String fileName, InputStream file) throws IOException {
                        return new WebJarResourcesFilter.FilterResult(file, true);
                    }
                }).build());
        devUIWebJarProducer.produce(new DevUIWebJarBuildItem(UI_JAR, DEVUI));
    }
}

Now you use the full known name when importing:

import { someKey } from `io.quarkiverse.jokes.quarkus-jokes-common-data`;

Let me know if this works for you

[Dudeplayz]

Thanks for the detailed description. I have tried it now, but unfortunately it doesn't work fully. The build time data is accessible as expected, but I can't get the component link working on the card page of an extending module. If there is no card page for the the commons module the web component isn't loaded. It also doesn't appear in the browser as it does normally.

public class QuarkusHillaDevUICommonsProcessor {

    private static final GACT UI_JAR = new GACT("com.github.mcollovati", "quarkus-hilla-commons", null, "jar");
    private static final String NAMESPACE = UI_JAR.getGroupId() + "." + UI_JAR.getArtifactId();
    private static final String DEV_UI = "dev-ui";

    private static final DotName SIGNALS_HANDLER =
            DotName.createSimple("com.vaadin.hilla.signals.handler.SignalsHandler");

    @BuildStep(onlyIf = IsDevelopment.class)
    public EndpointBuildItem collectEndpoints(CombinedIndexBuildItem combinedIndexBuildItem) {
        final var endpointAnnotated =
                combinedIndexBuildItem.getComputingIndex().getAnnotations(EndpointInfo.ENDPOINT_ANNOTATION);
        final var browserCallableAnnotated =
                combinedIndexBuildItem.getComputingIndex().getAnnotations(EndpointInfo.BROWSER_CALLABLE_ANNOTATION);
        final var endpoints = Stream.concat(endpointAnnotated.stream(), browserCallableAnnotated.stream())
                .map(AnnotationInstance::target)
                .filter(target -> target.kind().equals(AnnotationTarget.Kind.CLASS))
                .map(AnnotationTarget::asClass)
                .filter(c -> !SIGNALS_HANDLER.equals(c.name()))
                .map ( e -> EndpointInfo.from(e, combinedIndexBuildItem.getIndex()))
                .toList();
        return new EndpointBuildItem(endpoints);
    }

    @BuildStep(onlyIf = IsDevelopment.class)
    public PageBuildItem pages(BuildProducer<CardPageBuildItem> producer, EndpointBuildItem endpointBuildItems) {
        CardPageBuildItem cardPageBuildItem = new CardPageBuildItem();
//        cardPageBuildItem.addBuildTimeData("hillaEndpoints", endpointBuildItems.getEndpoints());
        final var page = Page.webComponentPageBuilder()
                .title("Browser callables")
                .icon("font-awesome-solid:table-list")
                .componentLink("qwc-quarkus-hilla-browser-callables.js")
                .namespace(NAMESPACE)
                .staticLabel(String.valueOf(endpointBuildItems.getEndpoints().stream()
                        .mapToInt(a -> a.children().size())
                        .sum()));
//        cardPageBuildItem.addPage(page);
//        producer.produce(cardPageBuildItem);
        return new PageBuildItem(page);
    }

    @BuildStep(onlyIf = IsDevelopment.class)
    void createShared(BuildProducer<WebJarBuildItem> webJarBuildProducer,
                      BuildProducer<DevUIWebJarBuildItem> devUIWebJarProducer,
                      BuildProducer<BuildTimeConstBuildItem> buildTimeConstProducer,
                      EndpointBuildItem endpointBuildItem) {

        Map<String, Object> buildTimeData = new HashMap<>();
        buildTimeData.put("hillaEndpoints", endpointBuildItem.getEndpoints());
        buildTimeConstProducer.produce(new BuildTimeConstBuildItem(NAMESPACE, buildTimeData));

        String buildTimeDataImport = NAMESPACE + "-data";

        webJarBuildProducer.produce(WebJarBuildItem.builder()
                .artifactKey(UI_JAR)
                .root(DEV_UI + "/")
                .filter((fileName, file) -> {

                    // If you want to use import { someKey } from `build-time-data`; you need this below:
                    if (fileName.endsWith(".js")) {
                        String content = new String(file.readAllBytes(), StandardCharsets.UTF_8);
                        content = content.replaceAll("build-time-data", buildTimeDataImport); // This part replace `build-time-data` with `io.quarkiverse.jokes.quarkus-jokes-common-data`
                        return new WebJarResourcesFilter.FilterResult(
                                new ByteArrayInputStream(content.getBytes(StandardCharsets.UTF_8)), true);
                    }

                    return new WebJarResourcesFilter.FilterResult(file, false);
                }).build());

        devUIWebJarProducer.produce(new DevUIWebJarBuildItem(UI_JAR, DEV_UI));
    }
}

public class QuarkusHillaDevUIProcessor {

    @BuildStep(onlyIf = IsDevelopment.class)
    public void pages(PageBuildItem pageBuildItem, BuildProducer<CardPageBuildItem> cardPageProducer) {
        CardPageBuildItem cardPageBuildItem = new CardPageBuildItem();
        cardPageBuildItem.addPage(pageBuildItem.getPage());
        cardPageProducer.produce(cardPageBuildItem);
    }

}

I created a BuildItem to pass a pre-configured page from commons module to "normal" module. I am not sure if that is the desired way.

[phillip-kruger]

Yes, you can not add it to the card, you can add your own (very basic) page that imports that component and add it to the render. See my example here

[Dudeplayz]

public class QuarkusHillaDevUIProcessor {

    @BuildStep(onlyIf = IsDevelopment.class)
    public void pages(
            BuildProducer<CardPageBuildItem> cardPageProducer,
            EndpointBuildItem endpointBuildItems) {
        CardPageBuildItem cardPageBuildItem = new CardPageBuildItem();
        final var page = Page.webComponentPageBuilder()
                .title("Browser callables")
                .icon("font-awesome-solid:table-list")
                .componentLink("qwc-quarkus-hilla.js")
                .staticLabel(String.valueOf(endpointBuildItems.getEndpoints().stream()
                        .mapToInt(a -> a.children().size())
                        .sum()));
        cardPageBuildItem.addPage(page);
        cardPageProducer.produce(cardPageBuildItem);
    }
}

import { LitElement, html} from 'lit';
import './../com.github.mcollovati.quarkus-hilla-commons/qwc-quarkus-hilla-browser-callables.js';

export class QwcQuarkusHilla extends LitElement {

    render() {
        return html`
            <qwc-quarkus-hilla-browser-callables
                    namespace='com.github.mcollovati.quarkus-hilla'></qwc-quarkus-hilla-browser-callables>`;
    }
}
customElements.define('qwc-quarkus-hilla', QwcQuarkusHilla);

I updated it according to your example. But if I don't add a card page with the component from the commons extension, it doesn't work. If I add a card page, for the commons extension, then the code above works. Otherwise the qwc-quarkus-hilla-browser-callable.js isn't loaded.
I couldn't find the reason for that. Maybe because the commons is an extension itself, which is deployed and not just a maven module like in your example ?

[phillip-kruger]

Ok, let me change my common to be an extension and see if I can get to your issue. I'll do this tomorrow

[phillip-kruger]

Ok, it seems to be working for me even when I do this in a extension.

See attached example extension greeting-extension.zip

This has the same code as the common module in jokes (Just renamed to common). It does not add any cards of it's own, only the shared build time data and the javascript file (via WebjarBuildItem)

The only changes I made to Jokes are :

I added the dependency in both deployment and runtime:

     <dependency>
          <groupId>org.acme</groupId>
          <artifactId>greeting-extension-deployment</artifactId>
          <version>1.0.0-SNAPSHOT</version>
      </dependency>

in jokes deployment and

     <dependency>
          <groupId>org.acme</groupId>
          <artifactId>greeting-extension</artifactId>
          <version>1.0.0-SNAPSHOT</version>
      </dependency>

in jokes runtime.

Then I added this in jokes-menu.js:

import { LitElement, html, css} from 'lit';
import './../io.quarkiverse.jokes.quarkus-jokes-common/qwc-joke.js';
import './../org.acme.greeting-extension/qwc-common.js';

/**
 * This component shows how to add to the section menu
 */
export class QwcJokesMenu extends LitElement {
    
    render() {
        return html`<qwc-joke namespace='io.quarkiverse.jokes.quarkus-jokes'></qwc-joke>
                    <qwc-common namespace='io.quarkiverse.jokes.quarkus-jokes'></qwc-common>`;    
    }
}
customElements.define('qwc-jokes-menu', QwcJokesMenu);

So no you will see the joke from jokes common and from the greeting extension:

Let me know if you could get this to work on your side.

[Dudeplayz]

Oh man, thanks for the greetings example. With it I was able to figure out my "small" mistake:

Before:

private static final GACT UI_JAR = new GACT("com.github.mcollovati", "quarkus-hilla-commons", null, "jar");

After:

private static final GACT UI_JAR = new GACT("com.github.mcollovati", "quarkus-hilla-commons-deployment", null, "jar");

I missed to add the -deployment from the example before. I thought it's not required here.

Now everything works, also with unlisted extension. Thank you very much for your effort and time! 🙇‍♂️

After all this effort, I am still not sure if this could be any easier? For example just sharing the webcomponent from the commons extension to be used in the other extensions via a BuildItem?

[phillip-kruger]

Great ! I am glad this works for you. Yes this can definitely be made easier with a Build Item, but that would require some work and I wanted to get you working without having to wait for it. What we can do is open a new item on the Dev UI Ideas list (#35178) and if we get more people asking for this we can spend some time on it. At least for now you have a working way and your extensions can work on current and older version of Quarkus.

[phillip-kruger]

Closing here. We now have an item on the Dev UI Ideas list.

[Dudeplayz]

Thank you very much! Have a great time.