[Features] Leaverage transitive inclusion and build source-features #20

HannesWell · 2022-04-20T22:21:46Z

This PR proposes to build source-features for all non-sdk features of equinox-bundles and to simplify the feature content by leveraging transitive inclusion and the now generated source features.

Leveraging transitive inclusion avoids effective duplicates and reduces the maintenance effort. Furthermore it becomes more unlikely to forget to include a source-plugin. My intention is also to reduce the subsequent maintenance effort of eclipse-equinox/equinox.framework#41.

Besides that, it removes the .feature suffix from the project-name org.eclipse.equinox.server.core.feature to match the feature name.

HannesWell · 2022-04-20T22:31:30Z

The initial target-platform resolution fails with the following errors:

[INFO] {osgi.os=linux, osgi.ws=gtk, org.eclipse.update.install.features=true, osgi.arch=x86_64}
[ERROR] Cannot resolve project dependencies:
[ERROR]   Software being installed: org.eclipse.equinox.sdk.feature.group 3.23.400.qualifier
[ERROR]   Missing requirement: org.eclipse.equinox.core.sdk.feature.group 3.23.400.qualifier requires 'org.eclipse.equinox.p2.iu; org.eclipse.equinox.core.feature.source.feature.group 0.0.0' but it could not be found
[ERROR]   Cannot satisfy dependency: org.eclipse.equinox.sdk.feature.group 3.23.400.qualifier depends on: org.eclipse.equinox.p2.iu; org.eclipse.equinox.core.sdk.feature.group 0.0.0
[ERROR] 
[ERROR] See https://wiki.eclipse.org/Tycho/Dependency_Resolution_Troubleshooting for help.

[INFO] Resolving dependencies of MavenProject: org.eclipse.equinox.feature:org.eclipse.equinox.core.sdk:3.23.400-SNAPSHOT @ /home/jenkins/agent/workspace/rt.equinox.bundles_PR-20/features/org.eclipse.equinox.core.sdk/pom.xml
[INFO] {osgi.os=linux, osgi.ws=gtk, org.eclipse.update.install.features=true, osgi.arch=x86_64}
[ERROR] Cannot resolve project dependencies:
[ERROR]   Software being installed: org.eclipse.equinox.core.sdk.feature.group 3.23.400.qualifier
[ERROR]   Missing requirement: org.eclipse.equinox.core.sdk.feature.group 3.23.400.qualifier requires 'org.eclipse.equinox.p2.iu; org.eclipse.equinox.core.feature.source.feature.group 0.0.0' but it could not be found
[ERROR] 
[ERROR] See https://wiki.eclipse.org/Tycho/Dependency_Resolution_Troubleshooting for help.

[INFO] Resolving dependencies of MavenProject: org.eclipse.equinox.feature:org.eclipse.equinox.serverside.sdk:3.23.400-SNAPSHOT @ /home/jenkins/agent/workspace/rt.equinox.bundles_PR-20/features/org.eclipse.equinox.serverside.sdk/pom.xml
[INFO] {osgi.os=linux, osgi.ws=gtk, org.eclipse.update.install.features=true, osgi.arch=x86_64}
[ERROR] Cannot resolve project dependencies:
[ERROR]   Software being installed: org.eclipse.equinox.serverside.sdk.feature.group 3.23.400.qualifier
[ERROR]   Missing requirement: org.eclipse.equinox.serverside.sdk.feature.group 3.23.400.qualifier requires 'org.eclipse.equinox.p2.iu; org.eclipse.equinox.server.core.source.feature.group 0.0.0' but it could not be found
[ERROR] 
[ERROR] See https://wiki.eclipse.org/Tycho/Dependency_Resolution_Troubleshooting for help.

I'm puzzled why this fails. @laeubi @mickaelistria do you have any idea?
In m2e we use the same pattern for the m2e.sdk feature that also includes source-features that are generated during build:
https://github.com/eclipse-m2e/m2e-core/blob/master/org.eclipse.m2e.sdk.feature/feature.xml

The only difference I see is that Tycho-pomless is not (yet but I'm working on it) used in equinox-bundles and the features of their own pom.xml.

HannesWell · 2022-04-20T23:09:16Z

Adding the following configuration to the tycho-p2-plugin did not help either (I didn't expect much, but hope dies last).

        <configuration>
          <defaultP2Metadata>false</defaultP2Metadata>
          <baselineReplace>common</baselineReplace>
        </configuration>

But I think I have to add <baselineReplace>common</baselineReplace> anyway until it has been added to the base-line.

laeubi · 2022-04-21T05:11:58Z

I have no idea, but would start with the bare minimum and only remove the source plugins/features instead of any "transitive" for me this is causing more trouble as is really solves and the real issue we should care is eclipse-tycho/tycho#898

merks · 2022-04-21T07:14:29Z

I've run into problems similar to this too. I think it's a bit of a chicken and egg problem because the source is generated so is not available up front somehow. It seems to me I had to do this to make the problem go away at the root level.

https://git.eclipse.org/c/oomph/org.eclipse.oomph.git/tree/features/org.eclipse.oomph.all-feature/p2.inf

This is the particular feature that strictly controls what is and is not in Oomph's update site via transitive inclusion...

Let me look if there are any other things I can think of...

merks · 2022-04-21T07:18:59Z

I assume that this is configured for these things in the right places:

  <build>
    <plugins>
      <plugin>
        <groupId>org.eclipse.tycho.extras</groupId>
        <artifactId>tycho-source-feature-plugin</artifactId>
        <executions>
          <execution>
            <id>generate-source-feature</id>
            <goals>
              <goal>source-feature</goal>
            </goals>
            <configuration>
              <excludes>
                <plugin id="org.apache.commons.codec"/>
                <plugin id="org.apache.commons.logging"/>
                <plugin id="org.apache.httpcomponents.client5.httpclient5"/>
                <plugin id="org.apache.httpcomponents.core5.httpcore5"/>
                <plugin id="org.apache.httpcomponents.core5.httpcore5-h2"/>
                <plugin id="org.apache.httpcomponents.httpclient"/>
                <plugin id="org.apache.httpcomponents.httpcore"/>
                <plugin id="org.bouncycastle.bcpg"/>
                <plugin id="org.bouncycastle.bcprov"/>
                <plugin id="org.slf4j.api"/>
                <plugin id="org.eclipse.nebula.widgets.tablecombo"/>
                <plugin id="org.eclipse.oomph.releng.helpcenter"/>
              </excludes>
            </configuration>
          </execution>
        </executions>
      </plugin>

HannesWell · 2022-04-21T07:39:32Z

Thanks for your help.
I will look into this tomorrow, for today I unfortunately have no time left.

HannesWell · 2022-04-23T21:32:24Z

The first reason of the failure was that Tycho requires a specific execution-id for the tycho-source-plugin:feature-source goal in order to consider the source-features that will be generated during the initial dependency resolution.
I have already created a PR to improve that: eclipse-tycho/tycho#912

A second reason this is failing is that the tycho-p2-plugin is removing source-features that are not present in the baseline. I hope this is solved with corresponding enforced qualifier updates.

Signed-off-by: Hannes Wellmann <[email protected]>

HannesWell · 2022-04-23T21:55:08Z

Great, that worked well.
I just reverted the unnecessary qualifier updates for the o.e.equinox.console.jaas.fragment (it doesn't need a source bundle) as well as the unnecessary version bumps of the o.e.equinox.core.feature.
The the remaining enforced qualifier updates are necessary because the p2-plugins baseline is the latest I-build repo. So in order not get the use the newly introduced source features removed their build qualifier has to be updated.

The new source features are:

org.eclipse.equinox.core.feature.source
org.eclipse.equinox.server.core.source
org.eclipse.equinox.server.jetty.source
org.eclipse.equinox.server.p2.source

HannesWell · 2022-04-24T09:21:04Z

I double checked that no plug-ins are effectively removed, so I think this is ready for submission.

HannesWell · 2022-04-24T09:26:28Z

I assume that this is configured for these things in the right places:
      <plugin>
        <groupId>org.eclipse.tycho.extras</groupId>
        <artifactId>tycho-source-feature-plugin</artifactId>
      </plugin>

@merks the tycho-source-feature-plugin is deprecated and I assume it is will be removed in Tycho 3.0.0 (@laeubi can you confirm this?).
In Tycho 2.7.2/3.0.0 you can replace it with the org.eclipse.tycho:tycho-source-plugin and its feature-source goal.
For older versions of Tycho you have to make sure the goal's execution has the id feature-source. :)

merks · 2022-04-24T09:30:44Z

@HannesWell Yes, I remember updating some of my builds to use the new one, but the Oomph build still uses the old one.

There are still these uses in the platform repos:

And I thought the builds were being done with some 3.0.* version of Tycho...

HannesWell · 2022-04-24T15:09:45Z

Thanks for the hints.
I just created the following PRs to address this:

And I thought the builds were being done with some 3.0.* version of Tycho...

That's right, the tycho-source-feature-plugin is not yet removed. But I expect it will be removed before the 3.0.0 release. At least a new major version would be a good opportunity to remove deprecated code. But I don't know if there are reasons against a removal or if it just was not done yet, maybe the remaining usages are the 'blocker'?

laeubi · 2022-04-24T15:19:45Z

But I don't know if there are reasons against a removal or if it just was not done yet, maybe the remaining usages are the 'blocker'?

No, there is a clear warning and it is marked for removal in 3.x and there is a replacement for it so no need to retain it.

HannesWell mentioned this pull request Apr 22, 2022

Replace content of o.e.osgi.util by osgi-bundles from Maven-Central eclipse-equinox/equinox.framework#41

Merged

HannesWell force-pushed the simplifyFeatures branch from 130702c to 0cb1784 Compare April 23, 2022 20:50

HannesWell mentioned this pull request Apr 23, 2022

Make checks for 'feature-source'/'p2-metadata' executions more precise eclipse-tycho/tycho#912

Merged

HannesWell force-pushed the simplifyFeatures branch from 0cb1784 to 703f9ff Compare April 23, 2022 21:23

[Features] Leverage transitive inclusion and build source-features

24313ff

Signed-off-by: Hannes Wellmann <[email protected]>

HannesWell force-pushed the simplifyFeatures branch from 703f9ff to 24313ff Compare April 23, 2022 21:49

HannesWell merged commit 240b319 into eclipse-equinox:master Apr 24, 2022

HannesWell deleted the simplifyFeatures branch April 24, 2022 09:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Features] Leaverage transitive inclusion and build source-features #20

[Features] Leaverage transitive inclusion and build source-features #20

HannesWell commented Apr 20, 2022 •

edited

Loading

HannesWell commented Apr 20, 2022

HannesWell commented Apr 20, 2022

laeubi commented Apr 21, 2022

merks commented Apr 21, 2022

merks commented Apr 21, 2022

HannesWell commented Apr 21, 2022

HannesWell commented Apr 23, 2022

HannesWell commented Apr 23, 2022 •

edited

Loading

HannesWell commented Apr 24, 2022

HannesWell commented Apr 24, 2022

merks commented Apr 24, 2022

HannesWell commented Apr 24, 2022 •

edited

Loading

laeubi commented Apr 24, 2022

[Features] Leaverage transitive inclusion and build source-features #20

[Features] Leaverage transitive inclusion and build source-features #20

Conversation

HannesWell commented Apr 20, 2022 • edited Loading

HannesWell commented Apr 20, 2022

HannesWell commented Apr 20, 2022

laeubi commented Apr 21, 2022

merks commented Apr 21, 2022

merks commented Apr 21, 2022

HannesWell commented Apr 21, 2022

HannesWell commented Apr 23, 2022

HannesWell commented Apr 23, 2022 • edited Loading

HannesWell commented Apr 24, 2022

HannesWell commented Apr 24, 2022

merks commented Apr 24, 2022

HannesWell commented Apr 24, 2022 • edited Loading

laeubi commented Apr 24, 2022

HannesWell commented Apr 20, 2022 •

edited

Loading

HannesWell commented Apr 23, 2022 •

edited

Loading

HannesWell commented Apr 24, 2022 •

edited

Loading