API to push artifacts to check

[juergen-albert]

As discussed at the EclipseCon:

We have Opensource Projects, that are no Eclipse Projects but that are used in e.g. the Eclipse Sensinact Project. Thus we are looking for a possibility to get our new Releases checked proactively during our automated release Process.

As the IP Dash takes maven coordinates to maven central we can't really use this either, because we first upload to the sonartype staging repo and the sync to maven central may take an undetermined amount of time.

Creating gitlab issue manually would be a bit annoying, so some kind of API where we can hand over our jar with the sources in it would be great.

[waynebeaton]

The Dash License Tool uses the standard GitLab API to create issues. If you make an issue in the correct form, it will "just work".

Theoretically, we could extend the scanner to look at the staging repo for sources. We can easily add new Nexus repositories if we know the base artifacts URL. The problem with this is that the content in the staging repository isn't necessarily stable. In fact, it disappears by design.

It occurs to me that the source stability problem actually extends to any content for which we pull sources from a staging server: whether the corresponding issues are automatically created or created via some API or automation.

Pointing to staged artifacts that are inevitably removed is actually an antipattern.

[juergen-albert]

I understand where you are coming from, together with the issue you see.

My main issue, that I want to make the acceptance by the Dash Tool a quality gate. Right now I have to release something and then I will find out if you are happy. Additionally anyone who uses our new Version for an Eclipse Porject will need to do a more or less manual step, before their project is accepted by Eclipse process for release.

Just for my understanding: After an artifact is checked and deemed acceptable, I guess you will identify it later via its coordinates in a stable Repository (like Maven Central)?

My main issue is, with an artifact I deploy to Maven Central:

1. I build and release it to sonartype staging (automated).
2. Then I already have to manually go to the sonartype repo.
3. Close it.
4. Wait till the checks are green.
5. Mark it for release.
6. Pray and wait (as sometimes it does not publish it, even if all the checks are green).

Now I would need to add another manual step, which is only possible after the Artifact is really synced to maven central and trigger the dash process.

Right now, I don't have a good idea how an acceptable solution could look like. :-(
Ideas?

[waynebeaton]

Just for my understanding: After an artifact is checked and deemed acceptable, I guess you will identify it later via its coordinates in a stable Repository (like Maven Central)?

Yes.

Random idea...

If you were to create an issue that points to a commit on a GitHub repository, it will scan a snapshot based on that.

e.g., git/github/eclipse/dash-licenses/65decbd4f82198acd764737166f11edc3ecbe60d

IPLab will grab a ZIP of the github.com/eclipse/dash-licenses repository at the specified commit and scan that.

The coordinates that we stuff into our database would be correct, but not useful enough to be reused.

[waynebeaton]

I'll note that the problem that we're trying to solve here is not "make the Dash License Tool happy", but to more generally improve the odds that any attempts to automate license scanning will be successful. Anything that you can do to make your project's license intention more consistently discoverable via automation will be a net benefit for every adopter.

The IPLab scanner uses Scancode Toolkit. Scancode identifies licenses, it doesn't make any decisions. Our scanner interprets the results produced by Scancode, but doesn't do anything particularly magical or complicated. The one big optimisation that we do is take a license declaration made with an SPDX-License-Identifier in the header at face value and ignore any other license information that it finds in the file.

So... if you put the SPDX-License-Identifier header in every file, then you'd be good-to-go. Alternatively, if you at least put that header in the files that make references to other licenses, most of the problem would be addressed (BND, for example, has a bunch of files that make solid references to all manner of licenses).

[waynebeaton]

If, for example, you added a license header with an SPDX-License-Identifier tag to the top of the license files in bndtools/bnd (e.g., GPL_2_0.java), then automated tools would have a fighting chance of understanding the license of the file.

I believe that Scancode is getting fetched up on the GPL reference in this comment:

/**
 * An annotation to indicate that the type depends on the GNU General Public
 * License v2.0. Applying this annotation will add a Bundle-License clause.
 *
 * @deprecated Replaced by {@link GPL_2_0_only} or {@link GPL_2_0_or_later}.
 */

While this file does represent the GPL, it is not itself distributed under the GPL. If you express your intention unambiguously with an SPDX-License-Identifier then Scancode, Fossology, FOSSA, ... will have a fighting chance of making sense of it without human intervention.

FYI, I've added a rule to the IPLab scanner to ignore that directory. Making good resilient rules is actually pretty hard in the general case, so this is something that we do very carefully.

[waynebeaton]

Returning back to the original request...

You can create an IPLab issue with an attachment.

[waynebeaton]

Any thoughts on any of the above, @juergen-albert?