Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detect and follow Maven group and artifact moves #5667

Open
ChristianMurphy opened this issue Mar 8, 2020 · 18 comments · May be fixed by #32550
Open

Detect and follow Maven group and artifact moves #5667

ChristianMurphy opened this issue Mar 8, 2020 · 18 comments · May be fixed by #32550
Labels
priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others type:feature Feature (new functionality)

Comments

@ChristianMurphy
Copy link
Contributor

ChristianMurphy commented Mar 8, 2020

What would you like Renovate to be able to do?
Some maven artifacts change group and artifact ids on major releases.
Many projects include a link forward to the new location, for example the "Note: This artifact was moved to" sections of:

Describe the solution you'd like
When group and artifacts have their next major version linked in maven central, allow renovate updates to upgrade to the new version at the new group/artifact location.

Describe alternatives you've considered
N/A

Additional context

NOTE: this applies to all maven based package managers, including gradle and sbt

@rarkins
Copy link
Collaborator

rarkins commented Mar 8, 2020

Is essentially the same concept as #5535 ?

@ChristianMurphy
Copy link
Contributor Author

Similar concept, this would not be a manual rename, but could be automatically extracted from maven central.
More related to #2223

@ChristianMurphy
Copy link
Contributor Author

ran into this on uPortal-Project/SimpleContentPortlet#152

@rarkins
Copy link
Collaborator

rarkins commented Mar 8, 2020

OK, we would need to get that info out of the API somehow though.

@ChristianMurphy
Copy link
Contributor Author

It looks like this can be detected through the relocation metadata https://maven.apache.org/guides/mini/guide-relocation.html

@zharinov zharinov self-assigned this Mar 9, 2020
@rarkins
Copy link
Collaborator

rarkins commented Mar 9, 2020

OK, waiting on #2223

@rarkins rarkins added blocked type:feature Feature (new functionality) priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others labels Mar 9, 2020
@rarkins rarkins removed the blocked label Jun 18, 2020
@rarkins rarkins added status:blocked Issue is blocked by another issue or external requirement status:in-progress Someone is working on implementation and removed status:in-progress Someone is working on implementation labels Jan 12, 2021
@froque
Copy link
Contributor

froque commented Feb 5, 2021

There is a maven plugin, oga-maven-plugin, to detect these artifact moves.

It works with a curated JSON file: https://github.com/jonathanlermitage/oga-maven-plugin/blob/master/uc/og-definitions.json

@nielsbasjes
Copy link
Contributor

Yesterday (almost) all of my projects got the same update from renovate that broke the CI build.

Update dependency pl.project13.maven:git-commit-id-plugin to v4.9.9

all of them failed with

Error:  Plugin pl.project13.maven:git-commit-id-plugin:4.9.9 or one of its dependencies could not be resolved: Could not find artifact io.github.git-commit-id:git-commit-id-maven-plugin:jar:4.9.9 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

The rootcause was that this plugin got moved to a different package.
The people who build this software did this in exactly the way Maven tells you:

https://maven.apache.org/guides/mini/guide-relocation.html

They created a 4.9.9 version in the old package which is ONLY a pom.xml with a redirect and does not contain any jar/war/... files. The redirect in this case looks like this:

<distributionManagement>
  <relocation>
    <groupId>io.github.git-commit-id</groupId>
    <artifactId>git-commit-id-maven-plugin</artifactId>
  </relocation>
</distributionManagement>

The effect can for example be seen here
https://search.maven.org/artifact/pl.project13.maven/git-commit-id-plugin
which shows:

Relocated to	io.github.git-commit-id:git-commit-id-maven-plugin

@rarkins since this is apparently the standard way of doing this I would really like this maven standard way to be supported by renovate.

@rarkins rarkins added the auto:reproduction A minimal reproduction is necessary to proceed label Jul 12, 2021
@github-actions
Copy link
Contributor

Hi there,

Help us by making a minimal reproduction repository.

Before we can start work on your issue we first need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction to understand what is needed.

We may close the issue if you (or someone else) have not provided a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

@rarkins
Copy link
Collaborator

rarkins commented Jul 12, 2021

Can you create a minimal reproduction of that case? We are still blocked on a generic way to "replace" packages though

@nielsbasjes
Copy link
Contributor

nielsbasjes commented Jul 12, 2021

@rarkins
This is the smallest I could think of (including renovate and a failing CI build):
https://github.com/nielsbasjes/Renovate5667/
nielsbasjes/Renovate5667#3

The full error here is

[INFO] Scanning for projects...
[INFO] 
[INFO] -----------------< nl.basjes.bugreports:renovate5667 >------------------
[INFO] Building renovate5667 0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/pl/project13/maven/git-commit-id-plugin/4.9.9/git-commit-id-plugin-4.9.9.pom
[INFO] Downloaded from central: https://repo.maven.apache.org/maven2/pl/project13/maven/git-commit-id-plugin/4.9.9/git-commit-id-plugin-4.9.9.pom (1.4 kB at 1.3 kB/s)
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/pl/project13/maven/git-commit-id-plugin-parent/4.9.9/git-commit-id-plugin-parent-4.9.9.pom
[INFO] Downloaded from central: https://repo.maven.apache.org/maven2/pl/project13/maven/git-commit-id-plugin-parent/4.9.9/git-commit-id-plugin-parent-4.9.9.pom (2.5 kB at 189 kB/s)
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/io/github/git-commit-id/git-commit-id-maven-plugin/4.9.9/git-commit-id-maven-plugin-4.9.9.pom
Warning:  The POM for io.github.git-commit-id:git-commit-id-maven-plugin:jar:4.9.9 is missing, no dependency information available
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/io/github/git-commit-id/git-commit-id-maven-plugin/4.9.9/git-commit-id-maven-plugin-4.9.9.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.388 s
[INFO] Finished at: 2021-07-12T15:41:42Z
[INFO] ------------------------------------------------------------------------
Error:  Plugin pl.project13.maven:git-commit-id-plugin:4.9.9 or one of its dependencies could not be resolved: Could not find artifact io.github.git-commit-id:git-commit-id-maven-plugin:jar:4.9.9 in central (https://repo.maven.apache.org/maven2) -> [Help 1]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
Error: Process completed with exit code 1.

@github-actions
Copy link
Contributor

Thank you for providing a reproduction! 🎉 🚀

The Renovate team will take a look at the reproduction repository.

@TheSnoozer
Copy link

TheSnoozer commented Jul 15, 2021

Hello,
I'm one of the remaining maintainers of the git-commit-id-plugin project.
Let me start with that the maven relocation guide really needs improvement. Whatever I have done or messed up, please do not treat it as "standard". I really have no idea what I'm doing this was my first "official" and perhaps last relocation.

IMHO I just wanted to make users aware that the plugin moved to new coordinates and thus published a relocation pom only. I intentionally did not publish a jar since I wanted to keep the option open to publish hotfix releases with old coordinates (I really can't be asked to change the coordinates for old releases).
As a result my perhaps stupid idea was to publish a "4.9.9"-release containing the relocation info only. Under the new coordinates only a "5.0.0" exists. Perhaps one could resolve the "can't find artifact"-issue if I would publish a 4.9.9 under the new coordinates.

If this works (now confirmed) then the relocation can be one of the following publication-types:

  1. A (full) publication with a jar and a normal pom stating the relocation (see below).
  2. A pom-only publication (no dependencies declared, essentially a naked pom as I have had it published) pointing to the new location, however the jar is residing under the new location

Edit: I can confirm that after publishing the plugin to the new coordinates (git-commit-id/git-commit-id-maven-plugin#570) one can use the old artifact/group combination or new.
So in the case of 2 the relocation acts as "symlink" or whatever you want to call it.

I don't know what would happen if one published the jar to the old coordinates and the new coordinates....

Note: The relocation itself can be boiled down to the following snippet:

<distributionManagement>
  <relocation>
    <groupId>io.github.git-commit-id</groupId>
    <artifactId>git-commit-id-maven-plugin</artifactId>
  </relocation>
</distributionManagement>

For the full pom refer to https://repo1.maven.org/maven2/pl/project13/maven/git-commit-id-plugin/4.9.9/git-commit-id-plugin-4.9.9.pom

@astellingwerf
Copy link
Collaborator

Now that #5558 is merged into the trunk, I'd like to take a stab at this one. I had already worked on this this summer, but @JamieMagee was not fully done with the replacement impl back then.

@astellingwerf
Copy link
Collaborator

Watchers, can you guide me a bit, as I'm not too familiar with the standard development practices in this community?

There are the datasource aspects of this task, and then there are the managers. The datasource was easy to get done, but now that it detects the replacement, none (but the NPM changes set up by @JamieMagee) of the managers know how to handle it. The DD will propose to create a PR, but the manager is gonna ignore the replacement name while generating the proposed new code.

What's the default approach for phasing in such changes? Would it make sense to add a boolean property supportsReplacements to the ManagerApi interface that declares the support for replacing dependencies?

@rarkins
Copy link
Collaborator

rarkins commented Nov 15, 2021

Yes, could be good to add that flag and log a warning if we find an attempted replacement for a manager which doesn't support it.

@JamieMagee
Copy link
Contributor

I'm not sure we require a new flag. I was looking into this for Docker, and this is the key codepath:

const updateDependency = get(manager, 'updateDependency');
if (!updateDependency) {
let res = await doAutoReplace(
upgrade,
packageFileContent,
reuseExistingBranch
);

If a manager has a custom updateDependency method defined, we use that, otherwise use doAutoReplace. In the case of npm and maven managers, they do have a custom updateDependency. That's where you'll need to check if upgrade.newName is defined and handle it.

export function updateDependency({
fileContent,
upgrade,
}: UpdateDependencyConfig): string | null {
const offset = fileContent.indexOf('<');
const spaces = fileContent.slice(0, offset);
const restContent = fileContent.slice(offset);
const updatedContent = updateAtPosition(restContent, upgrade, '</');
if (!updatedContent) {
return null;
}
if (updatedContent === restContent) {
return fileContent;
}
return `${spaces}${updatedContent}`;
}

That should be the only change required for this issue.

@rarkins
Copy link
Collaborator

rarkins commented Nov 16, 2021

FWIW (not related to Maven) I'm assuming that autoReplace could be enhanced to support a replacement name in a similar way it today can support a new value/version. I would suggest we do them as two separate routines, i.e. don't try to replace both package name and version at once.

@rarkins rarkins removed the status:blocked Issue is blocked by another issue or external requirement label Oct 1, 2023
jonasrutishauser added a commit to jonasrutishauser/renovate that referenced this issue Nov 14, 2024
jonasrutishauser added a commit to jonasrutishauser/renovate that referenced this issue Nov 14, 2024
@jonasrutishauser jonasrutishauser linked a pull request Nov 14, 2024 that will close this issue
6 tasks
jonasrutishauser added a commit to jonasrutishauser/renovate that referenced this issue Nov 20, 2024
jonasrutishauser added a commit to jonasrutishauser/renovate that referenced this issue Nov 20, 2024
jonasrutishauser added a commit to jonasrutishauser/renovate that referenced this issue Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others type:feature Feature (new functionality)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants