How to allow custom resolvers that depend on external credentials.

[megri]

The initial conversation took place on Discord:

megri:
I'm rewriting a SBT-build in bleep and am stuck on how to work with resolvers. I didn't find any information on how to actually add them in the docs but I think I figured it out by looking at code and issues. My current problem is that I need to add a resolver for google artifact registry, which uses oauth2 for authentication. Basically I need to grab a somewhat short-lived token on each invocation and pass that. Should I use a build rewrite for this and if so, how can I make it apply automatically whenever I run my build?

oyvindberg:
Oh yeah, apologies - I'm behind on documenting resolvers! This particular thing I have no idea honestly, afaiu bleep just supports normal credentials. How do you use this repository in SBT? Does it work with coursier?

megri:
Coursier doesn't have native support for it. What I'm doing is basically using the Google artifact-registry auth library in a plugin (mill/sbt) to grab the default credentials and use those, while registering the registry.

I imagine the same thing could be done in bleep with a build rewrite thingy, but it'd need to happen on normal builds or else I imagine users would have to run a specialised build script.

oyvindberg:
A build rewrite likely won't help, unfortunately. That operates at the build definition level, while authentication lives at the user-level in a config file. A hack could be to write a script which downloads the artifacts to Coursier/ivy cache. Otherwise you could extend bleep with first-class support for it. I won't have time to look into it, but could provide tutoring

Summarising, the issue is that bleep doesn't support custom authorisation methods.

From what I've understood from the sources, build rewrites operate on a project-level, whereas the resolver configurations live one level above, at the build-level. As the signature of BuildRewrite#newExplodedProjects is (oldBuild: model.Build, buildPaths: BuildPaths): Map[model.CrossProjectName, model.Project] there doesn't seem to be a way to modify oldBuild itself.

The question then becomes how to fit this into the bleep lifecycle.

The first approach I came to think of was to offer a new rewrite option, like newExplodedBuild(oldBuild: model.Build, buildPaths: BuildPaths): (model.Build, Map[model.CrossProjectName, model.Project]), but I believe script/build rewrites happen after the build has been processed and dependencies downloaded. This creates a catch 22 where the build may rely on artefacts only available in the protected repository, and so will fail resolution which stops the build rewrite from ever happening.

Another approach would be to include the Google artifact-registry dependencies in bleep itself and add a schema configuration option for resolvers, letting the user choose among the various alternatives of authorisation. I don't think this approach is a good one as this would bind one version of bleep to one version of the dependencies from Google. It would also not solve the problem for other vendors' artefact registry solutions.

The approach taken by both mill and SBT is to introduce a meta-build step, allowing you to influence the actual build before it has been processed. This is not commonly used for adding resolvers in those tools, but could be a workable solution for bleep.

[oyvindberg]

Ok, I thought about it a bit. bleep is just the mashup of coursier, bloop, yaml and native-image. almost nothing more. support for this should be added to coursier, not to bleep. that way scala-cli and friends can benefit from it as well.

[i10416]

I'm interested in custom resolvers that depend on external credentials too.

AFAIK, with coursier, you can fetch dependencies hosted on GCP Artifact Registry using https schema instead of artifactregistry in the following way, where _json_key_base64 is base64-encoded service account key json file content.

Developers can also use oauth2accesstoken:$TOKEN instead of _json_key_base64:*****.

See the comment(GoogleCloudPlatform/artifact-registry-maven-tools#71 (comment)).

coursier fetch -r https://$REGION-maven.pkg.dev/$GCP_PROJECT_ID/$REPOSITORY \
    --credentials "$REGION-maven.pkg.dev _json_key_base64:***************" \
   "com.example:example_3:0.1.0"

There is also coursier-publish module to publish artifacts to repositories. It seems Scala CLI and mill use this module for publishing.

• https://github.com/coursier/publish
• https://scala-cli.virtuslab.org/docs/reference/password-options

Scala CLI's password-options is an approach similar to Cargo's credential-provider-protocol.

https://doc.rust-lang.org/cargo/reference/credential-provider-protocol.html

According to coursier docs, the coursier credentials can be set via environment variables or Java properties.
https://get-coursier.io/docs/other-credentials

One easier solution for bleep would be to allow COURSIER_CREDENTIALS environment variable to propagate from bleep process to coursier in the same way as Scala CLI.
https://scala-cli.virtuslab.org/docs/guides/power/repositories/

Another is to add auth options to repository types

bleep/bleep-model/src/scala/bleep/model/Repository.scala

Line 11 in dd55f1b

object Repository {