multiple aqua registries - Feature parity with Aqua

[dudicoco]

Hi,

While trying to migrate to Mise from Aqua + Direnv, I've come across a few issues with features that exist in Aqua but not in Mise.

• Support multiple registries: this is the most important feature that is missing. Within aqua.yaml you can define multiple registries including a local registry, so if you want to install a package that does not exist in the aqua standard registry you can define it locally: https://aquaproj.github.io/docs/develop-registry/#use-as-a-local-registry
• aqua cp: this command allows you to copy the executable to a target directory, which makes installing packages in a multi-stage dockerfile easy: https://aquaproj.github.io/docs/guides/build-container-image/
• aqua update-aqua: this command updates aqua to a different version. Unlike mise self-update, aqua update-aqua will keep multiple downloaded versions of aqua locally and symlink the one specified. This allows you to switch between different versions of aqua in different projects seamlessly without redownloading the binary every time.

Note: a better user experience here would be to automatically use the version of mise specified in mise.toml

• Renovate updates: aqua allows updating the entire "toolchain" of aqua with renovate.
  • aqua version
  • aqua installer version
  • renovate extension version
  • aqua packages versions
  • aqua registry version
• aqua g : fuzzy searches for packages and automatically adds them to the config file with the --install flag. Current workaround with mise - mise use --pin $(mise registry -b aqua | fzf --multi | awk '{print $2}').

Note: this workaround will also install the packages while aqua g -i only adds them to the config file. Nice to have but perhaps a flag could be added to decide if the package should be installed or just added to .mise.toml.

• mise doesn't support pkg format:

INFO  aqua:aws/[email protected]   generate checksum AWSCLIV2-2.22.7.pkg
INFO  aqua:aws/[email protected]   extract AWSCLIV2-2.22.7.pkg
Error:
   0: failed to install aqua:aws/[email protected]
   1: unsupported format: pkg

Location:
   src/backend/aqua.rs:494

Version:
   2024.11.37 macos-arm64 (7ac5ab3 2024-11-30)

• Filter packages with tags: aqua allows you to add tags to packages and specify tags when installing packages.

[jdx]

Support multiple registries: this is the most important feature that is missing. Within aqua.yaml you can define multiple registries including a local registry, so if you want to install a package that does not exist in the aqua standard registry you can define it locally: https://aquaproj.github.io/docs/develop-registry/#use-as-a-local-registry

I think we should support this. Right now you can override the entire registry but we lack a way to provide multiple which would be useful.

aqua cp: this command allows you to copy the executable to a target directory, which makes installing packages in a multi-stage dockerfile easy: https://aquaproj.github.io/docs/guides/build-container-image/

I don't think this needs a dedicated command. You can do this like this already which I think is fine:

cp $(mise which node) /tmp/node

There is also an existing ticket for #1039 which is probably better for the multi-stage docker use-case anyhow.

aqua update-aqua: this command updates aqua to a different version. Unlike mise self-update, aqua update-aqua will keep multiple downloaded versions of aqua locally and symlink the one specified. This allows you to switch between different versions of aqua in different projects seamlessly without redownloading the binary every time.

I don't think I understand the value of being able to have different mise versions side-by-side. This is not how mise is designed and doing so is not only a lot of work but would involve a lot of future maintenance burden to keep that a supported use-case. I don't believe anyone has ever asked for the ability to run different versions of mise together, they just need a way to downgrade mise globally to prevent breaking changes or bugs when the occur.

Renovate updates: aqua allows updating the entire "toolchain" of aqua with renovate.

I don't use renovate with mise or maintain that integration. As I understand it renovate does support mise already but any requests for improvements there would likely need to go to them.

aqua g : fuzzy searches for packages and automatically adds them to the config file with the --install flag. Current workaround with mise - mise use --pin $(mise registry -b aqua | fzf --multi | awk '{print $2}').

this is mentioned on the roadmap

mise doesn't support pkg format:

this can be added, we already extract pkg files for installing swift so the logic just needs to be extended to aqua

Filter packages with tags: aqua allows you to add tags to packages and specify tags when installing packages.

I think MISE_ENV covers this use-case in a different way but one that seems to be sufficient since nobody has asked for this before I don't think.

[dudicoco]

I don't think I understand the value of being able to have different mise versions side-by-side. This is not how mise is designed and doing so is not only a lot of work but would involve a lot of future maintenance burden to keep that a supported use-case. I don't believe anyone has ever asked for the ability to run different versions of mise together, they just need a way to downgrade mise globally to prevent breaking changes or bugs when the occur.

I believe there is a big value in having a locked version of Mise in each repo, just like there is value in locking all of the packages that Mise installs.

For example, if you've intrduced a new feature in a new version and also deprecated an existing feature in the same release, now the new version cannot be used globally - if I have one project which needs the new feature and one project that needs the deprecated feature as it didn't migrate from it yet.
You could say that they have to align with each other but in reality different projects use different versions of tools and move in different paces (unless you're in a monorepo, which would be ideal :-) )

mise doesn't support pkg format:

this can be added, we already extract pkg files for installing swift so the logic just needs to be extended to aqua

Should I open a separate issue for this?

Filter packages with tags: aqua allows you to add tags to packages and specify tags when installing packages.

I think MISE_ENV covers this use-case in a different way but one that seems to be sufficient since nobody has asked for this before I don't think.

This is different than the tags feature. Aqua has a similar feature which allows you to split the config files: https://aquaproj.github.io/docs/guides/split-config/

The tags feature augments this as you can specify a tag across all files and choose to install only those packages,

[jdx]

deprecated an existing feature in the same release

I don't believe I ever do this. Once a feature comes out of experimental it stays forever. If this ever were to be done it would involve a long migration path with warnings so that people have plenty of time to migrate without needing to have separate versions installed.

If you're talking about experimental features, well that comes with the territory. Wait until things become stable before relying on them.

The tags feature augments this as you can specify a tag across all files and choose to install only those packages,

The only use-case I can think of that tags would cover is you could use multiple envs, e.g.: MISE_ENV=ci,test which isn't supported today but we should add. I can't think of a use-case where tags would solve something MISE_ENV wouldn't.

[jdx]

I added some of these, namely MISE_ENV=test,ci and pkg support for aqua. I think the only remaining thing we should pick up here is multiple aqua registries, which we should do, but I see that as lower priority and probably not trivial work. I'm going to update the title to reflect that.

[dudicoco]

I have found more Aqua features which are missing from Mise:

1. No support for all package types:

mise WARN  failed to resolve toolset: [~/repositories/package-managers/.mise.toml] aqua:alpkeskin/[email protected]: packages[0].typ
e: unknown variant `go_install`, expected one of `github_archive`, `github_content`, `github_release`, `http` at line 2 column 
11

2. mise registry cannot find all packages in aqua registry, even when specifying a registry via aqua.registry_url = "https://github.com/aquaproj/aqua-registry" , for example aqua:iamhsa/pkenv cannot be found.

At this point I'm wondering if Mise should run the aqua binary directly instead of reimplementing all of its features.

[dudicoco]

deprecated an existing feature in the same release

I don't believe I ever do this. Once a feature comes out of experimental it stays forever. If this ever were to be done it would involve a long migration path with warnings so that people have plenty of time to migrate without needing to have separate versions installed.

I can give some more examples:

1. A bug in a new version which project a requires could break project b
2. A regression in a new version which project a requires could affect project b

What is the downside of "officially" supporting different mise versions per project? The global config file? If that's the reason then it should be a non-issue since it should be backwards compatible

[jdx]

What is the downside of "officially" supporting different mise versions per project?

it's a lot extra maintenance work to make sure mise is forwards and backwards compatible

[hverlin]

I think you could probably achieve an isolated installation today by updating various environment variables as mentioned in #3306 (comment) and ignoring the global config file using ignored_config_paths without adding new features into mise. Probably, worth prototyping

[syhol]

More things you can do to manage separate installs:

You can use the tool 'ubi:jdx/mise' with a pinned version. I don't think it will change the mise used for hook-env or cd/enter/leave hooks but it will fix the version for any calls to mise. This might not be officially supported but it should work.

Instead of using 'ubi:jdx/mise' you could use mise.run:

• curl https://mise.run | sh
• MISE_INSTALL_PATH=/some/path – change the binary path (default: ~/.local/bin/mise)
• MISE_VERSION=v2024.5.17 – install a specific version

As @hverlin mentioned you can change these https://mise.jdx.dev/directories.html#directory-structure values and https://mise.jdx.dev/configuration/settings.html#ignored_config_paths as you desire.

Also you have min_version https://mise.jdx.dev/configuration.html#minimum-mise-version

I'm curious if you need any more than this.

Personally min_version is enough for me, bugs are often fixed fast and I've not had any breaking changes that weren't hard to adapt to.

[dudicoco]

Thanks all, i'll look into the suggested workarounds.

@jdx what are your thoughts on #3364 (comment)?

[jdx]

I have found more Aqua features which are missing from Mise:

1. No support for all package types:

mise WARN  failed to resolve toolset: [~/repositories/package-managers/.mise.toml] aqua:alpkeskin/[email protected]: packages[0].typ
e: unknown variant `go_install`, expected one of `github_archive`, `github_content`, `github_release`, `http` at line 2 column 
11

You should use the go backend for that. I don't want to add support for package types that require dependencies like go.

2. mise registry cannot find all packages in aqua registry, even when specifying a registry via aqua.registry_url = "https://github.com/aquaproj/aqua-registry" , for example aqua:iamhsa/pkenv cannot be found.

Seems to work for me, unless this is something specific about overriding the registry_url:

~ ❯ mise x aqua:iamhsa/pkenv -- pkenv --version
mise aqua:iamhsa/[email protected]      ✓ installed                                                                                  pkenv 0.5.3

At this point I'm wondering if Mise should run the aqua binary directly instead of reimplementing all of its features.

Who said I was reimplementing all of its features?

[dudicoco]

2. mise registry cannot find all packages in aqua registry, even when specifying a registry via aqua.registry_url = "https://github.com/aquaproj/aqua-registry" , for example aqua:iamhsa/pkenv cannot be found.

Seems to work for me, unless this is something specific about overriding the registry_url:

~ ❯ mise x aqua:iamhsa/pkenv -- pkenv --version
mise aqua:iamhsa/[email protected]      ✓ installed                                                                                  pkenv 0.5.3

The problem is not with installing it, but with searching for it within the registry.

[jdx]

oh I understand: the mise registry and the aqua registry are not the same thing. Perhaps confusing naming but I don't think we can do anything about that.

[dudicoco]

@jdx doesn't mise bundle the aqua registry?

[jdx]

yes but mise registry shows what's in the mise registry, not the aqua registry. Perhaps we could add a top-level mise aqua subcommand for these kind of things but I think it's pretty low priority. We've never had that sort of thing for the equivalent functionality in asdf and it's been fine. I also would be hesitant to add that since not everything in the aqua can be installed with mise. Almost no mise users will care about this since they're just going to use tools from the mise registry.

I think there is some confusion about why the aqua backend exists in mise. It's definitely not for users coming from aqua that just want to manage a similar setup through mise. It's mostly so we can leverage the standard aqua registry as a data source for giving mise a tighter supply chain. Just because you can do something in aqua doesn't mean you should be able to do so in mise. I'm not interested in duplicating functionality—if you want that just use aqua.

[dudicoco]

So each entry from the aqua registry is explicitly added to the mise registry and only then it can be searched for?
If so, then what is aqua.registry_url for? Tool installation only and not searching?
And why shouldn't we be able to search for tools which are installable by mise (such as iamhsa/pkenv)?

[jdx]

It's a manual process to know if something can be installed/used via mise

[syhol]

So each entry from the aqua registry is explicitly added to the mise registry and only then it can be searched for?

Aqua is one of 9 backends (10 if you count core), and not all aqua tools are guaranteed to work in mise. Rather than trying to build a search feature for 10 different tools I personally think it makes more sense to just search the unified registry, and spend the effort getting more aqua tools into the mise registry.

Also some tools can't even be searched. Ubi has no index, it can be any github repo. asdf, vfox, and go have no index, it can be any git repo. And npm has over 3.1 million packages, so it might not make sense to maintain a local index for that.

You can also still install aqua for searching if you like. I install it with ubi.

That's just my take anyway.