[Module Catalog] Provide a --dry-run flag for the create command

[c-pius]

Description

Add a --dry-run flag to the create command. When --dry-run is true, the build should execute:

• print the resulting ModuleTemplate to the output instead of writing a file
• check if the OCI artifacts could be pushed to the registry (no conflicts)

Reasons

Neighbors have asked for a possibility to test the module build without actually pushing the artifacts to the registry. Major use case seems to be in their PR pipeline where they need automated checks that the module version is valid without pushing the actual image and template. If they would push it, this would prevent changes to the PR as the re-push of the same version would be rejected.

Attachments

Related Issues:

• feat: Add dry-run flag to prevent pushing module template to registry cli#2267
• Implement a validate command in modulectl #63

[c-pius]

Needs clarification what is needed exactly. E.g., would it also be possible to use a test registry and suffix the version with the commit sha. At the same time check with OCM tools if the descriptors are there in the production registry already.

[Sawthis]

@c-pius On the submission pipeline we use 2 artifact registries, dev-modules-internal for modules built on the pull request level and modules-internal for modules built on the main branch. We were asked to remove --module-archive-version-overwrite flag from the CLI command. To be able to verify if it won't fail after merge to main we had to introduce --dry-run flag in the Kyma CLI. To verify if the content of the OCM structure was not changed for the same version between the channels.

More details in the issue: https://github.tools.sap/kyma/test-infra/issues/469

[c-pius]

Sorry, I still don't get the full picture. I understand from the linked ticket, that for the PR artifacts you are okay already (you clear the registry after runs).
But what do you exactly need modulectl to do when --dry-run is set? Just build the OCM artifacts and ModuleTemplate and simply don't push the artifacts to the registry (just to see if the build works)? Or do you need some check that for the built artifacts, there are no existing entries in the registry?

[Sawthis]

@c-pius - yes, we don't push the image with the --dry-run flag enabled. Details: kyma-project/cli#2267

We need to be able to check if the build won't fail after the merge to the main. Currently, the module team can release the module in version 1.1.0 to the fast channel, and they should not change content for the regular channel. We need to be able to check it against the production registry before merging it into the main branch.

[Sawthis]

@c-pius For the dev registry we don't clear the registry, we use the --module-archive-version-overwrite flag.
https://github.tools.sap/kyma/test-infra/blob/main/ado/scripts/kyma-alpha-create-module.sh#L32

[c-pius]

@c-pius For the dev registry we don't clear the registry, we use the --module-archive-version-overwrite flag. https://github.tools.sap/kyma/test-infra/blob/main/ado/scripts/kyma-alpha-create-module.sh#L32

We didn't port the --module-archive-version-overwrite flag to modulectl as we didn't want to allow overwrites any longer. Would it be possible as part of the PR pipeline to first clean the existing artifacts based on the PR tag, if any? With cleaning I mean by a command line call like docker image rm, not via modulectl.

If this is not possible, we probably need to port that functionality to modulectl. We would just need to check how to best implement this so that the flag is not used accidentally.

[Sawthis]

@c-pius yes, we can remove the artifact in the dev registry using glcoud.

But if someone already released a module with version 1.1.0 in the prod registry, then we should not allow merging the PR. With the new modulectl, I assume that checking that such an artifact exists in the prod registry is enough. With the kyma cli we needed the dry-run flag, because it was allowed to release the same version of the module twice e.g. for fast and regular channels but we had to perform the check if the OCM structure was not changed. The best way to do it was dry-run flag in the kyma cli.

[c-pius]

@c-pius yes, we can remove the artifact in the dev registry using glcoud.

okay good, then no --module-archive-version-overwrite flag needed for modulectl right now

But if someone already released a module with version 1.1.0 in the prod registry, then we should not allow merging the PR. With the new modulectl, I assume that checking that such an artifact exists in the prod registry is enough. With the kyma cli we needed the dry-run flag, because it was allowed to release the same version of the module twice e.g. for fast and regular channels but we had to perform the check if the OCM structure was not changed. The best way to do it was dry-run flag in the kyma cli.

So in summary, I understand that we need two checks:

• A) modulectl create --dry-run to execute all build steps WITHOUT the eventual push to the registry (to check if the build works).
• B) a check if the resulting artifact version exists in the prod registry.

I am just wondering if B) needs to be a feature of the modulectl (--dry-run) or if this could also be done via the ocm/gcloud cli?

[Sawthis]

@c-pius
A) When we will have the feature of deleting artifacts from the dev registry then we don't need A).
B) can be also a check on the pipeline, currently, the version in the module-config needs to be bigger than previous one otherwise the pipeline fails.

The question is whether such features will be useful for other modulectl users. We used --module-archive-version-overwrite because it was provided by the Kyma CLI, and we don't need to delete the artifact on the pipeline in the dev registry. We needed --dry-run flag because with Kyma CLI it's allowed to release twice a module with the same version. If it is not allowed with the use of modulectl then we don't need it. We can just fail a pipeline in this case but what about other users of modulectl? Should it be allowed for them to release the module with the same version twice. It would be better when the modulectl will return error to us and other users.

[c-pius]

Understood, thanks for clarifying!

@janmedrek can you reach out in the PO call to the other module teams and check with the mdoule teams if and how they are using kymacli and in the future modulectl? We would need to clarify if they regularly want to "overwrite" existing module versions or if they need a dry run (build but don't push). Context is for their own development and testing purposes, not for the modules submitted to the neighbors pipeline.

[Sawthis]

@c-pius
one thing is the submission pipeline and other is the modulectl users who will use it without the use of the submission pipeline. We can implement some checks or delete the artifact from the dev registry but the question is if it shouldn't be handled by modulectl for users who will use it without the use of the submission pipeline.

The same case is in https://github.tools.sap/kyma/test-infra/issues/453. We can implement such check on the submission pipeline. But if someone tests it with modulectl locally or in the module repository GH action, then it will be allowed and won't fail.

[janmedrek]

@c-pius I believe having the dry-run no-push flow in the modulectl would be nice.

The context: even if Kyma teams rely solely on the submission pipeline, there are still some cases for the community modules that would benefit from such functionality.

We can lower the priority of this ticket as it is not crucial for migration, but would keep this one in our backlog for implementation.