-
Notifications
You must be signed in to change notification settings - Fork 69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ADR 45: Guidelines for developing tasks #216
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,54 @@ | ||||||
# 45. Guidelines for developing tasks | ||||||
|
||||||
* Date: 2024-11-06 | ||||||
|
||||||
## Status | ||||||
|
||||||
Proposed | ||||||
|
||||||
## Context | ||||||
|
||||||
One of the architectural goals for Konflux is extensibility. The core system provides opinionated [build pipelines] | ||||||
and [release pipelines], but lets users extend those and/or create their own. Beyond this goal, however, there has | ||||||
not been any clear guidance about where contributions should be added or the types of functionality that are typical | ||||||
for the stages of software development. | ||||||
|
||||||
## Decision | ||||||
|
||||||
* If any process is required to modify artifacts themselves, it should be done in [build pipelines]. This will ensure | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not aware any use cases that modify the artifacts from the build pipeline. Perhaps a footnote/appendix with examples of such modifications could be useful. Some OLM use cases I'm not aware of? This could be even broader/stricter, e.g. "any production of the artifact must be performed on the build pipelines with the intent that the produced artifact is shipped as-is". There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. konflux-ci/release-service-catalog#656 modifies the sbom, which is an artifact from the build pipeline |
||||||
that Konflux users can appropriately test and verify these artifacts before triggering releases with them. | ||||||
* Any tasks which need to be included in the artifacts' provenance (enabling verification with EC policies) must be | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 👍 |
||||||
included in the [build pipelines]. | ||||||
* Whenever possible, a task that just changes the format of an artifact (for example creating a tar from a container image) | ||||||
should be performed in the same pipeline that produced the artifact. These modified artifacts should be attached using | ||||||
the referrer's API and the blob digest reported via results to enable verification that the artifacts were produced | ||||||
from a trusted task. | ||||||
Comment on lines
+22
to
+25
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why? Converting to a different format can be done during release or even post-release without loss of trust There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It enables users to verify that the format change is done properly before those artifacts are released. I am not sure what you mean by post-release here. These are modifications that are needed before releases can progress. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Maybe I'd need a better example than the one in this doc. When I There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If you have an integration test scenario that does not use the ubi-migro image in registry but instead uses a tarball, you would need that tarball to verify the artifact. The need to verify isn't always required by a human but by some process. It should be possible to have all artifacts-to-be-released verified as is. If testing/verifying, a user or process shouldn't have to replicate the modifications done during a release pipeline within integration pipelines. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Agreed, but still think that the example is really bad. The use case is purely hypothetical, nobody is ever going to create an integration test that requires an image to be in the weird format before testing the image. And in this example the release pipeline doesn't do any modifications. If we're calling |
||||||
* Artifact-modifying steps should generally be avoided in [release pipelines]. When interacting with produced artifacts, | ||||||
the release pipeline should be limited to extracting artifacts from storage (i.e. an OCI registry), copying/pushing | ||||||
artifacts to target locations, and updating metadata associated with the artifacts. | ||||||
* Reusable tasks or pipelines that are intended to be used in IntegrationTestScenarios should be contributed to | ||||||
[pipeline samples]. | ||||||
|
||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we want to add a bullet point to where/how Tasks are contributed to the build (or even release) pipelines here? |
||||||
## Consequences | ||||||
|
||||||
* All artifacts produced in Konflux should have provenance attestations. These attestations will include information | ||||||
about which tests have run. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The word "test" could be a bit ambiguous, perhaps
Suggested change
|
||||||
* Whenever possible, the provenance attestations should be made available with the released artifacts. It may be | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure how this relates to the topic of developing tasks. |
||||||
necessary to update this metadata during the release process (for example, signing provenance with a different key | ||||||
or updating the provenance to indicate that a build was hermetic). If it is not possible to publish the provenance | ||||||
as-is, a summary attestation can be used instead. | ||||||
* It should be possible to test any artifact that is going to be released. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Perhaps not related to the topic of developing tasks? |
||||||
|
||||||
## Footnotes | ||||||
|
||||||
* It is not currently possible to produce trusted build-time tasks without pushing them to the [build definitions] | ||||||
repository. Once the process for trusting tasks from various locations has been established, a new ADR should be | ||||||
created to update the recommendations presented here. | ||||||
* Once it is possible to generate attestations for integration tests, the requirement to add all gating tasks to the | ||||||
[build definitions] can be relaxed. This will, of course, require that the tasks and their results can be | ||||||
appropriately trusted. | ||||||
|
||||||
[build pipelines]: https://github.com/konflux-ci/build-definitions/ | ||||||
[build definitions]: https://github.com/konflux-ci/build-definitions/ | ||||||
[release pipelines]: https://github.com/konflux-ci/release-service-catalog | ||||||
[pipeline samples]: https://github.com/konflux-ci/pipeline-samples |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would mentioning Task policies be of value here? As in: "Tasks contributed to Konflux need to adhere to the Build and Task policies"