Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

image: build and upload as one step #2798

Merged
merged 8 commits into from
Jan 15, 2024

Conversation

malt3
Copy link
Contributor

@malt3 malt3 commented Jan 5, 2024

Context

GitHub Actions has changed the API for Artifacts. This limits the amount of artifacts per workflow, doesn't allow overwriting artifacts in a Workflow run and changed the way globs are handled.
Since the old API will likely be disabled soon, we have to refactor the image build pipeline.

As a nice bonus, the pipeline is now roughly twice as fast (1h -> 30min).

Proposed change(s)

  • use uplosi instead of custom Go code for CSP-specific OS image upload
  • add bazel targets under //image/system:upload_* for uploading images
  • refactor CI workflow to use single job for upload (removed usages of GitHub Artifacts)

Additional info

Debug image for testing:

ref/feat-image-build-and-upload-as-one-step/stream/debug/v2.15.0-pre.0.20240105103004-3d9404f299d6

Console image for testing:

ref/feat-image-build-and-upload-as-one-step/stream/console/v2.15.0-pre.0.20240105121312-66211185bb14

Command for uploading debug images from your workstation: bazel run //image/system:upload_debug -- --ref foo --upload-measurements --fake-sign

Checklist

  • Add labels (e.g., for changelog category)
  • Is PR title adequate for changelog?
  • Link to Milestone

@malt3 malt3 added the no changelog Change won't be listed in release changelog label Jan 5, 2024
@malt3 malt3 added this to the v2.15.0 milestone Jan 5, 2024

This comment was marked as off-topic.

@malt3 malt3 force-pushed the feat/image/build-and-upload-as-one-step branch 2 times, most recently from 4b0724a to a4c5a91 Compare January 5, 2024 12:45
@malt3 malt3 requested a review from 3u13r January 5, 2024 14:53
Copy link
Member

@daniel-weisse daniel-weisse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not an in-depth review, mostly skimmed the changes

image/upload/internal/cmd/measurementsenvelope.go Outdated Show resolved Hide resolved
image/upload/internal/cmd/uplosi.go Outdated Show resolved Hide resolved
@malt3 malt3 force-pushed the feat/image/build-and-upload-as-one-step branch 2 times, most recently from 3ef8b89 to 2e0ad1f Compare January 8, 2024 14:12
@msanft msanft force-pushed the feat/image/build-and-upload-as-one-step branch from 2e0ad1f to c1e981e Compare January 15, 2024 10:14
@malt3 malt3 force-pushed the feat/image/build-and-upload-as-one-step branch from c1e981e to 9ef9d22 Compare January 15, 2024 10:38
malt3 added 7 commits January 15, 2024 11:40
This logic was previously performed in a GitHub Actions workflow
using yq.
Since every step should now be performed in Bazel, this now needs to happen here.
This implementation will replace the custom Go code in
internal/osimage/{aws|azure|gcp} and still conforms to the same interface.
This rule combines uplosi, the upload command, measurement code and cosign
to upload OS images, extract measurements, sign them and upload the measurements.
@malt3 malt3 force-pushed the feat/image/build-and-upload-as-one-step branch from 9ef9d22 to 8c02536 Compare January 15, 2024 10:40
Copy link
Contributor

Coverage report

Package Old New Trend
image/upload [no test files] [no test files] 🚧
image/upload/internal/cmd [no test files] [no test files] 🚧
internal/attestation/measurements 84.30% 75.90% ↘️
internal/osimage [no test files] [no test files] 🚧
internal/osimage/uplosi 0.00% [no test files] 🚨

Copy link
Contributor

@msanft msanft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went over the new Bazel rule and the Go code invoking / parsing uplosi, which both look fine to me.

internal/osimage/uplosi/uplosiupload.go Show resolved Hide resolved
@malt3 malt3 merged commit 9a27e7b into main Jan 15, 2024
10 checks passed
@malt3 malt3 deleted the feat/image/build-and-upload-as-one-step branch January 15, 2024 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no changelog Change won't be listed in release changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants