Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend NRP build workflow to support creating/updating Github releases with artifacts #797

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jepio
Copy link
Member

@jepio jepio commented Nov 8, 2024

Description

Currently test NRP artifacts are published to Github releases manually, and the policy json is separately committed to main here: https://github.com/Azure/azure-osconfig/blob/main/src/adapters/mc/asb/AzureLinuxBaseline_DeployIfNotExists.json (ditto for ssh policy). This manual process is error-prone and the two artifacts (zip+policy json) regularly get out of sync. This has led to confusion among team members trying to test recent releases.

Improve the situation by extending the NRP build workflow to support automatically creating a Github release with all the relevant artifacts: zip files and matching policy json. The policy json template is filled with the correct checksum and URL to the release being published. The body of the release will now contain a reference to the commit that the policy was built from for easy cross-referencing.

Passing an empty string for "release name" will result in no Github release being created. An existing "release name" can be passed which will update artifacts and body of that release.

The static policy.json files (non templates) have been removed as they will no longer need to be kept in sync.

Checklist

  • I have read the contribution guidelines.
  • I added unit-tests to validate my changes. All unit tests are passing.
  • I have merged the latest main branch prior to this PR submission.
  • I submitted this PR against the main branch.

@jepio jepio requested a review from a team as a code owner November 8, 2024 14:32
Copy link
Contributor

@danielszot danielszot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have a template now I would opt for deleting *.json file with policy definition to rely in 100% on the files we have in the releases instead of the file in the repo which is not up to date when not manually updated.

Besides of the above, LGTM

sed -e "s|@HASH@|${ssh_hash}|g" -e "s|@URI@|${ssh_uri}|g" ${ssh_template} > ${ssh_output}

- name: Create or update release with new artifacts
uses: softprops/action-gh-release@v2
Copy link
Contributor

@danielszot danielszot Nov 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@v2 is mutable AFAIK. I would opt for freezing it with a commit hash for security reasons.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, in the past i considered this overkill, it would need to be done for all actions and we'd likely forget to keep them updated.

I'm interested in hearing more perspectives on this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean is: the attack vectors are supply chain attacks on the one hand and outdated vulnerable dependencies on the other. Dependabot might be able to help.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love everything being hermetic and having an automated way of creating PR with bumped up dependencies for just a manual reviews.
It minimizes flakiness and maintains source code governance so positively impacts security.

I'm not going to insist here ofc. We don't have such automations here yet.

Copy link
Member

@robertschaedler3 robertschaedler3 Nov 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the past we have used gh release create to automate the release creation on GitHub as part of a script after the "official" GitHub release action is no longer maintained (and to avoid using 3rd party actions for publishing releases which could result in supply chain as you mentioned).

https://cli.github.com/manual/gh_release_create

For prerelease this seems fine to me though, we can revisit it later for official releases as needed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on using gh cli, i prefer that over 3p actions (albeit being mentioned on the official https://github.com/actions/create-release action) but wont block on it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll give it a try (hopefully next week).

@jepio
Copy link
Member Author

jepio commented Nov 8, 2024

If we have a template now I would opt for deleting *.json file with policy definition to rely in 100% on the files we have in the releases instead of the file in the repo which is not up to date when not manually updated.

Besides of the above, LGTM

Good idea, did this just now.

MariusNi

This comment was marked as outdated.

@jepio
Copy link
Member Author

jepio commented Nov 12, 2024

I've reverted the static policy json files, keeping those does not interfere with the purpose of this PR (nor does this PR interfere with other usecases). I need some more time to respond to the comments.

Copy link
Contributor

@AhmedBM AhmedBM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @jepio, this PR conflicts with some work we were doing for SFI compliance. We are moving away from building/publishing directly using out GitHub infrastructure and migrating over to OneBranch for building/publishing so we can still leverage our GitHub workflows for testing.
So far the changes have been internal with the exception of our test workflow as we wanted to take advantage of our GitHub test automation, see #802. We like your changes on automating the ARM templates used for the custom policy deployment but the policy definitions are not needed for releasing the policies, only for custom policy deployment for our own selfhosting, bug bashing, manual validation, etc.

@jepio
Copy link
Member Author

jepio commented Nov 20, 2024

We like your changes on automating the ARM templates used for the custom policy deployment but the policy definitions are not needed for releasing the policies, only for custom policy deployment for our own selfhosting, bug bashing, manual validation, etc

That's exactly what I needed it for - custom policy deployments for manual testing. Would you be able to generate the ARM templates through the internal workflows and push them together with any (pre)release assets to Github? That would make this PR obsolete and I could close it.

@AhmedBM
Copy link
Contributor

AhmedBM commented Nov 20, 2024

We like your changes on automating the ARM templates used for the custom policy deployment but the policy definitions are not needed for releasing the policies, only for custom policy deployment for our own selfhosting, bug bashing, manual validation, etc

That's exactly what I needed it for - custom policy deployments for manual testing. Would you be able to generate the ARM templates through the internal workflows and push them together with any (pre)release assets to Github? That would make this PR obsolete and I could close it.

@jepio ahh ok, it makes sense for that scenario, let me have a chat with @MariusNi on the matter. Ill follow-up with a plan to include those ARM templates somehow.

Copy link
Contributor

@AhmedBM AhmedBM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌

Copy link
Contributor

@AhmedBM AhmedBM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, mistakenly approved the wrong PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants