Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ensure things are properly signed before uploading to stage #425

Merged
merged 5 commits into from
Sep 13, 2024
Merged

ensure things are properly signed before uploading to stage #425

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
72ee6e0
ensure things are properly signed before uploading to stage
evgeni Sep 9, 2024
272fba5
move ARCHES to settings
evgeni Sep 9, 2024
076dc25
move stage local base to a setting
evgeni Sep 9, 2024
5773a8f
check everything that has a key
evgeni Sep 9, 2024
8f5db87
always run verify
evgeni Sep 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions settings
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
SIGNER="[email protected]"
SIGNER_NAME="Foreman Automatic Signing Key"
OSES=""
ARCHES="x86_64 source"
RPMDIR="$RELEASEDIR/rpms"
KEYDIR="$RELEASEDIR/gnupg"
GNUPGHOME="$KEYDIR"
Expand All @@ -67,6 +68,7 @@
RPM_PACKAGES=()
PACKAGING_PR=${PACKAGING_PR:-true}
GPG_EXPIRE="1y"
STAGE_LOCAL_BASE="tmp/$PROJECT/$VERSION"

Check warning

Code scanning / shellcheck

YUM_HOSTS appears unused. Verify use (or export if used externally). Warning

YUM_HOSTS appears unused. Verify use (or export if used externally).
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could also use RPMDIR (which is currently unused) here… questions and choices.

@ehelms do you recall why tmp/ was used vs the previously existing path?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect it was previously used by the other script and @ehelms didn't want to conflict with it, but now that koji is cleaned up it's an unused variable. When I introduced it, it made sense to me to store it in the release itself.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ehelms do you recall why tmp/ was used vs the previously existing path?

What previously existing path? I was the first to introduce the notion of generating staging repositories locally :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But not the first one who had to download rpms, sign them and upload the signatures, which we did in RPMDIR

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair. My motivation was that I wanted the generation of the repositories to be co-located, easy to clean up and to mimic the structure they would have on the staging repository server to allow easier testing and verification of them.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aye. It's probably fine being in tmp and thrown away later. Probably bit if cleanup we can perform later.


load_settings

Expand Down
8 changes: 2 additions & 6 deletions sign_stage_rpms
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,13 @@ set -e

. settings

ARCHES="x86_64 source"

for os in $OSES; do
BASE="tmp/$PROJECT/$VERSION/$os"

for arch in $ARCHES; do
UNSIGNED_RPMS=$(./list_unsigned_rpms "$BASE/$arch" "$HALFGPGKEY")
UNSIGNED_RPMS=$(./list_unsigned_rpms "$STAGE_LOCAL_BASE/$os/$arch" "$HALFGPGKEY")

if [[ -n "$UNSIGNED_RPMS" ]]; then
echo "$UNSIGNED_RPMS" | xargs ./sign_rpms
createrepo_c --general-compress-type gz --database --update "$BASE/$arch"
createrepo_c --general-compress-type gz --database --update "$STAGE_LOCAL_BASE/$os/$arch"
fi
done
done
4 changes: 3 additions & 1 deletion upload_stage_rpms
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,6 @@
USER='yumrepostage'
HOST='web01.osuosl.theforeman.org'

rsync --checksum --times --perms --recursive --links --verbose --partial --one-file-system --delete-after "tmp/$PROJECT/$VERSION/" "$USER@$HOST:rsync_cache/$PROJECT/$VERSION/"
./verify_stage_sigs
evgeni marked this conversation as resolved.
Show resolved Hide resolved

rsync --checksum --times --perms --recursive --links --verbose --partial --one-file-system --delete-after "${STAGE_LOCAL_BASE}/" "$USER@$HOST:rsync_cache/$PROJECT/$VERSION/"
29 changes: 29 additions & 0 deletions verify_stage_sigs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#!/bin/bash

set -e

. settings

if [[ -z "${FULLGPGKEY}" ]]; then
echo "No signing configured."
exit 0
fi

EXIT_CODE=0

for os in $OSES; do
for arch in $ARCHES; do
UNSIGNED_RPMS=$(./list_unsigned_rpms "$STAGE_LOCAL_BASE/$os/$arch" "$HALFGPGKEY")
evgeni marked this conversation as resolved.
Show resolved Hide resolved

if [[ -n "$UNSIGNED_RPMS" ]]; then
echo "$UNSIGNED_RPMS"
EXIT_CODE=1
fi
done
done

if [[ "${EXIT_CODE}" != "0" ]]; then
echo "RPMs not signed with ${HALFGPGKEY} found, aborting"
fi

exit ${EXIT_CODE}
Loading