[DH-301] Python script to replace bash scaffolding in CI/CD and simplify github workflows

[shaneknapp]

by adding a small python script to determine if all hubs, or just a subset need to be deployed we can now use a single workflow to do this instead of two.

[shaneknapp]

i also thought about consolidating the deploy-hubs-to-staging and deploy-hubs-to-prod workflow jobs, but i do kinda like having discrete ones for each branch (even though it absolutely, 110% duplicates code, which is kinda dumb)

[shaneknapp]

output showing how the script interacts w/the env vars that are set by the get-labels-on-push github action:

(dh) ➜  datahub git:(python-scripts-replace-bash) ✗ env | grep ^GITHUB
GITHUB_PR_LABEL_THIS=1
GITHUB_PR_LABEL_THAT=1
GITHUB_PR_LABEL_HUB_STAT20=1
GITHUB_PR_LABEL_HUB_DATAHUB=1
GITHUB_PR_LABEL_JUPYTERHUB_DEPLOYMENT=1
(dh) ➜  datahub git:(python-scripts-replace-bash) ✗ python .github/scripts/determine-hub-deployments.py # all hubs will be deployed due to GITHUB_PR_LABEL_JUPYTERHUB_DEPLOYMENT=1
a11y
astro
biology
cee
data100
data101
data102
data8
datahub
dev
dlab
edx
eecs
gradebook
highschool
ischool
julia
logodev
prob140
publichealth
r
shiny
stat159
stat20
ugr01
workshop
(dh) ➜  datahub git:(python-scripts-replace-bash) ✗ python .github/scripts/determine-hub-deployments.py --only-deploy logodev datahub # of all hubs, only deploy datahub and logodev
datahub
logodev
(dh) ➜  datahub git:(python-scripts-replace-bash) ✗ python .github/scripts/determine-hub-deployments.py --ignore data100 data101 data102 data8 datahub dev dlab # of all hubs, deploy everything except those we don't want to
a11y
astro
biology
cee
edx
eecs
gradebook
highschool
ischool
julia
logodev
prob140
publichealth
r
shiny
stat159
stat20
template
ugr01
workshop
(dh) ➜  datahub git:(python-scripts-replace-bash) ✗ unset GITHUB_PR_LABEL_JUPYTERHUB_DEPLOYMENT # remove this just to deploy stat20 and datahub
(dh) ➜  datahub git:(python-scripts-replace-bash) ✗ env | grep ^GITHUB
GITHUB_PR_LABEL_THIS=1
GITHUB_PR_LABEL_THAT=1
GITHUB_PR_LABEL_HUB_STAT20=1
GITHUB_PR_LABEL_HUB_DATAHUB=1
(dh) ➜  datahub git:(python-scripts-replace-bash) ✗ python .github/scripts/determine-hub-deployments.py
datahub
stat20
(dh) ➜  datahub git:(python-scripts-replace-bash) ✗ python .github/scripts/determine-hub-deployments.py --ignore stat20
datahub
(dh) ➜  datahub git:(python-scripts-replace-bash) ✗ python .github/scripts/determine-hub-deployments.py --only-deploy stat20
stat20

[shaneknapp]

@felder : sanity check here. the github action that grabs labels from a push exports them to env vars in the format of GITHUB_PR_LABEL_<whatever>=1. in this case, i want to check for the labels of jupyterhub-deployment and/or hub-images (stored as the vars in the code snippet above).

if i want to check if these are indeed set, the way i do it here (if [[ -n ${GITHUB_PR_LABEL_HUB_IMAGES} || -n ${GITHUB_PR_LABEL_JUPYTERHUB_DEPLOYMENT} ]]) is correct, yes?

(i did a bunch of testing on the CLI and this seems to give us what we want but a 2nd pair of eyes is always nice etc)

[felder]

@shaneknapp -n evaluates to true if the variable is not set to the empty string. You may want to use -v instead which evaluates to true if the variable is set. -v also expects the name of the variable as opposed to an expansion of the variable.

So you may want to tryif [[ -v GITHUB_PR_LABEL_HUB_IMAGES || -v GITHUB_PR_LABEL_JUPYTERHUB_DEPLOYMENT ]]

Example:

#!/usr/bin/env bash

TEST=""

if [[ ! -n ${TEST} ]]; then
  echo "TEST is empty"
fi

if [[ -v TEST ]]; then
  echo "TEST is set"
fi

unset TEST

if [[ ! -v TEST ]]; then
  echo "TEST is no longer set"
fi
ras-29-57:~ felder$ ./test.sh
TEST is empty
TEST is set
TEST is no longer set

[shaneknapp]

alright, lemme try that... i'll set some labels to test this when i merge to staging