Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor CI/CD so support and staging jobs are separate, and multiple staging hubs are detected #5289

Merged
merged 35 commits into from
Dec 19, 2024
Merged

Refactor CI/CD so support and staging jobs are separate, and multiple staging hubs are detected #5289

merged 35 commits into from
Dec 19, 2024

Conversation

sgibson91
Copy link
Member

@sgibson91 sgibson91 force-pushed the ci-cd/separate-support-staging branch from 34766eb to 8621b4f Compare December 16, 2024 14:49
@sgibson91 sgibson91 changed the title Ci cd/separate support staging CI/CD refactor Dec 16, 2024
@sgibson91 sgibson91 changed the title CI/CD refactor WIP CI/CD refactor Dec 16, 2024
@github-actions GitHub Actions
Copy link

Merging this PR will trigger the following deployment actions.

Support deployments

No support upgrades will be triggered

Staging deployments

Cloud Provider Cluster Name Hub Name Reason for Redeploy
aws projectpythia staging Core infrastructure has been modified
aws victor staging Core infrastructure has been modified
aws nmfs-openscapes staging Core infrastructure has been modified
gcp 2i2c staging Core infrastructure has been modified
gcp 2i2c dask-staging Core infrastructure has been modified
gcp 2i2c ucmerced-staging Core infrastructure has been modified
aws openscapes staging Core infrastructure has been modified
aws strudel staging Core infrastructure has been modified
aws ubc-eoas staging Core infrastructure has been modified
aws jupyter-health staging Core infrastructure has been modified
aws maap staging Core infrastructure has been modified
aws nasa-ghg staging Core infrastructure has been modified
kubeconfig queensu staging Core infrastructure has been modified
aws catalystproject-africa staging Core infrastructure has been modified
gcp cloudbank staging Core infrastructure has been modified
aws nasa-veda staging Core infrastructure has been modified
aws smithsonian staging Core infrastructure has been modified
kubeconfig utoronto staging Core infrastructure has been modified
kubeconfig utoronto r-staging Core infrastructure has been modified
gcp dubois staging Core infrastructure has been modified
gcp leap staging Core infrastructure has been modified
gcp pangeo-hubs staging Core infrastructure has been modified
aws 2i2c-aws-us staging Core infrastructure has been modified
aws 2i2c-aws-us dask-staging Core infrastructure has been modified
gcp awi-ciroh staging Core infrastructure has been modified
aws nasa-cryo staging Core infrastructure has been modified
aws gridsst staging Core infrastructure has been modified
aws opensci staging Core infrastructure has been modified
aws kitware staging Core infrastructure has been modified
aws earthscope staging Core infrastructure has been modified
gcp catalystproject-latam staging Core infrastructure has been modified
gcp 2i2c-uk staging Core infrastructure has been modified
aws jupyter-meets-the-earth staging Core infrastructure has been modified
gcp hhmi staging Core infrastructure has been modified

Production deployments

Cloud Provider Cluster Name Hub Name Reason for Redeploy
aws projectpythia prod Core infrastructure has been modified
aws projectpythia pythia-binder Core infrastructure has been modified
aws victor prod Core infrastructure has been modified
aws nmfs-openscapes prod Core infrastructure has been modified
gcp 2i2c imagebuilding-demo Core infrastructure has been modified
gcp 2i2c binderhub-ui-demo Core infrastructure has been modified
gcp 2i2c demo Core infrastructure has been modified
gcp 2i2c ohw Core infrastructure has been modified
gcp 2i2c temple Core infrastructure has been modified
gcp 2i2c ucmerced Core infrastructure has been modified
gcp 2i2c mtu Core infrastructure has been modified
aws openscapes prod Core infrastructure has been modified
aws openscapes workshop Core infrastructure has been modified
aws strudel prod Core infrastructure has been modified
aws ubc-eoas prod Core infrastructure has been modified
aws jupyter-health prod Core infrastructure has been modified
aws maap prod Core infrastructure has been modified
aws nasa-ghg prod Core infrastructure has been modified
kubeconfig queensu prod Core infrastructure has been modified
aws catalystproject-africa nm-aist Core infrastructure has been modified
aws catalystproject-africa must Core infrastructure has been modified
aws catalystproject-africa uvri Core infrastructure has been modified
aws catalystproject-africa wits Core infrastructure has been modified
aws catalystproject-africa kush Core infrastructure has been modified
aws catalystproject-africa molerhealth Core infrastructure has been modified
aws catalystproject-africa aibst Core infrastructure has been modified
aws catalystproject-africa bhki Core infrastructure has been modified
aws catalystproject-africa bon Core infrastructure has been modified
gcp cloudbank bcc Core infrastructure has been modified
gcp cloudbank ccc Core infrastructure has been modified
gcp cloudbank ccsf Core infrastructure has been modified
gcp cloudbank chabot Core infrastructure has been modified
gcp cloudbank csm Core infrastructure has been modified
gcp cloudbank csulb Core infrastructure has been modified
gcp cloudbank csum Core infrastructure has been modified
gcp cloudbank demo Core infrastructure has been modified
gcp cloudbank dvc Core infrastructure has been modified
gcp cloudbank elac Core infrastructure has been modified
gcp cloudbank elcamino Core infrastructure has been modified
gcp cloudbank evc Core infrastructure has been modified
gcp cloudbank fresno Core infrastructure has been modified
gcp cloudbank foothill Core infrastructure has been modified
gcp cloudbank glendale Core infrastructure has been modified
gcp cloudbank high Core infrastructure has been modified
gcp cloudbank howard Core infrastructure has been modified
gcp cloudbank humboldt Core infrastructure has been modified
gcp cloudbank lacc Core infrastructure has been modified
gcp cloudbank lamission Core infrastructure has been modified
gcp cloudbank laney Core infrastructure has been modified
gcp cloudbank lbcc Core infrastructure has been modified
gcp cloudbank mendocino Core infrastructure has been modified
gcp cloudbank merced Core infrastructure has been modified
gcp cloudbank mills Core infrastructure has been modified
gcp cloudbank miracosta Core infrastructure has been modified
gcp cloudbank mission Core infrastructure has been modified
gcp cloudbank moreno Core infrastructure has been modified
gcp cloudbank norco Core infrastructure has been modified
gcp cloudbank palomar Core infrastructure has been modified
gcp cloudbank pasadena Core infrastructure has been modified
gcp cloudbank reedley Core infrastructure has been modified
gcp cloudbank riohondo Core infrastructure has been modified
gcp cloudbank sacramento Core infrastructure has been modified
gcp cloudbank saddleback Core infrastructure has been modified
gcp cloudbank santiago Core infrastructure has been modified
gcp cloudbank sbcc Core infrastructure has been modified
gcp cloudbank sbcc-dev Core infrastructure has been modified
gcp cloudbank sierra Core infrastructure has been modified
gcp cloudbank sjcc Core infrastructure has been modified
gcp cloudbank sjsu Core infrastructure has been modified
gcp cloudbank skyline Core infrastructure has been modified
gcp cloudbank srjc Core infrastructure has been modified
gcp cloudbank tuskegee Core infrastructure has been modified
gcp cloudbank wlac Core infrastructure has been modified
aws nasa-veda prod Core infrastructure has been modified
aws nasa-veda binder Core infrastructure has been modified
aws smithsonian prod Core infrastructure has been modified
kubeconfig utoronto prod Core infrastructure has been modified
kubeconfig utoronto r-prod Core infrastructure has been modified
gcp dubois prod Core infrastructure has been modified
gcp dubois ephemeral Core infrastructure has been modified
gcp leap prod Core infrastructure has been modified
gcp leap public Core infrastructure has been modified
gcp pangeo-hubs prod Core infrastructure has been modified
gcp pangeo-hubs coessing Core infrastructure has been modified
aws 2i2c-aws-us showcase Core infrastructure has been modified
aws 2i2c-aws-us ncar-cisl Core infrastructure has been modified
aws 2i2c-aws-us itcoocean Core infrastructure has been modified
aws 2i2c-aws-us cosmicds Core infrastructure has been modified
gcp awi-ciroh prod Core infrastructure has been modified
aws nasa-cryo prod Core infrastructure has been modified
aws gridsst prod Core infrastructure has been modified
aws opensci sciencecore Core infrastructure has been modified
aws opensci climaterisk Core infrastructure has been modified
aws opensci small-binder Core infrastructure has been modified
aws opensci big-binder Core infrastructure has been modified
aws kitware prod Core infrastructure has been modified
aws earthscope prod Core infrastructure has been modified
gcp catalystproject-latam unitefa-conicet Core infrastructure has been modified
gcp catalystproject-latam cicada Core infrastructure has been modified
gcp catalystproject-latam gita Core infrastructure has been modified
gcp catalystproject-latam iner Core infrastructure has been modified
gcp catalystproject-latam plnc Core infrastructure has been modified
gcp catalystproject-latam unam Core infrastructure has been modified
gcp catalystproject-latam cabana Core infrastructure has been modified
gcp catalystproject-latam nnb-ccg Core infrastructure has been modified
gcp catalystproject-latam labi Core infrastructure has been modified
gcp catalystproject-latam areciboc3 Core infrastructure has been modified
gcp catalystproject-latam valledellili Core infrastructure has been modified
gcp 2i2c-uk lis Core infrastructure has been modified
aws jupyter-meets-the-earth prod Core infrastructure has been modified
gcp hhmi prod Core infrastructure has been modified
gcp hhmi spyglass Core infrastructure has been modified
gcp hhmi binder Core infrastructure has been modified

@sgibson91 sgibson91 added the deployer:skip-deploy Skips deployment of anything (support, staging, prod) label Dec 17, 2024
sgibson91 and others added 20 commits December 17, 2024 20:07
@sgibson91
Update comments
5978b1e
@sgibson91
Rename filter_staging_hubs_to_staging_matrix function and combine int…
c919237 
…o generate_hub_matrix_jobs
@sgibson91
Update assign_staging_jobs_for_missing_clusters so staging jobs stay …
6b87a6b 
…separate from support and mimic format of prod hub jobs
@sgibson91
Update generate_support_matrix_jobs to only expect support jobs
b08d106
@sgibson91
Update pretty_print_matrix_jobs to print 3 tables instead of 2
01f121a
@sgibson91
Ensure 3 matrices are passed to GitHub Actions and written as a comment
f721e60
@sgibson91
fix typo in comment
5956567
@sgibson91
ci/cd: update generate-jobs to handle 3 output matrices
054ed39
@sgibson91
ci/cd: ensure a specific job for support chart upgrades
7ad0bff
@sgibson91
ci/cd: add job to filter staging and prod jobs based on failed suppor…
712ffe2 
…t jobs
@sgibson91
ci/cd: create a job specifically for upgrading staging hubs
6eb7c09
@sgibson91
ci/cd: add a job to further filter prod hub deployments if a staging …
49725a9 
…hub has failed
@sgibson91
ci/cd: ensure prod hub jobs is inheriting refiltered matrix from fail…
3ae3e78 
…ed staging job
@sgibson91
Remove no longer used function
9cb5a3d
@sgibson91
Run pre-commit
d6cbec9
@sgibson91
fix-up deployer helm-upgrade-decision tests
e901dcc
@sgibson91
With a separate support matrix, we no longer need upgrade_support field
98c72c0
@pre-commit-ci @sgibson91
[pre-commit.ci] auto fixes from pre-commit.com hooks
dba9546 
for more information, see https://pre-commit.ci
@sgibson91
Fix job name references in workflow file
003b348
@sgibson91
Ensure variable name consistency
51be2fe
@sgibson91 sgibson91 force-pushed the ci-cd/separate-support-staging branch from b30d023 to 51be2fe Compare December 17, 2024 20:16
@sgibson91 sgibson91 force-pushed the ci-cd/separate-support-staging branch from 6c200ed to be8b0a8 Compare December 18, 2024 14:00
@sgibson91 sgibson91 removed the deployer:skip-deploy Skips deployment of anything (support, staging, prod) label Dec 19, 2024
@sgibson91 sgibson91 changed the title WIP CI/CD refactor Refactor CI/CD so support and staging jobs are separate, and multiple staging hubs are detected Dec 19, 2024
@sgibson91 sgibson91 marked this pull request as ready for review December 19, 2024 16:22
@sgibson91 sgibson91 merged commit a43e8df into 2i2c-org:main Dec 19, 2024
42 checks passed
@sgibson91 sgibson91 deleted the ci-cd/separate-support-staging branch December 19, 2024 16:22
@github-actions GitHub Actions
Copy link

🎉🎉🎉🎉

Monitor the deployment of the hubs here 👉 https://github.com/2i2c-org/infrastructure/actions/runs/12416674701

This was referenced Dec 19, 2024
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Refactor the deployer and CI/CD pipeline to permit deployment of multiple staging hubs per cluster
1 participant
@sgibson91