Add ucmerced staging hub with profileList for Python and R images

[GeorgianaElena]

Fixes #3150

Related

• Reach out to communities to help them transition from 2i2c-hubs-image #2674
• Proto-goal: Reduce amount of custom images we have to maintain #2583
• Enable UC Merced hub users to select between R and Python focused images #3188

[github-actions]

Merging this PR will trigger the following deployment actions.

Support and Staging deployments

Cloud Provider	Cluster Name	Upgrade Support?	Reason for Support Redeploy	Upgrade Staging?	Reason for Staging Redeploy
gcp	2i2c	No		Yes	Following helm chart values files were modified: enc-ucmerced-staging.secret.values.yaml, ucmerced-staging.values.yaml, ucmerced-common.values.yaml

Production deployments

Cloud Provider	Cluster Name	Hub Name	Reason for Redeploy
gcp	2i2c	ucmerced	Following helm chart values files were modified: ucmerced.values.yaml, ucmerced-common.values.yaml

[sgibson91]

Is it worth having a ucmerced-common.values.yaml file to reduce repetition, e.g., homepage config?

[GeorgianaElena]

Thanks @consideRatio and @sgibson91 for the review. I've updated the PR based on the feedback. Also, I've manually deployed this and is runnitng at https://staging.ucmerced.2i2c.cloud/ What do you think?

[consideRatio]

I agree with the adding a staging hub as a way to help the many users of ucmerced test the experience of choosing one of these images while migrating away from the one we have maintained.

Can we add a note in cluster.yaml next to this staging hub that this is to be considered an exception, and that once they want the same experience in the prod hub, we remove this staging hub?

[sgibson91]

@GeorgianaElena what was the motivation for only using profile lists on staging? I actually don't see this mentioned in the original issue

[consideRatio]

@sgibson91 as a strategy to help them move away from using the old 2i2c maintained image, this relates to #2674 and #2583

[sgibson91]

Right, I just didn't think that was why the original issue was opened. I think it was opened because I tanked prod while trying to debug something right before a class 😅

[GeorgianaElena]

Sorry for not linking the main issue @sgibson91 (#3188)

I have now also added a comment in cluster.yaml about this as well.

[github-actions]

🎉🎉🎉🎉

Monitor the deployment of the hubs here 👉 https://github.com/2i2c-org/infrastructure/actions/runs/6408123254

[consideRatio]

Oops I understood this initially as a "let this hub be deployed first to catch issues via our automation", but its more like a production hub that is deployed side by side with the real production ucmerced hub - used as a staging hub for the end users.

I think long term if we do this again, it could be good to call this "trial hub" or similar to avoid that kind of confusion. That would then also have implied that its a short term hub.

[sgibson91]

I'm surprised it didn't launch as a staging job given this logic

infrastructure/deployer/helm_upgrade_decision.py

Lines 315 to 322 in e9a7ac7

	# Separate the jobs for hubs with "staging" in their name (including "dask-staging")
	# from those without staging in their name
	staging_hub_jobs = [
	job for job in all_hub_matrix_jobs if "staging" in job["hub_name"]
	]
	prod_hub_matrix_jobs = [
	job for job in all_hub_matrix_jobs if "staging" not in job["hub_name"]
	]

Edited to add: Actually, I think what the workflow file can't (yet) handle is multiple staging hubs on a given cluster. We make an explicit exception for "staging" and "dask-staging" on the 2i2c cluster.