No more auth0

[GeorgianaElena]

Closes #1304

Straightforward changes:

• Deleted auth0 config from everywhere:
  • hub configs
  • schema
  • deployer
  • extra_scripts
• Changed auth0 config for CILogon 🤞🏼 Hopefully the validation should catch any errors here
  • cilogon client credentials were generated by the deployer and stored encrypted into each cluster dir
  • ❗ missconfiguration here can cause failures
• Found some small issues in oauthenticator [CILogonOAuthenticator] Add profile to default scope, fix detail following recent refactoring jupyterhub/oauthenticator#575

Worth some more 👀 on it:

• Updated the cilogon_app.py to:

  • no more classes as unnecessarily
  • don't assume that the encrypted configuration file of a hub (enc-{hub_name}.secret.values.yaml) can only contain cilogon specific configuration
  • allow deleting from and adding to an enc-{hub_name}.secret.values.yaml file that's either empty or has prior existing config.

• Turned existing auth0 client management into a deployer subcommand in case we want to use it still in exceptional cases [Question] Should we turn the existing Auth0 OAuth client management code into a deployer subcommand #2328

Todo:

• generate cilogon client apps for the hubs that don't use auth0 anymore
• any other auth0 cleanup needed
• update deployer readme deployer: Update readme and docs to match latest deployer scripts updates #2408
• update deployer docs?
• manual step once all hubs have been migrated: delete all auth0 apps
• [CILogonOAuthenticator] Add profile to default scope, fix detail following recent refactoring jupyterhub/oauthenticator#575 added profile to the default scope list and we don't need to be explicit about it anymore. Open an issue to track the cleanup of our config.

[GeorgianaElena]

Looks like I've hit a max number of OAuth apps that ca be created, which is 50.
Need to check with CILogon folks if this number can be increased 🤔
✅ Solved https://2i2c.freshdesk.com/a/tickets/536. We can now create up to 100 max clients

[damianavila]

We can now create up to 100 max clients

Do you foresee needing more in the short future? Btw, the request was fulfilled pretty quickly, IIRC, so it should not be a problem to ask for more, WDYT?

[GeorgianaElena]

Do you foresee needing more in the short future?
I think we should be fine with this number in the short future

Btw, the request was fulfilled pretty quickly, IIRC, so it should not be a problem to ask for more, WDYT?
Yes, they were quick

[GeorgianaElena]

This should be ready for review now. Sorry for the PR size :/
I've updated the top comment to reflect the changes.

Note, that merging this PR, will redeploy most of the hubs. This shouldn't be disruptive, as the hub usernames would remain the same, and the same providers will be used.

The only notable difference for the users would be that instead of choosing their authorization provider from an Auth0 login screen, they will be presented with the CILogon one.

Things will break only if there are OAuthenticator miscofigurations (hopefully I didn't miss anything 🤞🏼 )

[damianavila]

Note, that merging this PR, will redeploy most of the hubs. This shouldn't be disruptive, as the hub usernames would remain the same, and the same providers will be used.

Let's discuss quickly in the sprint planning how/when we operationally deploy this one!

[consideRatio]

Wieee thanks for your thorough work on this!

This PR now switches from Auth0 to CILogon consistently, also when GitHubOAuthenticator could have been used. Do you suggest new hubs should go for CILogonOAuthenticator -> GitHub Auth, and/or migrating existing setups etc?

I think for now, this is a step in the right direction no matter what - so let's not think of that question as a blocker!

This LGTM but I made some comments that could be relevant to consider.

[GeorgianaElena]

Thank you very much for reviewing and approving this @consideRatio ❤️

This PR now switches from Auth0 to CILogon consistently, also when GitHubOAuthenticator could have been used. Do you suggest new hubs should go for CILogonOAuthenticator -> GitHub Auth, and/or migrating existing setups etc?

@consideRatio, I believe we should go for CILogonOAuthenticator by default, as it was the case for Auth0.
It is my understanding that existing setups that use the GitHubOAuthenticator direclty are the ones that needed GitHub team/org membership authorization.

And one of the strong points in favour of CILogon vs using OAuth specific authenticators (apart from being able to support loging in with multiple providers on the same hub) is that we can create and manage the oidc client apps programatically. This is not the case with GitHub for example, where we create the OAuth apps from the UI :/

Let's discuss quickly in the sprint planning how/when we operationally deploy this one!

@damianavila, I agree 👍🏼 I'll also share what I have in mind async to help me have the ideas in line:

Because our health check only checks spawning and not authentication, testing must happen manually I believe.
I was thinking that tomorrow, my morning I manually deploy the changes here, one cluster at a time, and manually check logging in on the hubs.

If everything works, I go for a merge.

[consideRatio]

• @GeorgianaElena I've opened a PR that needs to be merged before this one to avoid putting it in a inconsistent state, where the node pools doesn't match the profileList. This is nasa-cryo: k8s 1.22 to 1.25, node sharing setup #2374 specifically, I don't think victor: k8s 1.22 to 1.25, core node from m5 to r5, dask nodes from 4 different m5 to one r5.4xlarge  #2375 and 2373 will break via inconsistent state if this is merged though!

[GeorgianaElena]

@GeorgianaElena I've opened a PR that needs to be merged before this one to avoid putting it in a inconsistent state, where the node pools doesn't match the profileList.

Thanks for the heads up @consideRatio! Now that the PRs you linked are merged, I believe this should be good to go, right?

I was thinking that tomorrow, my morning I manually deploy the changes here, one cluster at a time, and manually check logging in on the hubs.

This actually got posed to investigate #2302, but I'll start the manual deploy and testing today.

[consideRatio]

Absolutely!

[GeorgianaElena]

Manually deployed all hubs that had transitioned from auth0 to cilogon and verified that I can login and that I am an admin. Everything was fine 🎉

Will go for a merge.

[github-actions]

🎉🎉🎉🎉

Monitor the deployment of the hubs here 👉 https://github.com/2i2c-org/infrastructure/actions/runs/4479186946

[consideRatio]

@GeorgianaElena wieeee nice work!!!!!!