Stop using repo2docker's dev releases, use only stable (or non-dev pre-releases)

[consideRatio]

I understand that repo2docker releases are tested on mybinder.org before they are made - sometimes at least, its not part of a documented procedure. I think that is a bit too tight coupling and we shouldn't to avoid complexity and maintenance burden. I think we can and should trust our repo2docker test suite enough to not break things on mybinder.org.

If we don't do that, then we have to make judgement calls in individual situations on release procedure in repo2docker and bump procedures of repo2docker version in mybinder.org-deploy that becomes a bit messy. I'd like instead that we just stick to something simple: to make regular releases of repo2docker, and to bump to those regular releases on mybinder.org-deploy, and if needed, make an announcement as part of or before bumping if there is a breaking change of relevance.

Related discussion

• "ideally we also want to test the refrozen environments by deploying to mybinder.org", from Bring our release practices in line with rest of JupyterHub org repo2docker#1280 (comment)
• "How about we make an announcement before merging/deploying this to mybinder.org", from Upgrade base image from to Ubuntu 22.04 from 18.04 repo2docker#1287 (comment)

[yuvipanda]

IMO, this works only if releases of repo2docker are fully automated - running on a cron every month in GHA. Given that the last repo2docker release was 8 months ago, I am worried that if we do this it'll just mean that we move from a continuous deployment model to a 'deploy every x months' model where we spend every deploy debugging many months worth of possible problems in one go.

[yuvipanda]

Continuous deployment really helps, as the amount of changes you deploy is as small as possible. Otherwise whoever is doing the deploy is really signing up for like, whatever amount of unreleased changes is going out. So the 'deploy master of repo2docker on mybinder.org-deploy' was a very explicit choice I made when setting this up, born out of a lot of pain.

[consideRatio]

I think the net amount of work involved in managing repo2docker and mybinder.org goes down by this, and that the separation of concerns makes a person wanting to contribute to repo2docker not need to be a person involved in mybinder.org-deploy operations etc.

I have not made repo2docker releases because of a missing changelog and vague procedures, but now the changelog is in place in a markdown format, and the key blocker remaining for me to engage and make releases there is the coupling as indicated by "ideally we also want to test the refrozen environments by deploying to mybinder.org".

Otherwise whoever is doing the deploy is really signing up for like, whatever amount of unreleased changes is going out.

I think mybinder.org deployments become simpler assuming repo2docker manages to get releases out more regularly.

If we don't get releases out regularly, we have other issues as well, such as "oh but this worked on mybinder.org, but not on repo2docker - whats going on?"

---

Looking back 20 or so merged PRs to bump repo2docker image, it seems like they have been merged mostly by @minrk and @manics. What do you think?

[minrk]

I understand that repo2docker releases are tested on mybinder.org before they are made - sometimes at least, its not part of a documented procedure

What's documented is that mybinder.org runs on repo2docker@main and is continuously updated. We don't deploy specific changes for testing, we always deploy the latest version all the time, unless a change comes in that we specifically want to hold for communication/testing/etc. (like jupyterhub/repo2docker#1287).

We could slow this down. It would probably be good to give ourselves pressure to tag releases of repo2docker more often. If we do go this route, we would probably want to bump the repo2docker release process to releasing every ~week or so (approximately on every merged PR). Lacking active development time, a pretty significant number of PRs to repo2docker are specifically fixing bugs specifically for mybinder.org users, so want to be deployed promptly. Waiting for tests and review on a release PR, that changes the repo2docker merge -> mybinder deploy cycle from ~1hr to probably a few days to fix a bug on mybinder.org. That may be a good thing, but I'm sure will be frustrating sometimes.

I don't think changing whether we deploy releases vs main affects the communication/merge decisions on this repo. It only adds a pretty big, slow step in the middle. It's the same decision - are we ready to deploy current repo2docker@main? If so, now we merge the mybinder.org-deploy PR (if not, we wait to merge). After making this change, we would run a whole release process for repo2docker with the same information and for the same reason, and then run the same process we have now. That would likely benefit users of repo2docker from PyPI, but I'm not sure it improves the situation surrounding the problem discussed here.

[consideRatio]

If we do go this route, we would probably want to bump the repo2docker release process to releasing every ~week or so (approximately on every merged PR).

Oh this surprised me @minrk. From a maintenance chore perspective, it seems to me that its not helpful to not bump that often, and from a user perspective, it seems easier to relate to changes between non-dev version and non-important to get the absolute latest repo2docker updates that hasn't been released yet.

Can you clarify what makes you think we want to bump very often @minrk?

(If we do want to bump this regularly on mybinder.org-deploy, then I figure we should not make that force releases of repo2docker every week, so mybinder.org-deploy should use the dev releases.)

[manics]

I think repo2docker and BinderHub deployments are two separate things:

• mybinder.org and many BinderHub deployments are live web applications. In a commercial context with modern dev-ops practices every PR might be continually and automatically deployed, with or without micro-releases, even if it means 10+ deployments to a live site everyday.
• repo2docker is a both a user-facing command line application which is where the drive for standard releases arises, and a key component of binder where frequent updates is valuable to BinderHub users
• BinderHub is somewhere in the middle

We've had a policy of not making breaking user-facing changes unless unavoidable, the only ones I'm aware of are JupyterLab, updating the default Python version, and the current discussion about updating the base image.

In addition I think release changelogs and Discourse/mybinder announcements have different purposes- the former is a technical outline of changes, the latter should be aimed at end-users who may not be technically knowledgeable.

[minrk]

Can you clarify what makes you think we want to bump very often @minrk?

Yes, from continuous deployment perspective, deploying smaller changes more frequently is generally easier and less disruptive. Adding a release step makes deployments larger and less frequent, and more likely to be disruptive. The downside is, as you said, needing to using judgement in merging the auto-update PRs to decide "this change is big, or should wait for another related PR, an announcement, etc." when almost all should be deployed right away.

Essentially, I think we should continue to deploy repo2docker changes to mybinder.org about as often as we already do. To accomplish that with only released repo2docker versions, we'd need to make releases about as often as we currently deploy to mybinder.org.

[consideRatio]

Thanks for the discussion @yuvipanda @manics @minrk!

With @minrk and @manics merging most repo2docker bump PRs anyhow, and a system summarizing the changes that works, I'm absolutely fine to settle and keep bumping repo2docker to dev releases.