CI: Make Zenodo archival dependent on PyPI publish

[MattF-NSIDC]

Part of doing a release is publishing to PyPI and another part is archiving with Zenodo. Before one of those artifacts is there, we can't really say we've "released". Once either is published, we can't go back. Zenodo just zips up our source code, so it is not likely to fail, but what if PyPI publish fails? Currently, we would have a Zenodo release, so we'd have to do a patch bump to try PyPI again.

I think it makes sense to re-tag to resolve failed PyPI publishes, so it'd be better to have not already published to Zenodo. I ran in to that situation today, hence v0.6.1 :)

Currently, both PyPI publishes and Zenodo archival are triggered by GH Releases. I think we should change our triggers such that contributor tags -> PyPI builds -> contributor makes GH release -> Zenodo archives. If the failure is early, we can just try again by re-tagging.

[betolink]

What's the status of this one @MattF-NSIDC ?

[MattF-NSIDC]

0% progress :)

[asteiker]

@jhkennedy This could be related to the change in our release process to automatically published from a PR? @danielfromearth just created #857 for further discussion on implementation approaches.

[jhkennedy]

🤔 I'd like to hear a little bit more about @mfisher87's motivation here as I personally am inclined to keep it as is.

That is, I'm curious what "we" collectively feel the intent/purpose is for these four things:

1. v* tags (e.g., "v0.11.0")
2. GitHub releases
3. Zenodo archive
4. PyPI package distributions (and the downstream conda-forge distributions)

That is, in my mind a tag is how "we" demarcate a product increment, GitHub releases how we broadcast that to the community, and zenodo is how we preserve that increment and allow it to be specified in "scientific" communication (cited).

The distribution packages on PyPI (and conda-forge) are a mutable subset of all releases, as broken releases can get yanked (e.g., you don't realize it's broken until after the fact). They will also get picked up by the community automatically in many cases when it's installed into new applications and environments. So to me, I don't think we should push to PyPI until we have the release announcement, as that's how we communicate to users what the release is. And once we have a GH release, I want an archive of that increment, irrespective of whether the distribution package is available (especially since you can pip install from a GH link).

Similarly, if we don't want a release announcement until the distribution package is available, I don't see why that wouldn't also apply to conda-forge.

I don't see a problem with vX.Y.Z failing to push to PyPI for whatever reason, and a vX.Y.Z+1 fix that just communicates that it's fixing a failed distribution.

---

I'm also curious to hear the circumstances that led to v0.6.0 failing to go to PyPI (I probably knew once and have since forgotten) and whether that's an expected problem and/or if automation (e.g., #857) would largely resolve that.

[mfisher87]

My main motivation was, IIRC, to avoid creation of a Zenodo archive when our packaging is actually broken and the build and upload to PyPI (and therefore also a pip install from a Git repo URL) would fail.