Separate out "badges" for different stages

[yarikoptic]

ATM, for each OS we have a single badge which summarizes

• git-annex built ok
• git-annex tested ok in multiple filesystem test scenarios
• datalad of 3 "versions" (master, maint, release) tested ok using this git-annex

It would have been nice to have at least 3 separate badges for each. That would be also of help for @joeyh since he would likely care mostly about only first 2.

But I am not sure how/if this could be accomplished.
Could we e.g. split into multiple workflows and somehow trigger one from another?
Or may be github is working on providing per step badges? (have not researched yet)

Any ideas @datalad/developers?

[yarikoptic]

Could we e.g. split into multiple workflows and somehow trigger one from another?

most likely it is the way to go: so we just need to split the workflow,

• move datalad testing into a separate workflow
• keep only git-annex related in existing one, and if build succeeds, trigger a run of a corresponding (different OSes, although may be time to generalize?) "datalad-tests" workflow.
• to separate between versions of datalad, add input option "datalad-version" to the "datalad-tests" workflow defaulting to e.g. "master", see https://docs.github.com/en/free-pro-team@latest/actions/reference/events-that-trigger-workflows#manual-events and then we could trigger 3 (for "release", "maint", "master").

[yarikoptic]

NB not yet 100% sure it is the desired way to proceed, since might need more orchestration/tracking of action builds to possibly support PRs for git-annex code itself (e.g. like now I wish we had it setup so I could quickly try out possible solutions to https://git-annex.branchable.com/bugs/Buggy_external_special_remote_stalls_after_7245a9e/)

[yarikoptic]

One other idea which came to my mind is to

• have a final step in each workflow which runs AFTER all other steps (but even if they fail; if that is possible) which would
  • query github API on the status of the other steps (if that is possible while entire workflow is not yet "done")
  • save those results into a dedicated status-{workflow} branch as some .json file (not yet sure what for really ;)) + render badges for all steps
  • git push -f that branch to this repo
• README.md then could have a table pointing to all those badges in workflow specific branches

@jwodder do you think above could be possible or see any better way to have this issue addressed?

[jwodder]

@yarikoptic

have a final step in each workflow

Workflows don't have steps; they have jobs, and jobs have steps. A step can be added to a job that runs even if previous steps fail, but you can't make a job run after a failed job. Even if a step is added to the end of each job that reports the job's status, we would run into race conditions/concurrency issues due to multiple test-annex and test-datalad jobs running at once and thus potentially pushing to the workflow's status branch at once. We could create separate status branches for each job in each workflow, which would lead to one badge per job, which may be too many badges.

The only other way I can think of to do this is to have a scheduled action periodically check the statuses of the jobs in the latest workflow runs and update either the README or a badges folder/branch.

[yarikoptic]

We could create separate status branches for each job in each workflow, which would lead to one badge per job, which may be too many badges.

In general I am not afraid of branches, more scared of the tags ATM ! ;)
I think that would be ok since we need all of them! so each badge would have its own branch, e.g. status-{workflow_codename}-{job}[-{matrix_id}], e.g. status-build-macos-test-annex-normal, which would just carry that single badge image... indeed would be a lot. Let's think more then.

race conditions/concurrency issues due to multiple test-annex and test-datalad jobs running at once

just make it upload of badges on CRON jobs off the master branch. Should not be colliding AFAIK. If we were to be real fancy, could add latest-release- branches and abuse for the builds of the latest release and show that one too separately.

re branches -- I thought we might be able to (ab)use refs directly. So I created a temp branch with image and pushed it into a "refs" which aren't "heads" (branches) (git push origin temp-badges:refs/badges/the-upgrayedd), and it shows up on https://github.com/datalad/git-annex/tree/badges/the-upgrayedd but you can't choose any file or directory, and https://raw.githubusercontent.com/datalad/git-annex/badges/the-upgrayedd/README.md also doesn't work -- I guess all of those urls require them to be refs/heads. going by commit works, but it would not work for us since it would then be needing to change README.md every time.