Switch to use reuseable workflows

[peterdudfield]

Detailed Description

Is it worth swicthing to reusable workflows?

Context

• reduces code

Possible Implementation

see code in openclimatefix/.github/.github/workflows/[email protected] for example

[devsjc]

It may not be worth it at the moment for this repository, as it won't play nice with our current workflows for a number of reasons, due to decisions made during the development process:

1. This project requires external binaries to be installed for tests and builds, neccessitating the conda container on the steps. Neither this container nor the external binaries installation would be present on the reusable workflows.
2. This project uses a pyproject.toml file to define the built package and dependencies (see the README section on requirements), which our reusable workflows are not currently set up to utilise (python-test for example looks for setup.py or requirements.txt which are not present in this repo)
3. The workflow is sped up using caching of the build venv between the jobs which would be lost transitioning to reusable workflows
4. The versioning is not carried out via bumpversion as expected via the python-release workflow but rather setuptools inbuilt setuptools-git-versioning plugin based upon semantic-formatted git tags as opposed to github-specific releases.

My thoughts at the moment then are that this would in fact increase code, not reduce it, due to the requirement of either making new reusable workflows or performing backwards-compatible modifications to the currently existing ones.

[peterdudfield]

yea, perhaps we park it for now. Probably worth noting,

1. the old nwp consuerm does use reuseable workflows, so we should be able to use it here. That might mean not using external binaries.

2. I think pip install -e . still wotks with pyproject.toml, so it might be ok.

3. Maybe a good excuse to try to add caching to the reusuable workflows, so all repos benefit from it

4. perhaps we should use bumpversion.

I'm happy to leave it for the moment, and we can come back to it later, if we want to.

[devsjc]

1. Good point!
2. It does but that workflow is looking for setup.py currently so would need modifying
3. For sure
4. I'm not a fan of bumpversion if I'm honest - feel like it clutters commit history and forces a lot of re-pulling when working locally. This nice thing about this git versioning is it's a git native feature we're leveraging so it's ubiquitous in terms of what it means, on github or locally or otherwise.

When I've more time I'm happy to look into making the changes to the reuseable workflows.