shard unit tests in CI for faster CI runs #5572
Conversation
mat-if
force-pushed
the
mat/shard-ci-tests
branch
2 times, most recently
from
8d00928 to
89fe063
Compare
mat-if
force-pushed
the
mat/shard-ci-tests
branch
from
89fe063 to
64fa8cb
Compare
mat-if
changed the title
.github/workflows/ci.yml
Outdated
| @@ -63,7 +66,7 @@ jobs: | |||
| run: yarn --non-interactive --frozen-lockfile | |||
|
|
|||
| - name: Run tests | |||
| run: yarn test:coverage --maxWorkers=2 --workerIdleMemoryLimit=2000MB | |||
| run: yarn test:coverage --maxWorkers=2 --workerIdleMemoryLimit=2000MB --shard=${{ matrix.shard }}/${{ strategy.job-total }} | |||
There was a problem hiding this comment.
LGTM, but it's not clear to me where strategy.job-total comes from, or how it is configured
There was a problem hiding this comment.
This has some documentation about the strategy context. It's not ideal, because if we for some reason add another matrix, for example for NodeJS versions, we will need to come up with a different solution here. At that point, we may have to build a custom script to manage it because it's not clear to me that github actions has a way to get the length of a specific matrix array or anything.
There was a problem hiding this comment.
Ok, came up with a slightly less brittle solution - defining the matrix jobs as '1/3', '2/3', '3/3', etc. It doesn't feel great but it's less likely to randomly blow up in the future.
the strategy.job_total figure can easily change, which would cause pretty big problems with our tests. if we manually define the entire shard piece instead of trying to calculate it, this makes the job more robust to future changes.
Summary
Shards the main test suite across 3 shards, and slow tests across 2 shards, resulting in a CI runtime reduction from around 15 minutes to around 6.5-7 minutes: https://github.com/iron-fish/ironfish/actions/runs/11486776859
Testing Plan
Re-ran CI workflow multiple times across 2 branches
Documentation
Does this change require any updates to the Iron Fish Docs (ex. the RPC API
Reference)? If yes, link a
related documentation pull request for the website.
Breaking Change
Is this a breaking change? If yes, add notes below on why this is breaking and label it with
breaking-change-rpcorbreaking-change-sdk.