Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate circle-ci workflow to github actions #991

Merged
merged 10 commits into from
Aug 13, 2024

Conversation

cooldracula
Copy link
Contributor

@cooldracula cooldracula commented Aug 8, 2024

This PR adds a number of github workflows that are a near direct port of the analogous jobs in the existing .circleci/then.yml file.

The specific mapping:

circleci gh action
lint lint-and-check-licenses.yaml
test test-runtime-benchmarks.yaml
build build-and-run-node-test.yaml
chainspecs check-chainspecs.yaml
documentation check-doc-build.yaml

This PR would resolve one of the tasks in our devops-infra issue #59


note: I did not remove the .circleci directory as part of this PR as there are other commands in the then.yml that, while not used in the jobs, might still be used or useful somewhere else? For example: the comment-on-pr command

Copy link
Contributor

@ameba23 ameba23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥 great to have this. and kinda lucky that CI is failing because we can see that the same errors are coming up in circle CI as gh actions.

Re: the comment-on-pr thing, i don't know what that is for, i guess it makes a pr comment based on ci output? it looks like john added it.

Was there anything else in the circle CI file that isn't migrated? To me it looks like thats everything. If not then fine by me to go ahead and remove the circle ci script.

Copy link
Collaborator

@HCastano HCastano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this 🤩

I haven't checked to see what the CI failures are about, but let me know if you need help addressing them

.github/actions/install-dependencies/action.yaml Outdated Show resolved Hide resolved
.github/actions/install-dependencies/action.yaml Outdated Show resolved Hide resolved
.github/workflows/build-and-run-node-test.yaml Outdated Show resolved Hide resolved
.github/workflows/build-and-run-node-test.yaml Outdated Show resolved Hide resolved
@cooldracula
Copy link
Contributor Author

Thanks for this 🤩

I haven't checked to see what the CI failures are about, but let me know if you need help addressing them

Thank you (and @ameba23 ) for the review! For the CI failures, they are all failing during compilation due to the following Rust error:

error[E0282]: type annotations needed for `Box<_>`
  --> /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/time-0.3.34/src/format_description/parse/mod.rs:83:9
   |
83 |     let items = format_items
   |         ^^^^^
...
86 |     Ok(items.into())
   |              ---- type must be known at this point
   |
help: consider giving `items` an explicit type, where the placeholders `_` are specified
   |
83 |     let items: Box<_> = format_items
   |              ++++++++

It seems to be an error bubbling up from a dependency? I would love help addressing it though.

@ameba23
Copy link
Contributor

ameba23 commented Aug 9, 2024

@cooldracula this needs the time crate to be bumped to 0.3.36. It looks like this is fixed now in master (there are constantly things being updated by dependabot). So i think if you merge or rebase master CI should pass. TBH you could probably just merge as is, as the issue is anyway unrelated to this PR.

@HCastano
Copy link
Collaborator

HCastano commented Aug 9, 2024

Yeah Peg is right, this was fixed in master like two weeks ago 😄

@HCastano
Copy link
Collaborator

HCastano commented Aug 9, 2024

@cooldracula looks like the GHA job failed with a timeout now. Are we using different runners for Circle CI and GHA?

Ideally we don't just work around this by bumping the timeout to, say, 60 minutes

This was originally added as an analog to the circle-ci's "no_output_timeout",
but they are not actually analogs.  circle-ci's step measures how long
to wait without any output, while gh's timeout is how long to run the
step in general before timing out, regardless of output.  This could
cause the job to timeout when it was still successfully running, and
still within github's usage limits.
@cooldracula
Copy link
Contributor Author

cooldracula commented Aug 12, 2024

@cooldracula looks like the GHA job failed with a timeout now. Are we using different runners for Circle CI and GHA?

Ideally we don't just work around this by bumping the timeout to, say, 60 minutes

@HCastano: We are using different runners for the two services. In Circle-Ci, we are using a self-hosted runner (Linux X-Large), while with GHA, we are using the standard github hosted runner. If we wanted, we could use the custom GHA runner, currently used for the release builds, for this job too?

Also, for the failed job, I think this was due to a timeout I should not have added to the testing step. The original circle-ci job had the property no_output_timeout: 45m, so I added the most closely equivalent github property timeout-minutes. However, github's timeout is based on the total execution time of the step, regardless of if the step was still producing output. So the job running on smaller machine, with a hard timeout, failed. I removed the timeout property in my most recent commit (b6a3eb2) as it introduced a constraint not present in the original job.

This job should now complete successfully, though slower than on circle-ci. If we want it to be faster, it might be good to use our larger custom runner for it.

All relevant workflows have been ported to github actions.
@HCastano
Copy link
Collaborator

@cooldracula yeah ideally we keep runners with similar specs.

I don't see the GHA build and test step for the tests runs though 🤔 I'd be curious to know how long that step takes with the default runners

@cooldracula
Copy link
Contributor Author

ideally we keep runners with similar specs.

I agree, @HCastano . I updated the job to use our custom runner(core-build-runner), whose specs are roughly equivalent to our circle-ci runner.

I don't see the GHA build and test step for the tests runs though 🤔 I'd be curious to know how long that step takes with the default runners

This is the correct behaviour for this PR, as the job is set to only run if there was a change to files within the crates,pallets,node, or runtime directories.

I pushed a small edit to crates/README.md to trigger the job. It completed successfully in 39 minutes. This is about the same as circle-ci. In a recent circle-ci run, the equivalent job completed in 40 minutes.

@HCastano
Copy link
Collaborator

@cooldracula after that last comment feel free to merge this!

@cooldracula cooldracula merged commit 2fc472f into master Aug 13, 2024
7 of 8 checks passed
@cooldracula cooldracula deleted the zach/migrate-circleci-jobs branch August 13, 2024 21:01
ameba23 added a commit that referenced this pull request Aug 15, 2024
* master:
  Bump serde_json from 1.0.124 to 1.0.125 in the patch-dependencies group (#1007)
  Signing flow with derived accounts (#990)
  Add `network-jumpstart` command to `entropy-test-cli` (#1004)
  Refactor reshare (#994)
  Migrate circle-ci workflow to github actions (#991)
  Delete old keyshare if not in next_signers (#999)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants