Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Add self-hosted runners v2 #4506

Closed
wants to merge 37 commits into from
Closed

[Merged by Bors] - Add self-hosted runners v2 #4506

Changes from 32 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
6e639fe
run bigger tasks on self hosted runners
antondlr Mar 20, 2023
769b130
Merge branch 'unstable' into self-hosted-runners
paulhauner Jul 14, 2023
da921b0
Revert dependency installation
paulhauner Jul 14, 2023
2eb7457
Add libpq-dev
paulhauner Jul 14, 2023
42ca4c8
Empty commit
paulhauner Jul 14, 2023
9a4c459
Run debug tests on self-hosted runners
paulhauner Jul 14, 2023
d369c18
Remove Rust update for self-hosted tasks
paulhauner Jul 14, 2023
eba6dd3
Revert to Github runner for Windows tests
paulhauner Jul 14, 2023
9e5e0f2
Add runners as variables
paulhauner Jul 14, 2023
8bc4e99
Remove sudo when installing libpq-dev
paulhauner Jul 14, 2023
3472e60
Run rustup update when not using self-hosted
paulhauner Jul 14, 2023
e705411
Use repository-level environment variable
paulhauner Jul 14, 2023
853d447
Remove bangs
paulhauner Jul 14, 2023
d85891e
Use `== 'true'` in ternary operator
paulhauner Jul 14, 2023
bc883e6
Add temporary debug stuff
paulhauner Jul 14, 2023
a90df60
Add another debug line
paulhauner Jul 14, 2023
7148b13
Modify debug statements
paulhauner Jul 14, 2023
04deb0f
More debugging
paulhauner Jul 14, 2023
5880179
Start using `github.repository`
paulhauner Jul 14, 2023
4749645
Try using toJson
paulhauner Jul 14, 2023
1cd9e9d
Revert "Try using toJson"
paulhauner Jul 14, 2023
74542b7
Test with just string for `runs-on`
paulhauner Jul 14, 2023
c3625d3
Try fromJson
paulhauner Jul 14, 2023
7db66c7
Implement fromJson globally
paulhauner Jul 14, 2023
e008422
Switch windows to self-hosted
paulhauner Jul 14, 2023
b62e79a
Swap vars to env
paulhauner Jul 15, 2023
c7862f9
Fix ef-tests
paulhauner Jul 15, 2023
d92bf62
Use sudo for self-hosted runners
paulhauner Jul 16, 2023
a0619ac
Add 7zip to windows runner
paulhauner Jul 16, 2023
b57c114
Install `7zip` rather than `7zip.install`
paulhauner Jul 17, 2023
c9476bd
Disable LLVM update for Windows runner
paulhauner Jul 17, 2023
7f45652
Merge branch 'unstable' into self-hosted-runners-paul
paulhauner Jul 17, 2023
2f9243f
Change watch DB host
paulhauner Jul 20, 2023
0adaf83
Add watch host for debug tests too
paulhauner Jul 20, 2023
21be30c
Fix compile error
paulhauner Jul 20, 2023
acb9c29
Remove libpq install
paulhauner Jul 21, 2023
ce4ed6e
Allow `WATCH_HOST` to be `localhost` on Github runners
paulhauner Jul 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 27 additions & 5 deletions .github/workflows/test-suite.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ env:
PINNED_NIGHTLY: nightly-2023-04-16
# Prevent Github API rate limiting.
LIGHTHOUSE_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Enable self-hosted runners for the sigp repo only.
SELF_HOSTED_RUNNERS: ${{ github.repository == 'sigp/lighthouse' }}
jobs:
target-branch-check:
name: target-branch-check
Expand Down Expand Up @@ -48,27 +50,35 @@ jobs:
run: make cargo-fmt
release-tests-ubuntu:
name: release-tests-ubuntu
runs-on: ubuntu-latest
# Use self-hosted runners only on the sigp repo.
runs-on: ${{ github.repository == 'sigp/lighthouse' && fromJson('["self-hosted", "linux", "large"]') || 'ubuntu-latest' }}
needs: cargo-fmt
steps:
- uses: actions/checkout@v3
- name: Get latest version of stable Rust
if: env.SELF_HOSTED_RUNNERS == false
run: rustup update stable
- name: Install Protoc
uses: arduino/setup-protoc@e52d9eb8f7b63115df1ac544a1376fdbf5a39612
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: Install Foundry (anvil)
uses: foundry-rs/foundry-toolchain@v1
- name: Install apt dependencies
if: env.SELF_HOSTED_RUNNERS
run: |
sudo apt-get update
sudo apt-get install libpq-dev # Required for Postgres in ./watch.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could probably remove this once this is baked into the self hosted runner image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed!

- name: Run tests in release
run: make test-release
release-tests-windows:
name: release-tests-windows
runs-on: windows-2019
runs-on: ${{ github.repository == 'sigp/lighthouse' && fromJson('["self-hosted", "windows"]') || 'windows-2019' }}
needs: cargo-fmt
steps:
- uses: actions/checkout@v3
- name: Get latest version of stable Rust
if: env.SELF_HOSTED_RUNNERS == false
run: rustup update stable
- name: Use Node.js
uses: actions/setup-node@v2
Expand All @@ -83,6 +93,7 @@ jobs:
- name: Install make
run: choco install -y make
- uses: KyleMayes/install-llvm-action@v1
if: env.SELF_HOSTED_RUNNERS == false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're good with this but just something worth thinking - Is there any benefit of leaving these conditionals out?

There are a few benefits:

  • CI be always the same dep versions on self-hosted / github-hosted runners
  • For Rust, the workflow will automatically update to latest version as soon as it's released, it could take an extra 15s or more to install. But then we'll know we need to update our runner image.

Downside:

  • it could be a bit slower, but AFAIK rustup update stable just returns immediately if we're already on the same version.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC the self-hosted runners fail if we run that llvm install action.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same with the Rust update ones.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh privileges! happy to leave it as it is 👍

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it would be ideal if we can have the images perform exactly the same. Perhaps we can make an issue to come back for it.

I suspect that we're going to eventually drift into the territory of Github-hosted runners failing, though. If we're not actually testing on the Github-hosted ones they're bound to bit-rot.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's a good point - I'm hesitant to make the workflow more complicated, but if we really want to run GitHub-hosted runners, we could still run them on PRs, and use the self-hosted ones on staging 😛 , but perhaps something to think about later down the line.

with:
version: "15.0"
directory: ${{ runner.temp }}/llvm
Expand All @@ -92,11 +103,13 @@ jobs:
run: make test-release
beacon-chain-tests:
name: beacon-chain-tests
runs-on: ubuntu-latest
# Use self-hosted runners only on the sigp repo.
runs-on: ${{ github.repository == 'sigp/lighthouse' && fromJson('["self-hosted", "linux", "large"]') || 'ubuntu-latest' }}
needs: cargo-fmt
steps:
- uses: actions/checkout@v3
- name: Get latest version of stable Rust
if: env.SELF_HOSTED_RUNNERS == false
run: rustup update stable
- name: Install Protoc
uses: arduino/setup-protoc@e52d9eb8f7b63115df1ac544a1376fdbf5a39612
Expand Down Expand Up @@ -130,18 +143,25 @@ jobs:
run: make test-slasher
debug-tests-ubuntu:
name: debug-tests-ubuntu
runs-on: ubuntu-22.04
paulhauner marked this conversation as resolved.
Show resolved Hide resolved
# Use self-hosted runners only on the sigp repo.
runs-on: ${{ github.repository == 'sigp/lighthouse' && fromJson('["self-hosted", "linux", "large"]') || 'ubuntu-latest' }}
needs: cargo-fmt
steps:
- uses: actions/checkout@v3
- name: Get latest version of stable Rust
if: env.SELF_HOSTED_RUNNERS == false
run: rustup update stable
- name: Install Protoc
uses: arduino/setup-protoc@e52d9eb8f7b63115df1ac544a1376fdbf5a39612
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: Install Foundry (anvil)
uses: foundry-rs/foundry-toolchain@v1
- name: Install apt dependencies
if: env.SELF_HOSTED_RUNNERS
run: |
sudo apt-get update
sudo apt-get install libpq-dev # Required for Postgres in ./watch.
paulhauner marked this conversation as resolved.
Show resolved Hide resolved
- name: Run tests in debug
run: make test-debug
state-transition-vectors-ubuntu:
Expand All @@ -160,11 +180,13 @@ jobs:
run: make run-state-transition-tests
ef-tests-ubuntu:
name: ef-tests-ubuntu
runs-on: ubuntu-latest
# Use self-hosted runners only on the sigp repo.
runs-on: ${{ github.repository == 'sigp/lighthouse' && fromJson('["self-hosted", "linux", "small"]') || 'ubuntu-latest' }}
needs: cargo-fmt
steps:
- uses: actions/checkout@v3
- name: Get latest version of stable Rust
if: env.SELF_HOSTED_RUNNERS == false
run: rustup update stable
- name: Install Protoc
uses: arduino/setup-protoc@e52d9eb8f7b63115df1ac544a1376fdbf5a39612
Expand Down