Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move our runners to aws_autoscaling, since we need a bit more RAM #238

Closed
martun opened this issue Nov 21, 2023 · 1 comment · Fixed by #253
Closed

Move our runners to aws_autoscaling, since we need a bit more RAM #238

martun opened this issue Nov 21, 2023 · 1 comment · Fixed by #253
Assignees

Comments

@martun
Copy link
Contributor

martun commented Nov 21, 2023

After look-ups and Poseidon to the Placeholder proof-system are integrated into our code, our tests will require slightly more RAM (Probably 8 GB is enough for us). Our current runners have less, so we need to change the runners. To do that, we need to create a Dockerfile on which our test will run.

We have a naive infrastructure for self-hosted runners. It can be used to create a proper setup for auto-scalable self-hosted runners: https://github.com/NilFoundation/deployment-infrastructure

We need to write a Dockerfile, similar to 1 or 2.

Then add a script like this:
https://github.com/NilFoundation/proof-market/blob/master/scripts/ci.sh

Use it in CI like this:
https://github.com/NilFoundation/proof-market/blob/master/.github/workflows/main.yml

Requirements for the autoscaling machines:

  • Ubuntu 22.04 x86-64
  • Mac x86
  • Max M1
  • Mac M2

Probably we need two types of machines basing on our RAM requirements:

  • For zk tests, about 12 Gb is probably enough
  • For look-up tests, 64Gb is required now

We also need to test on two different compilers, since we often have trouble tracking compiler-specific changes:

  • clang (14 will work)
  • gcc

Add a tag to run it on aws.
runs-on: ["self-hosted", "aws_autoscaling"]

@nkaskov nkaskov changed the title Move our runners to aws_autoscaling, since we need a bit more RAM to run the tests after Poseidon is added. Move our runners to aws_autoscaling, since we need a bit more RAM Nov 27, 2023
@nkaskov
Copy link
Contributor

nkaskov commented Nov 27, 2023

Probably, you will need to create a couple of volumes with Boost installed from source, which will be attached to the machine every time we run tests.

Also, please check if we can use ccache for optimizing builds in the same way.

@vo-nil vo-nil self-assigned this Nov 28, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
vo-nil added a commit that referenced this issue Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging a pull request may close this issue.

3 participants