Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Smoke test takes too long to run #3076

Open
hdevalence opened this issue Sep 22, 2023 · 5 comments
Open

Smoke test takes too long to run #3076

hdevalence opened this issue Sep 22, 2023 · 5 comments
Labels
A-CI/CD Relates to continuous integration & deployment of Penumbra _P-medium Medium priority

Comments

@hdevalence
Copy link
Member

Describe the bug
A recent smoke test run took 16 minutes. Previously, we had it running in 4-6 minutes.

We should identify why it takes so much longer and find a way to cut the runtime.

@erwanor
Copy link
Member

erwanor commented Sep 22, 2023

This is likely because we added submitting delegator voting / proposal submission to the smoke test in #2969

@erwanor erwanor moved this to Next (Steal from here) in Testnets Sep 22, 2023
@conorsch
Copy link
Contributor

We have the smoke tests set to run for an extra two minutes after the tests have completed, which we can easily shave off. More substantial gains can be found on the CometMock spike PR though: #2845 since we can dial the block time down even lower than the 500ms supported by CometBFT. Right now the delegate_and_undelegate test is failing, but I haven't even implemented block time travel yet, and the whole smoke test clocks in at <5m, which is a huge win. Will keep trying.

@conorsch conorsch moved this from Next (Steal from here) to In Progress (Already claimed) in Testnets Sep 26, 2023
@hdevalence
Copy link
Member Author

Is it desirable to use cometmock for the smoke test, though? Don't we want the smoke test to be testing the actual code we'll be running?

@conorsch conorsch moved this from In Progress (Already claimed) to Next (Steal from here) in Testnets Sep 29, 2023
@conorsch conorsch added the A-CI/CD Relates to continuous integration & deployment of Penumbra label Oct 3, 2023
@aubrika aubrika added this to Penumbra Oct 30, 2023
@github-project-automation github-project-automation bot moved this to 🗄️ Backlog in Penumbra Oct 30, 2023
@hdevalence hdevalence added the _P-medium Medium priority label Feb 8, 2024
@hdevalence
Copy link
Member Author

We can solve this problem by removing tests from the smoke test as our application-wide integration testing story matures (cc @cratelyn)

@cratelyn
Copy link
Contributor

cratelyn commented Feb 9, 2024

x-ref: #3588

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-CI/CD Relates to continuous integration & deployment of Penumbra _P-medium Medium priority
Projects
Status: Backlog
Status: Next
Development

No branches or pull requests

4 participants