Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration tests are very slow and test bench build is not cached properly locally #219

Open
ml-evs opened this issue Nov 27, 2024 · 0 comments

Comments

@ml-evs
Copy link
Member

ml-evs commented Nov 27, 2024

PR #160 comes with a fairly significant performance cost on the integration tests. These are not run by default in the local test suite (can be forced with env var CI=1), but this should probably be revisited soon. There is discussion in the comments of #160, but it seems the main slowdown is the fact that docker-py does not seem to be make good use of the existing cache, perhaps due to using some features not fully supported (docker-py cannot use buildkit).

We can switch over to something that maps more closely to the Docker CLI, which should improve performance. More generally, the integration tests can be a bit slow as the containers are simulating real queues that have their own polling times. At the very least, we can parallelize the integration tests over Slurm/SGE, though this will be a bit fiddly with the current setup. I've added the pytest-durations plugin to track these slow steps.

@ml-evs ml-evs changed the title Integration tests are very slow Integration tests are very slow and test bench build is not cached properly locally Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant