Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split tests into smaller sub-tests #316

Open
asmeurer opened this issue Nov 20, 2024 · 0 comments
Open

Split tests into smaller sub-tests #316

asmeurer opened this issue Nov 20, 2024 · 0 comments
Labels
low priority Low priority issue

Comments

@asmeurer
Copy link
Member

Right now there is one test per function. This has worked so far, but it has limitations. The main problem is that if a single thing fails in a test, you can't really get at any of the assertions that come after it. And even if hypothesis could generate separate examples for separate assertions, if you are maintaining a library and you want to xfail a given assertion, you have to xfail the whole test.

This is going to be important for reporting, because it will make it easier to report on exactly which things a given function isn't compliant on. However, this is also a pretty big refactor. It also could be challenging in some cases if some assertion actually depends on a previous assertion working to make sense (e.g., maybe some assertion will implicitly assume a given shape because it was already asserted). For this reason, I would consider this to be lower priority for now, and not worry about implementing it until it becomes needed for reporting (although it certainly doesn't hurt to start thinking about how it might work).

@asmeurer asmeurer added the low priority Low priority issue label Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
low priority Low priority issue
Projects
None yet
Development

No branches or pull requests

1 participant