[Spec lib improvements] Overhaul Spec execution pipeline #140
Labels
feature-request
Any request for a new feature of the language. Includes both syntax and library features.
stdlib
Any issue relating to the standard library (Myst code) of Myst.
This is part of a family of issues relating to Spec library improvements.
Currently, when an AssertionFailure is raised, the Spec library immediately halts, shows the failure, then exits with a non-zero exit status.
While this can be useful in some cases (e.g., long running suites), in most cases, it's nice to continue running the remaining specs and then show all of the failures at the end.
I think the proper way to implement this change would be to split spec execution into two phases. The first phase is the "definition" phase, where the Spec code is read, and each
it
block is saved as aSingleSpec
instance that captures the block to execute, and added to a list of Specs to run.The second phase iterates that list, executing each block and tracking the result as either a pass or failure. Specs that fail would get added to a separate list of failing specs, and after all specs have finished executing, the failed specs would be printed out in a failure summary.
Using this approach will eventually allow us to filter Specs based on command line options (line numbers, tags, previously-failed specs, etc.).
The text was updated successfully, but these errors were encountered: