You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, if a test in cli-tests.md returns a non-zero exit code, we do not know, whether it was the expected behavior or not. For instance, if we expect the simulator to find a violation, and the tool is passed incorrect CLI arguments, the test is considered as passed, though it should not.
We should differentiate exit codes that are caused by different exit paths. For instance, all CLI parsing should return one exit code and quint run and quint test should return another error code.
The text was updated successfully, but these errors were encountered:
Just wanted to express interest in this, it would be helpful to be able to tell whether an invariant violation was found or not just from the exit code not only for testing, but also for user-side tooling :)
Currently, if a test in cli-tests.md returns a non-zero exit code, we do not know, whether it was the expected behavior or not. For instance, if we expect the simulator to find a violation, and the tool is passed incorrect CLI arguments, the test is considered as passed, though it should not.
We should differentiate exit codes that are caused by different exit paths. For instance, all CLI parsing should return one exit code and
quint run
andquint test
should return another error code.The text was updated successfully, but these errors were encountered: