Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI should produce different error codes #1064

Open
konnov opened this issue Jul 20, 2023 · 2 comments
Open

CLI should produce different error codes #1064

konnov opened this issue Jul 20, 2023 · 2 comments
Assignees

Comments

@konnov
Copy link
Contributor

konnov commented Jul 20, 2023

Currently, if a test in cli-tests.md returns a non-zero exit code, we do not know, whether it was the expected behavior or not. For instance, if we expect the simulator to find a violation, and the tool is passed incorrect CLI arguments, the test is considered as passed, though it should not.

We should differentiate exit codes that are caused by different exit paths. For instance, all CLI parsing should return one exit code and quint run and quint test should return another error code.

@konnov konnov added the devops label Jul 20, 2023
@p-offtermatt
Copy link
Member

Just wanted to express interest in this, it would be helpful to be able to tell whether an invariant violation was found or not just from the exit code not only for testing, but also for user-side tooling :)

@konnov konnov self-assigned this Nov 22, 2023
@konnov
Copy link
Contributor Author

konnov commented Nov 22, 2023

I've added it on my stack

@shonfeder shonfeder removed the devops label Jan 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants