Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add_tests generic properties and regression tests #109

Open
benoitdenkinger opened this issue Dec 5, 2024 · 2 comments
Open

add_tests generic properties and regression tests #109

benoitdenkinger opened this issue Dec 5, 2024 · 2 comments

Comments

@benoitdenkinger
Copy link
Contributor

I recently hard-coded properties (SREC_TEXT_FILE and SREC_DATA_FILE) that need to be retrieved in the add_tests function (#107). It would be better to pass a list of properties so the add_tests function is more generic.

I also added extra arguments to be able to create multiple tests using the same executable but with different arguments passed to it. I wanted to be able to create regression tests for all the possible parameters, but I only implemented a solution to pass a single argument and a list of value with the TESTCASE and TESTCASE_PARAM add_tests function arguments. It would be better to pass a dictionary to cover an arbitrary number of testcases and corresponding values, unfortunately cmake doesn't support such dictionary data type so I don't know if there is a clean solution to implement this feature. Maybe this should also be a function on it's own which calls add_tests internally.

@benoitdenkinger
Copy link
Contributor Author

There is a GoogleTest module offering a wrapper around CTest. The also have extra arguments and it could be interesting to check if they have any interesting options that we could use to implementing above.

@Risto97
Copy link
Contributor

Risto97 commented Dec 5, 2024

I think Google Test requires you to create tests in C++ like this.
It can work I guess with Verilator, but not sure if it would solve this issue.

The issue with the implementation we have in SoCMake at the moment is that it only works for running executables, but I do recognize that this should be extended to SystemVerilog-on-top simulations with commercial simulators.
The latest pull request should address that: #106

It would be better to pass a dictionary to cover an arbitrary number of testcases and corresponding values, unfortunately cmake doesn't support such dictionary data type so I don't know if there is a clean solution to implement this feature. Maybe this should also be a function on it's own which calls add_tests internally.

Well dictionary can be implemented pretty easily, but I am afraid this would be a bit extreme.
CMake language certainly has many limitations, however the alternatives are not any better.
Hopefully one day someone will rewrite CMake's model in another language.

There are some crazy projects online though that implement Dicts and Classes in CMake:
https://github.com/CMakePP/CMakePPLang
https://github.com/toeb/cmakepp?tab=readme-ov-file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants