Whether my implementation for maximum_search_time incorrect or its could not be work?

[longvd336]

I want to apply Pynguin to generate the test cases for my project in a specific time. However, the "maximum_search_time" option in Pynguin's command could not work. My command was bellow. I do not know whether my implementation is correct or if there are some problems with Penguin. Even if I set the maximum search time, the algorithms will run a long time and will not stop if they cannot generate a test suite. The docker container of the module I want to generate the test stays up forever, as I observe. I built Penguin using Docker.

CMD="docker run --rm \ -v "${PROJECT_PATH}:/input:ro" \ -v "${OUTPUT_PATH}:/output" \ -v "$(dirname "$REQUIREMENTS_PATH"):/package:ro" \ pynguin-coverage \ --project-path /input \ --module-name $MODULE_NAME \ --output-path /output \ --maximum_search_time 120 \ --seed $SEED \ --coverage-metrics BRANCH \ --create-coverage-report True \ --report-dir $OUTPUT_PATH \ "

[darklambda]

Hi! If I remember correctly, the maximum_search_time only makes Pynguin stop by checking the time passed after the generation of the initial population of each Search Algorithm (which is updated after every iteration). Did you check the logs of the Docker container to see if the actual search starts?

[stephanlukasczyk]

As @darklambda pointed out, --maximum-search-time only affects the test-generation algorithm, e.g., DynaMOSA or MIO, but not the other components of the Pynguin framework. Besides, this check is only applied at every loop iteration. Thus, if one iteration of the test-generation loop lasts long, e.g., because the execution of the generated test cases lasts long, the timeout will probably applied a couple of seconds later.

Regarding your concrete issue: could you please provide a fully working minimal example that shows this behaviour? Otherwise, I could only guess what the actual reason could be.