Shrinking multicore

[aviggiano]

Describe the desired feature

Currently, it seems like Echidna uses a single core/process/thread to shrink failed sequences.

In some cases, however, we're interested in using 100% of the machine's resources to extract the results of that particular sequence. For example, this is often the case when I am using stopOnFail: true and workers: N in a multicore setup. I don't care about other failed properties, I only care about that particular one that I know to have failed. The problem is that this takes forever even if I bump up N or the number of cores.

It seems like trying to shrink a sequence on a c5.large instance takes about the same amount of time as on a c5.4xlarge (benchmark pending), which is unexpected.

[ggrieco-tob]

It seems it was already reported here: #1105.. by you 😄

[aviggiano]

Haha yeah I remember that issue, but this one is different.

• [Bug-Candidate]: Echidna multicore produces slower shrinking behavior #1105 relates to how different workers result in a slower shrinking behavior than using 1 worker
• Shrinking multicore #1249 (this issue) relates to how echidna could use all cores to make shrinking run faster

For example, suppose there was a threads: 32 (or something like that); even with workers: 1, I would want echidna to use all of my 32-vCPU instance cores to make shrinking run faster

[ggrieco-tob]

Shrinking with multiple cores is hard with our current (stochastic) approach, since the number of messages required between the core is likely to be high, and that will cause an overhead that will offset the gains for multiple workers. If you can @aviggiano, please test #1250 to see if that improved the shrinking speed on a single worker.

[arcz]

We are actually shrinking on multiple cores, but the synchronization is bad. In #1280, I locked shrinking so that tests are not shared between workers. I think with proper synchronization, it will be possible to share test shrinking between workers, but this requires some code reorganization.