On the issue of protocol coverage collection

[lily11111222]

Hi, may I ask if you have integrated EnvFuzz into profuzzbench? Or how did you build the client side of the protocol and collect coverage?
Thank you very much!

[lily11111222]

May I ask what tools do you use to collect coverage?

[GJDuck]

I believe we used gcov and @mengrj can confirm.

However, there may be an issue gcov and the repo version of EnvFuzz . I need to check when I am back.

[lily11111222]

Thank you for your reply. I also use gcov to collect, but I don't understand how to collect. In fact, when I tested live555 with EnvFuzz, I started the./env-fuzz fuzz command, but didn't see live555's port 8554 start listening, which could mean live555 didn't start, which confused me.

[lily11111222]

Please let me know if you have checked. Thanks~

[lily11111222]

Hi, may I ask will the object be restarted by EnvFuzz when replay or fuzz after record (e.g. I execute./env-fuzz fuzz)? Or do I need to start it manually before fuzz?

[GJDuck]

Hi, can you please clarify what you mean by "object"?

[lily11111222]

Oh I'm sorry. That was a bad choice of words. I mean the target program, such as gnome-calculator or live555.

[GJDuck]

Yes, the program is automatically (re)started each time the env-fuzz command is run. There is no need to do anything manually. For example, the workflow for gnome-calculator is simply:

    $ ./env-fuzz record gnome-calculator
    $ ./env-fuzz fuzz

For the fuzz (or replay) commands, gnome-calculator is run automatically again under-the-hood.

Note that you will only see the GUI for the record command. For replay/fuzz, you will not see the GUI because the gnome-calculator is interacting directly with the replay infrastructure rather than the real X11 server. But gnome-calculator is still running.

[lily11111222]

Alright, so I use the './env-fuzz record live555/testProgs/testOnDemandRTSPServer' and then use the './env-fuzz fuzz', live555 should also start right?

[GJDuck]

Yes, assuming nothing went wrong. The program will be running, but it will be exclusively interacting with the EnvFuzz replay infrastructure.

Thus in replay/fuzz mode, the program will not open any "real" (OS-level) socket. Rather, socket I/O system calls will be intercepted and emulated by the EnvFuzz infrastructure, and serviced/fuzzed from the data recorded in the RECORD.pcap.gz file.

[lily11111222]

Ok, thanks, but I'm still wondering why gcov can't collect coverage after fuzz, and the timeout option for EnvFuzz doesn't seem to be working, I'd appreciate any suggestions.

[GJDuck]

Ok, thanks, but I'm still wondering why gcov can't collect coverage after fuzz

I will have to look into the gcov issue eventually, but it is a low priority at the moment.

The timeout only applies to each individual test case, and it is not a global timeout for the overall fuzz campaign. The latter option does not exist yet, but probably should be added.

[lily11111222]

Got it, thanks for reply.

[GJDuck]

I added a --max-time SECONDS option to env-fuzz.

[lily11111222]

Okay. And I want to know can EnvFuzz to be integrated into profuzzbench and use a similar approach in profuzzbench to gather coverage?

[GJDuck]

Unfortunately, profuzzbench integration is not-so-easy since the EnvFuzz interface and fuzz target is somewhat different than other fuzzers.

[lily11111222]

Yes, I wanted to compare the coverage difference between EnvFuzz and AfLnet-based fuzzer, but EnvFuzz's coverage collection has been unsuccessful.
I tried to replay the interesting mutant interaction from fuzz over the target version compiled with the -fprofile-arcs -ftest-coverage option, but it never succeeded in generating the gcda file. I found that record could generate gcda files, but replay and fuzz could not generate them. I guess this is due to replay and fuzz when the system call is emulated and not actually executed?