fuzz: afl stability for process_message is low; improvement feasible? #22551

1. 
   agroce commented at 11:51 pm on July 25, 2021: contributor

   As shown below, the stability metric is not terrible, but not great. I believe the issue is multithreading, since code coverage runs with gcov show variance in mutex-related code.

   See section 8 of, e.g., https://afl-1.readthedocs.io/en/latest/user_guide.html

2. agroce added the label Feature on Jul 25, 2021
3. MarcoFalke added the label Brainstorming on Jul 26, 2021
4. MarcoFalke added the label Tests on Jul 26, 2021
5. MarcoFalke removed the label Feature on Jul 26, 2021
6. 
   MarcoFalke commented at 10:37 am on July 26, 2021: member

   Stability by itself doesn’t need to be increased if all true bugs are fully reproducible and deterministic. Though, if there is an easy way to increase it, why not?
7. 
   agroce commented at 1:43 pm on July 26, 2021: contributor

   Right, I don’t see a determinism problem here. It can make the fuzzers (somewhat, not sure how to measure) less efficient though, as they chase spurious coverage differences vs. real ones. I suspect this is why some afl ’new edges’ don’t actually ever get merged. They were just lucky hits on mutex/etc. changes. Not a big deal, but I don’t know how much effort it is.
8. 
   fanquake commented at 8:43 am on August 8, 2022: member

   @MarcoFalke is there something for us to follow up on here?
9. 
   agroce commented at 2:28 pm on August 8, 2022: contributor

   Not @MarcoFalke but I would guess it might be worth looking at those numbers from time to time, and if they dip a lot lower than the one I showed above, investigate thoroughly, but otherwise probably not a major concern. It is likely making things more deterministic would somewhat improve fuzzing, but the causes may be such that it would be a lot of work for modest gain.
10. 
    fanquake commented at 3:10 pm on August 8, 2022: member

    Ok. Going to close this for now, as it’s not an active bug / issue etc. If anyone actively fuzzing with afl notices an issue with the numbers they can report / let us know.
11. fanquake closed this on Aug 8, 2022

    ---

12. bitcoin locked this on Aug 8, 2023

Contributors
agroce MarcoFalke fanquake

Labels
Brainstorming Tests