Mutation test brainstorming #24949

issue brunoerg opened this issue on April 22, 2022
  1. brunoerg commented at 9:01 PM on April 22, 2022: contributor

    Hi, everyone!

    I've been researching about mutation tests for Bitcoin Core since a while. I'm opening this PR to concentrate the discussions about it here since I couldn't find any similar issue. Some questions:

    • Have you ever used any mutation lib/app for C++? Which are? Pro and cons?
    • How could we use it here?
    • What approaches are important to have? What do we want to mutate? Operators? Strategies?
  2. brunoerg added the label Feature on Apr 22, 2022
  3. brunoerg commented at 9:03 PM on April 22, 2022: contributor

    I met universalmutator by #22690 and I have been using it and just noticed some pros and cons. I just added it on my Core fork so you can test it as well. See: https://github.com/brunoerg/bitcoin/blob/2022-04-universal-mutation/test/mutation/mutation.py

    Pros: Use regex for mutation, scriptable, can be used in CI (although I think we shouldn't btw) and the module to analyze the 'mutators' is very good. We can set different "functional/unit/whatever" script for each file we've mutated (some libs/apps only allow you to set one script so it will be used for all 'mutators', this is bad because we don't want to execute test_runner ou make check for each one). See [here](" https://github.com/brunoerg/bitcoin/blob/2022-04-universal-mutation/test/mutation/mutation.py") I have a dictionary where I put the file to be mutated and the functional test to test it.

    Cons: It's not focused 100% on C++ - it's a generalist tool, I had to open a PR to include some C++ rules.


    I've also been testing mutate_cpp and I'll give my feedback here soon.

  4. brunoerg closed this on Feb 27, 2024

  5. bitcoin locked this on Feb 26, 2025
Contributors
Labels

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-02 12:14 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me