[RFC] Tests: use boost test framework for benchmarks #17063

1. 
   martinus commented at 8:06 pm on October 5, 2019: contributor

   This PR is a brainstorming to gather opinions if there is interest in rewrite the benchmarking framework, to make it more usable and extensible. I currently have a quick hack to test a new API. It shows a few ideas:

   • Move benchmarking code into tests: the macro BENCHMARK creates a boost test with the label benchmark, that is disabled by default. To run all benchmarks, execute ./test_bitcoin --run_test=@benchmark.
   • Configuration & running of a benchmark is one place:

     0Benchmark("resize").Iters(1000).Run([] {
     1   // code
     2});
     

     The configuration can be easily extended with other attributes, e.g. output configuration, automatic runtime configuration, etc.

   Please have a look at my change to prevector_tests.cpp which shows a working example.

   If there is interest in such a refactoring, I can spend some time on it

2. fanquake added the label Tests on Oct 5, 2019
3. fanquake added the label Needs Conceptual Review on Oct 5, 2019
4. martinus force-pushed on Oct 5, 2019
5. use boost test framework for benchmarks b4b1baf864
6. martinus force-pushed on Oct 5, 2019
7. 
   DrahtBot commented at 9:11 pm on October 5, 2019: member

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Conflicts

   Reviewers, this pull request conflicts with the following ones:

   • #16801 (faster & less memory for sync: bulk pool allocator for node based containers by martinus)

   If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

8. 
   laanwj commented at 9:06 am on October 6, 2019: member

   I agree your syntax looks better but—to be completely honest, I think the current benchmark framework is OK. It’s been though quite some iterations, and i don’t see it as a particular priority to completely change it. I also dislike increasing the reliance on boost. (FWIW there is a long-term idea to move away from boost, there’s also some PRs that move away the current tests from the boost test framework)
9. 
   martinus commented at 10:44 am on October 6, 2019: contributor

   Sure, that’s fine. I’ll close this PR.
10. martinus closed this on Oct 6, 2019

    ---

11. 
    laanwj commented at 11:17 am on October 6, 2019: member

    Sure, that’s fine. I’ll close this PR.

    Don’t leave it only up to me though.

12. DrahtBot locked this on Dec 16, 2021

Contributors
martinus DrahtBot laanwj

Labels
Tests Needs Conceptual Review