Deduplicate CMerkleBlock construction code, add test coverage #11293

pull jamesob wants to merge 2 commits into bitcoin:master from jamesob:dedup-cmerkleblock changing 7 files +119 −41
  1. jamesob commented at 5:00 PM on September 9, 2017: member

    What started as a simple task to add test coverage ended up giving way to a light refactoring. This consolidates the mostly-identical CMerkleBlock constructors into one (using C++11 constructor delegation) and adds coverage for the by-txids construction case.

    Before

    selection_006

    After

    selection_008

  2. jamesob force-pushed on Sep 9, 2017
  3. fanquake added the label Refactoring on Sep 10, 2017
  4. in src/merkleblock.h:164 in 5bd892c14b outdated
     152 | @@ -154,6 +153,10 @@ class CMerkleBlock
     153 |          READWRITE(header);
     154 |          READWRITE(txn);
     155 |      }
     156 | +
     157 | +private:
     158 | +    // Combined constructor to consolidate code
     159 | +    CMerkleBlock(const CBlock& block, CBloomFilter* filter, const std::set<uint256>* txids);
    


    promag commented at 9:26 AM on September 11, 2017:

    Instead of pointers, you can make it receive references, just pass either CBloomFilter() or std::set<uint256>(), and remove the != nullptr conditions above. Note that CBloomFilter is optimized when it is empty.

    If you do this then this constructor can be public.


    jamesob commented at 12:40 AM on September 12, 2017:

    I don't think this will work in the case of filter being unused. I'm getting the following compilation error:

    In file included from merkleblock.cpp:6:0:
    
    merkleblock.h: In constructor ‘CMerkleBlock::CMerkleBlock(const CBlock&, const std::set<uint256>&)’:
    merkleblock.h:145:93: error: invalid initialization of non-const reference of type ‘CBloomFilter&’ from an rvalue of type ‘CBloomFilter’
         CMerkleBlock(const CBlock& block, const std::set<uint256>& txids) : CMerkleBlock(block, CBloomFilter(), txids) {}
                                                                                                 ^~~~~~~~~~~~~~
    merkleblock.h:148:5: note:   initializing argument 2 of ‘CMerkleBlock::CMerkleBlock(const CBlock&, CBloomFilter&, const std::set<uint256>&)’
         CMerkleBlock(const CBlock& block, CBloomFilter& filter, const std::set<uint256>& txids);
         ^~~~~~~~~~~~
    

    Apparently we can't create an unnamed, non-const CBloomFilter reference from an rvalue. I'm not sure how to avoid this when using constructor delegation.


    sipa commented at 2:23 AM on September 12, 2017:

    You're not allowed to pass a temporary into a method that expects a non-const reference. Non-const references are intended for output arguments, and passing a temporary would mean there is effectively no output argument the caller can observe.

    It seems to be that using a pointer which can be optionally nullptr is the correct approach here.


    promag commented at 9:48 PM on September 12, 2017:

    Should have tested my suggestion. Agree with this approach.

  5. practicalswift commented at 9:29 AM on September 11, 2017: contributor

    Concept ACK. Very nice! :-)

  6. promag commented at 9:31 AM on September 11, 2017: member

    Separate refactor and test in 2 commits?

  7. jamesob force-pushed on Sep 12, 2017
  8. jamesob commented at 12:46 AM on September 12, 2017: member

    @promag I've split this into two commits as requested. Thanks for the look.

  9. in src/merkleblock.cpp:30 in 560d6bac75 outdated
      27 | +        if (txids != nullptr && (*txids).count(hash)) {
      28 |              vMatch.push_back(true);
      29 | -        else
      30 | +        } else if (filter != nullptr && (*filter).IsRelevantAndUpdate(*block.vtx[i])) {
      31 | +            vMatch.push_back(true);
      32 | +            vMatchedTxn.push_back(std::make_pair(i, hash));
    


    sipa commented at 2:12 AM on September 12, 2017:

    You can avoid constructing an intermediate std::pair<unsigned int, uint256>, by using:

    vMatchedTxn.emplace_back(i, hash);
    
  10. in src/merkleblock.cpp:26 in 560d6bac75 outdated
      22 | @@ -48,10 +23,14 @@ CMerkleBlock::CMerkleBlock(const CBlock& block, const std::set<uint256>& txids)
      23 |      for (unsigned int i = 0; i < block.vtx.size(); i++)
      24 |      {
      25 |          const uint256& hash = block.vtx[i]->GetHash();
      26 | -        if (txids.count(hash))
      27 | +        if (txids != nullptr && (*txids).count(hash)) {
    


    sipa commented at 2:13 AM on September 12, 2017:

    Use txids->count(hash) instead.

  11. in src/merkleblock.cpp:28 in 560d6bac75 outdated
      22 | @@ -48,10 +23,14 @@ CMerkleBlock::CMerkleBlock(const CBlock& block, const std::set<uint256>& txids)
      23 |      for (unsigned int i = 0; i < block.vtx.size(); i++)
      24 |      {
      25 |          const uint256& hash = block.vtx[i]->GetHash();
      26 | -        if (txids.count(hash))
      27 | +        if (txids != nullptr && (*txids).count(hash)) {
      28 |              vMatch.push_back(true);
      29 | -        else
      30 | +        } else if (filter != nullptr && (*filter).IsRelevantAndUpdate(*block.vtx[i])) {
    


    sipa commented at 2:14 AM on September 12, 2017:

    Use filter->IsRelevantAndUpdate(*block.vtx[i]).

  12. jamesob force-pushed on Sep 12, 2017
  13. jamesob commented at 2:58 AM on September 12, 2017: member

    Changed per feedback; thanks @sipa.

  14. in src/test/bloom_tests.cpp:17 in 1f6fe1317b outdated
      13 | @@ -14,6 +14,7 @@
      14 |  #include "uint256.h"
      15 |  #include "util.h"
      16 |  #include "utilstrencodings.h"
      17 | +#include "primitives/block.h"
    


    promag commented at 9:49 PM on September 12, 2017:

    Nit, move before #include "random.h".

  15. in src/merkleblock.cpp:26 in 1f6fe1317b outdated
      22 | @@ -48,10 +23,14 @@ CMerkleBlock::CMerkleBlock(const CBlock& block, const std::set<uint256>& txids)
      23 |      for (unsigned int i = 0; i < block.vtx.size(); i++)
      24 |      {
      25 |          const uint256& hash = block.vtx[i]->GetHash();
      26 | -        if (txids.count(hash))
      27 | +        if (txids != nullptr && txids->count(hash)) {
    


    promag commented at 9:51 PM on September 12, 2017:

    Nit, just

    if (txids && txids->count(hash)) {
    

    Same below.

  16. promag commented at 9:52 PM on September 12, 2017: member

    utACK, some nits though.

  17. jamesob force-pushed on Sep 13, 2017
  18. jamesob commented at 3:40 PM on September 13, 2017: member

    Incorporated feedback; thanks @promag.

  19. jamesob commented at 7:58 PM on September 15, 2017: member

    @promag @sipa anything else you'd like to see done here?

  20. sipa commented at 12:26 AM on September 16, 2017: member

    utACK 17dd0973a63706717905bb3865129a32a4287adf

  21. in src/test/merkleblock_tests.cpp:4 in 17dd0973a6 outdated
       0 | @@ -0,0 +1,61 @@
       1 | +// Copyright (c) 2012-2017 The Bitcoin Core developers
       2 | +// Distributed under the MIT software license, see the accompanying
       3 | +// file COPYING or http://www.opensource.org/licenses/mit-license.php.
       4 | +                          
    


    TheBlueMatt commented at 7:37 PM on September 19, 2017:

    You've added a bunch of trailing whitespace in this file (as well as test_bitcoin.cpp in 2 places and test_bitcoin.h in one). If you re-push this PR should fail travis due to #11300.


    jamesob commented at 3:37 AM on September 21, 2017:

    Ah, totally right. Editor config adjusted -- thanks.

  22. in src/test/merkleblock_tests.cpp:38 in 17dd0973a6 outdated
      23 | +    uint256 txhash1 = uint256S("0x74d681e0e03bafa802c8aa084379aa98d9fcd632ddc2ed9782b586ec87451f20");
      24 | +    txids1.insert(txhash1);
      25 | +    CMerkleBlock merkleBlock(block, txids1);
      26 | +
      27 | +    BOOST_CHECK_EQUAL(merkleBlock.header.GetHash().GetHex(), block.GetHash().GetHex());
      28 | +    BOOST_CHECK_EQUAL(merkleBlock.vMatchedTxn.size(), 0);
    


    TheBlueMatt commented at 9:00 PM on September 19, 2017:

    Seems a strange thing to test, here? Maybe document on merkleblock.h what this thing even means? (ie "Set only when constructing from a bloom filter to allow testing the set of things which matched the bloom filter directly").


    jamesob commented at 3:38 AM on September 21, 2017:

    Comment added inline with test as well as merkleblock.h. Basically just used your copy directly.

  23. in src/test/merkleblock_tests.cpp:35 in 17dd0973a6 outdated
      30 | +    std::vector<uint256> vMatched;
      31 | +    std::vector<unsigned int> vIndex;
      32 | +
      33 | +    BOOST_CHECK_EQUAL(merkleBlock.txn.ExtractMatches(vMatched, vIndex).GetHex(), block.hashMerkleRoot.GetHex()); 
      34 | +    BOOST_CHECK_EQUAL(vMatched.size(), 1);
      35 | +    BOOST_CHECK_EQUAL(vMatched[0].ToString(), txhash1.ToString());
    


    TheBlueMatt commented at 9:02 PM on September 19, 2017:

    Would be neat to also test the vIndex values here, too, no?


    jamesob commented at 3:38 AM on September 21, 2017:

    Good idea! Added.

  24. TheBlueMatt commented at 9:03 PM on September 19, 2017: member

    Looks good, would be nice to test a few more things since the goal is to add test coverage.

  25. Consolidate CMerkleBlock constructor into a single method
    Incorporates feedback suggested by @sipa, @promag, @TheBlueMatt.
    5ab586f90b
  26. Add tests for CMerkleBlock usage with txids specified 46ce223d15
  27. jamesob force-pushed on Sep 21, 2017
  28. jamesob commented at 4:15 AM on September 21, 2017: member

    @TheBlueMatt thanks for the look. I've updated the tests to be a little more comprehensive per your feedback.

  29. TheBlueMatt commented at 8:10 PM on September 21, 2017: member

    utACK 46ce223d15d4111d096f6342eb6f526d2507d7d7

  30. sipa commented at 8:20 PM on September 21, 2017: member

    utACK 46ce223d15d4111d096f6342eb6f526d2507d7d7

  31. jamesob commented at 4:37 PM on September 23, 2017: member

    Anything else needed here @laanwj?

  32. gmaxwell approved
  33. gmaxwell commented at 7:43 PM on September 25, 2017: contributor

    ACK.

  34. jnewbery commented at 3:37 PM on October 2, 2017: member

    Tested ACK 46ce223d15d4111d096f6342eb6f526d2507d7d7

    The constructor can only be called with txids or filter specified, not both. How about making that explicit in the implementation with a comment or an assert (or both)? eg:

    diff --git a/src/merkleblock.cpp b/src/merkleblock.cpp
    index 3f07b4d..7fd2934 100644
    --- a/src/merkleblock.cpp
    +++ b/src/merkleblock.cpp
    @@ -12,6 +12,9 @@
     
     CMerkleBlock::CMerkleBlock(const CBlock& block, CBloomFilter* filter, const std::set<uint256>* txids)
     {
    +    // The constructor can only be called with filter or txids, not both.
    +    assert(!filter or !txids);
    +
         header = block.GetBlockHeader();
     
         std::vector<bool> vMatch;
    
  35. MarcoFalke commented at 11:19 AM on October 3, 2017: member

    utACK 46ce223d15d4111d096f6342eb6f526d2507d7d7

  36. promag commented at 11:58 AM on October 3, 2017: member

    utACK 46ce223.

  37. MarcoFalke added the label Tests on Oct 3, 2017
  38. MarcoFalke merged this on Oct 3, 2017
  39. MarcoFalke closed this on Oct 3, 2017

  40. MarcoFalke referenced this in commit dbc4ae0396 on Oct 3, 2017
  41. jasonbcox referenced this in commit eb8ed1fc66 on Dec 6, 2019
  42. PastaPastaPasta referenced this in commit e1afa91846 on Jan 31, 2020
  43. PastaPastaPasta referenced this in commit 97d43d2ab9 on Apr 6, 2020
  44. PastaPastaPasta referenced this in commit a9ac795173 on Apr 8, 2020
  45. ckti referenced this in commit b008cd81ea on Mar 28, 2021
  46. DrahtBot locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-22 06:15 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me