Skip tons of duplicate GetHash() calls in Block Commit Thread #1486

pull TheBlueMatt wants to merge 47 commits into bitcoin:master from TheBlueMatt:parallelprune changing 26 files +1576 −593
  1. TheBlueMatt commented at 8:53 PM on June 19, 2012: member

    Based on #1233 and #1405 (mostly just because I happened to be on that branch when I started and didnt feel like rebasing)...

    KCacheGrind screenshot of partial block downloads via -loadblock of just the block commit thread (the bottleneck) on tmpfs (note the highlighting of calls to SHA256): Before: http://i.imgur.com/l2dvV.png After: http://i.imgur.com/9xc5S.png (note that the single GetHash() on the block being committed goes from being 2.53% of the total cpu cycles to 5.2% of the total cpu cycles, calls to libcrypto.so.1.0.0 goes from 45.31% of cpu cycles to 14.52% and the total % of the commit block thread goes from 65.69% to 42.65%)

  2. Add a CHub for communication from p2p/wallet to blockstore.
    The goal is for p2p code/wallet to only get information/communicate
    information about the blockchain through CHub, giving Bitcoin a
    much more clearly-defined structure and allowing for the removal
    of a ton of the current global mess.
    877e3af8ba
  3. Add a basic CHubListener that can be extended. c42d0f15a2
  4. Add EmitBlock/CommitBlock functionality to CHub.
    Replacing ProcessBlock with EmitBlock, and creating callbacks for
    CommitBlock.
    dd3593c500
  5. Fix typo e6d39a8873
  6. Add HandleCommitBlock to net.cpp & move stuff from main.cpp to it 159c528cec
  7. Remove uiInterface.NotifyBlocksChanged and replace with CommitBlock
    Also rename NotifyBlocksChanged to NotifyNewBlock and remove from
    the uiInterface signals list.
    
    This removes some functionality, but NofiyBlocksChanged was not
    used anyway, so it shouldn't matter.  That said, if it is ever
    needed, it would be fairly trivial to add a new callback for it
    in CHub.
    94a440ae79
  8. Add EmitAlert/CommitAlert functionality to CHub.
    Replace ProcessAlert with calls to EmitAlert and create callbacks
    for CommitAlert.
    1aad2e99bb
  9. Use CommitAlert in qt/clientmodel.cpp 5365eca78c
  10. Add RegisterRemoveAlert functionality to CHub. 7da0bed1f2
  11. Replace NotifyAlertChanged with RegisterRemoveAlert. 8f6c206561
  12. Convert Orphan Tx storage to CTransactions from CDataStreams.
    There was no reason to use CDataStream as the transaction was
    already being serialized/deserialized several times, with this
    change, transactions coming in over network are deserialized once
    when received, and then only reserialized in the call to
    RelayMessage, which will be called in a callback thread, not
    blocking cs_main.
    c6c30dca1a
  13. Add a cs around mapAlreadyAskedFor. 9dc89066a8
  14. Add basic EmitTransaction/CommitTransaction functionality to CHub. ac33eeca2c
  15. Use EmitTransaction instead of AcceptToMemoryPool in sendrawtx. 3bfb60f64d
  16. Add CWallet support for registering with a CHub. c277c93f23
  17. Use HandleCommitTransactionToMemoryPool instead of SyncWithWallets. 916ffbfe9d
  18. Remove AcceptToMemoryPool and replace with EmitTransaction.
     * This removes not only CTransaction::AcceptToMemoryPool, but
        also CMerkleTx::AcceptToMemoryPool. It also moves
        CWalletTx::AcceptWalletTransaction to wallet.cpp
    
     * This adds a fCheckInputs flag to EmitTransaction, which is
        similar to the fCheckInputs flag to AcceptToMemoryPool,
        however, it has stricter guidlines that it should only be set
        "when transaction is a supporting tx for one of our own."
        Additionally, "fCheckInputs is ignored (and set to true)
        if !IsInitialBlockDownload() && !fClient"
    
        As a part of these guidelines,
        CWalletTx::AcceptWalletTransaction calls EmitTransaction with
        fCheckInputs set to true (the default) on the final
        transaction, whereas it used to call with fCheckInputs set to
        false. This has the important side-effect of allowing wallet-
        generated transactions to end up getting AddOrphanTx'd.
        However, if a supporting transaction to one of our own had
        previously been AddOrphanTx'd, it would immediately be added
        to memory pool as it is "a supporting tx for one of our own"
        and thus is re-added with fCheckInputs=false.
    
        Note that the possibility of a wallet transaction getting
        AddOrphanTx'd is very low, and should only happen if
        a) a transaction's input is a generate and we are missing that
           block (note that no transactions should be generated with a
           generation input if we don't have that block anyway).
        b) We match the !IsInitialBlockDownload() && !fClient check,
           are not caught up to the latest block, and an input is in a
           block we do not yet have (possible after the last
           checkpoint). This situation is temporary and should resolve
           itself once we catch up (though AddOrphanTx'd transactions
           may be permanently orphaned).
    
        Largely, these guidelines are there because there is no reason
        to add a transaction without checking its inputs, as we have
        those inputs available, and checking them as any other
        transaction would provides additional sanity-checks.
    
     * A second EmitTransaction was added with tx of type CMerkleTx.
        This keeps behavior of CMerkleTx::AcceptToMemoryPool the same
        in fClient mode. Note that new behavior was invented for
        CHub::EmitTransaction(CTransaction&...) in fClient mode,
        namely that ClientConnectInputs is only checked if
        fCheckInputs is true. This was chosen to make emitting a
        transaction possible in fClient mode even if its inputs are
        not available, but could be changed if support for that is not
        needed when fClient mode is actually implemented.
    e60084cae3
  19. Add a CBlockStore class to hold the blockstore. 41ff3cf816
  20. Add support for registering a CBlockStore with CHub. 9a268f437b
  21. Move EmitBlock handling from CHub to CBlockStore. 60ba767b3e
  22. Move block checking from CTxDB::LoadBlockIndex->LoadBlockIndex. 9b926c8d34
  23. Add CWallet::HandleCommitBlock and move junk from main.cpp to it.
    Note that this changes the way the GUI shows coinbases of users:
    it now shows them only after the first non-orphan block based on
    the generating block, instead of after the first block based on
    the generating one period.
    9a8cd1de9b
  24. Add ability to post_all() in CSemaphore. 03aaa47487
  25. Move AlreadyHave to CHub::NeedInv c54072bf68
  26. Add a setBlocksSeen and use it for duplicate checks. b8eecbe69d
  27. Add support for calling CNode->Misbehaving from CHub & CBlockStore. bf46b9fb0b
  28. Add support for emitting DoS callbacks from EmitBlock. 296f60d53f
  29. Lock cs_main in ProcessMessage instead of ProcessMessages.
    This removes the locking of cs_main before calling Emit* and should
    allow multiple ProcessMessage calls at once.
    
    It no doubt locks cs_main in places where it isnt neccessary, but
    to avoid complicated and unforseen interactions, cs_main is locked
    for nearly every message in ProcessMessage.
    8d006b6c78
  30. Add a CBlockStore::GetBlockIndex to encapsulate mapBlockIndex.
    It has a fBlocking flag to wait for the block being requested to
    be committed after having been emitted (for use in the
    "force request" block request in the "inv" message handler).
    
    This fixes a potential segfault if -blockbuffersize is overly big.
    235a0277cf
  31. Add support for EmitBlock concurrency. 2514150e66
  32. Use EmitBlock concurrency in ProcessMessage. c3782838fc
  33. Cache last checkpoint. 7522cc8dc1
  34. Move getdata handling out of cs_main in SendMessages.
    This resolves a bug where the block buffer is allowed to deplete
    because the getdata that is required to continue is not sent.
    0a6ab15771
  35. Add -blockbuffersize, default 20. fb20713497
  36. Call PushGetBlocks after last block, instead of just on orphans.
    This keeps the block buffer from running out until the final
    block's FinishEmitBlock is run.
    9c33625524
  37. Use EmitBlock concurrency in LoadExternalBlockFile. 7176054d23
  38. Fix improper use of STL Containers. 1e2aff6a68
  39. Merge branch 'prune' into parallelcheck 1e5c7b01be
  40. Basic genesis block load and hash test. 42dc417d8b
  41. Fix duplicate CheckBlock() calls 82408b4eb5
  42. Skip duplicate block.GetHash() calls. a035b2d3d6
  43. Skip more duplicate hash checks in CBlockIndexs. 3298e89b38
  44. Skip duplicate GetHash in CBlock::ReadFromDisk 9ad2f5788e
  45. Remove costly uint256/CBigNum::ToString() from SetBestChain. 6526933916
  46. Remove duplicate GetHash() in ConnectBlock. dc5f5395b8
  47. Use vMerkleTree as a cache of transactions hashes. 80e3ac4168
  48. Remove duplicate GetHash() in CBlockStore::CallbackCommitBlock. d2a0579676
  49. TheBlueMatt closed this on Jul 5, 2012

  50. suprnurd referenced this in commit 7155043571 on Dec 5, 2017
  51. lateminer referenced this in commit ded3d62588 on Jan 22, 2019
  52. lateminer referenced this in commit 3918ad3eae on Jan 22, 2019
  53. DrahtBot locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-20 00:16 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me