Move orphan processing to ActivateBestChain #8930

pull TheBlueMatt wants to merge 3 commits into bitcoin:master from TheBlueMatt:net_processing_3 changing 2 files +29 −20
  1. TheBlueMatt commented at 9:45 pm on October 15, 2016: member

    Based on #8928 because its the only way I can run tests. This will conflict a bit with #8865, but that’s easy to resolve, and I’d rather get some eyeballs on this sooner rather than later, so am leaving it freestanding. Once #8865 goes in the orphan processing will go into the CValidationInterface which manges “net processing” stuff.

    This further decouples “main” and “net” processing logic by moving orphan processing out of the chain-connecting cs_main lock and into its own cs_main lock, beside all of the other chain callbacks.

    Once further decoupling of net and main processing logic occurs, orphan handing should move to its own lock, out of cs_main.

    Note that this will introduce a race if there are any cases where we assume the orphan map to be consistent with the current chain tip, however I am confident there is no such case (ATMP will fail without DoS score in all such cases).

  2. TheBlueMatt force-pushed on Oct 16, 2016
  3. TheBlueMatt force-pushed on Oct 16, 2016
  4. MarcoFalke added the label Refactoring on Oct 16, 2016
  5. in src/main.cpp: in b4c3c3bd89 outdated
    3077+
    3078         // Notifications/callbacks that can run without cs_main
    3079         if(connman)
    3080             connman->SetBestHeight(nNewHeight);
    3081 
    3082+
    


    sipa commented at 7:29 am on October 31, 2016:
    Useful.
  6. in src/main.cpp: in b4c3c3bd89 outdated
    3047@@ -3068,10 +3048,38 @@ bool ActivateBestChain(CValidationState &state, const CChainParams& chainparams,
    3048         }
    3049         // When we reach this point, we switched to a new tip (stored in pindexNewTip).
    3050 
    3051+        // Remove orphan transactions with cs_main
    3052+        {
    3053+            LOCK(cs_main);
    


    sipa commented at 7:30 am on October 31, 2016:
    I hope we can separate nodestate and other message-processing related variables from cs_main…

    theuni commented at 8:52 pm on October 31, 2016:
    @sipa Absolutely, I’m planning on doing that after the dust settles here, unless @TheBlueMatt already has it queued up. I’m planning on moving lots of what’s currently in CNode to CNodeState, so breaking out the locks is a prerequisite for that.

    TheBlueMatt commented at 10:47 pm on October 31, 2016:
    Dont have it queued up, but, indeed, thats an obvious change after we split main.cpp.
  7. TheBlueMatt commented at 7:22 pm on November 3, 2016: member
    Closing for now. Will rebase on #9075 when its merged.
  8. TheBlueMatt closed this on Nov 3, 2016

  9. Move orphan processing to ActivateBestChain
    This further decouples "main" and "net" processing logic by moving
    orphan processing out of the chain-connecting cs_main lock and
    into its own cs_main lock, beside all of the other chain callbacks.
    
    Once further decoupling of net and main processing logic occurs,
    orphan handing should move to its own lock, out of cs_main.
    
    Note that this will introduce a race if there are any cases where
    we assume the orphan map to be consistent with the current chain
    tip, however I am confident there is no such case (ATMP will fail
    without DoS score in all such cases).
    ec4525ccc1
  10. Erase orphans per-transaction instead of per-block 97e28029c9
  11. TheBlueMatt reopened this on Nov 18, 2016

  12. TheBlueMatt force-pushed on Nov 18, 2016
  13. TheBlueMatt commented at 3:29 am on November 18, 2016: member
    Rebased on #9075…this now forms the last pull before main.cpp is split clean in two :).
  14. rebroad commented at 9:33 am on November 21, 2016: contributor
    What is the rationale behind this refactoring please? What future changes is it enabling? Or what is it fixing?
  15. laanwj commented at 12:45 pm on November 21, 2016: member

    @rebroad

    This further decouples “main” and “net” processing logic by moving orphan processing out of the chain-connecting cs_main lock and into its own cs_main lock, beside all of the other chain callbacks.

    Decoupling unrelated logic to be able to split up main into validation and network handling parts.

  16. sdaftuar commented at 2:26 pm on November 22, 2016: member
    @TheBlueMatt One thing we don’t currently do, but probably should, is try to reaccept orphan transactions to the mempool if their missing inputs appear in a block. After this PR, would you suggest we just do that inside PeerLogicValidation::SyncTransaction(), or would that be problematic?
  17. sipa commented at 10:53 pm on November 22, 2016: member
    utACK. You could get rid of the BOOST_FOREACH along the way, if you wish.
  18. TheBlueMatt commented at 0:28 am on November 23, 2016: member
    @sdaftuar that seems reasonable, yea. Not for this PR, though, indeed.
  19. sdaftuar commented at 0:41 am on November 23, 2016: member
    utACK
  20. in src/main.cpp: in 2dd652afe5 outdated
    4723@@ -4744,6 +4724,30 @@ PeerLogicValidation::PeerLogicValidation(CConnman* connmanIn) : connman(connmanI
    4724     recentRejects.reset(new CRollingBloomFilter(120000, 0.000001));
    4725 }
    4726 
    4727+void PeerLogicValidation::SyncTransaction(const CTransaction& tx, const CBlockIndex* pindex, int nPosInBlock) {
    4728+    LOCK(cs_main);
    


    theuni commented at 7:35 pm on November 23, 2016:
    Shouldn’t this check against CMainSignals::SYNC_TRANSACTION_NOT_IN_BLOCK?

    TheBlueMatt commented at 8:01 pm on November 23, 2016:
    Hmmmmm…good point…for corectness I’ll add that check to this PR, but I think we probably actually want to remove it in a future one since I believe it would allow us to clean up orphan cleanup logic in the ::TX message handling logic.

    theuni commented at 8:11 pm on November 23, 2016:
    Yep, makes sense. But for now, it can at least be a short-circuit for returning without locking cs_main (not sure if there’s any unlocked caller who would benefit, though).

    TheBlueMatt commented at 8:29 pm on November 23, 2016:
    Done

    morcos commented at 4:55 pm on November 28, 2016:
    @TheBlueMatt Your comment confuses me a bit. Wouldn’t this be broken without that check?

    TheBlueMatt commented at 7:04 pm on November 28, 2016:
    @morcos it was a reference to the fact that we might want to remove orphan transactions which conflict with txn which were added to mempool, instead of doing that work in the ::TX message handling. I believe without the check they would step all over each other, but this could be fixed in the future.

    morcos commented at 7:12 pm on November 28, 2016:
    I agree, nice to move all logic to this function. But for now, we don’t actually remove orphan txs which conflict with things accepted to mempool. We could, however, move recursive processing of orphan txs that depended on mempool accepted txs here, I agree. Also there are other places SyncTransaction is called, such as for disconnected block txs, where it would be incorrect to remove orphan txs which conflict with txs from disconnected blocks. In any case, the code is correct now, I just want to be sure we think about all the ways SyncTransaction is called when we change it in the future.
  21. TheBlueMatt force-pushed on Nov 23, 2016
  22. Move orphan-conflict removal from main logic into a callback
    This makes the orphan map a part of net-processing logic instead
    of main logic.
    d2b88f97a1
  23. TheBlueMatt force-pushed on Nov 23, 2016
  24. theuni commented at 10:50 pm on November 23, 2016: member
    utACK d2b88f97a1235057290f9e8ca0bf11437ba919f8
  25. sipa merged this on Nov 24, 2016
  26. sipa closed this on Nov 24, 2016

  27. sipa referenced this in commit 93566e0c37 on Nov 24, 2016
  28. in src/main.cpp: in d2b88f97a1
    4736+        auto itByPrev = mapOrphanTransactionsByPrev.find(tx.vin[j].prevout);
    4737+        if (itByPrev == mapOrphanTransactionsByPrev.end()) continue;
    4738+        for (auto mi = itByPrev->second.begin(); mi != itByPrev->second.end(); ++mi) {
    4739+            const CTransaction& orphanTx = (*mi)->second.tx;
    4740+            const uint256& orphanHash = orphanTx.GetHash();
    4741+            vOrphanErase.push_back(orphanHash);
    


    rebroad commented at 3:13 pm on December 22, 2016:
    Why create vOrphanErase at all? Why not simply run the EraseOrphanTx() here instead?
  29. codablock referenced this in commit 39560f0d32 on Jan 16, 2018
  30. codablock referenced this in commit c29df8a1ac on Jan 16, 2018
  31. codablock referenced this in commit 3a48d2b832 on Jan 17, 2018
  32. andvgal referenced this in commit 8a61e987c7 on Jan 6, 2019
  33. CryptoCentric referenced this in commit 84f43f8090 on Feb 25, 2019
  34. MarcoFalke locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-27 06:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me