assumeutxo: net_processing changes #24008

pull jamesob wants to merge 5 commits into bitcoin:master from jamesob:2022-01-au-netprocessing changing 3 files +184 −44
  1. jamesob commented at 7:08 pm on January 7, 2022: member

    This is part of the assumeutxo project (parent PR: #27596)


    This PR includes the changes necessary to perform network functionality with multiple chainstates in use. Various pieces of net_processing logic have to be modified in order to support block download that is simultaneous across numerous chainstates.

    Changes include

    • Modify FindNextBlocksToDownload() to parameterize the chainstate being worked on.

    • Change GetNodeStateStats to take the max nCommonHeight per peer across all chainstates.

    • Add CNodeState::chainstate_to_last_common_block

      • we need this to allow handling for a single peer to distinguish between separate chainstates we’re simultaneously downloading blocks for
    • Share requests_available across chainstates when finding the next blocks to download (during calls to FindNextBlocksToDownload()).


    This PR shares commit https://github.com/jamesob/bitcoin/commit/17906dd52543fb75d2c45de884799b35ec5721f4 with #24006, and is included here so that the two changes can be reviewed in parallel.

    This PR excludes a small net_processing commit, https://github.com/jamesob/bitcoin/commit/3e6164d96f9a42ecbf34359f6fd1af5413346933, which will be proposed for merge after #24006 since it relies on the introduction of the BackgroundBlockConnected() validationinterface event that the indexing changes introduce.


    Some commits here are best reviewed with --ignore-space-change.

    Unit-testing net_processing is notoriously difficult and with that in mind I haven’t included any unittests here, but in parallel with review of these changes I will attempt to write some tests. Note that this behavior is covered in the functional tests included in the parent PR: https://github.com/jamesob/bitcoin/commit/c4949f2daf05289e76123b3c705277bf735a79d6

  2. DrahtBot added the label P2P on Jan 7, 2022
  3. DrahtBot added the label Validation on Jan 7, 2022
  4. DrahtBot commented at 8:25 pm on January 7, 2022: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK fjahr
    Concept ACK naumenkogs, ariard, mzumsande
    Approach ACK jonatack
    Stale ACK ryanofsky

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #27626 (Parallel compact block downloads, take 3 by instagibbs)
    • #27596 (assumeutxo (2) by jamesob)
    • #26762 (refactor: Make CCheckQueue RAII-styled by hebasto)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  5. in src/validation.h:825 in c4d90167f0 outdated
    821@@ -822,6 +822,10 @@ class ChainstateManager
    822         CBlockIndex** ppindex) EXCLUSIVE_LOCKS_REQUIRED(cs_main);
    823     friend CChainState;
    824 
    825+    // Returns nullptr if no snapshot ahs been loaded.
    


    jonatack commented at 10:38 pm on February 1, 2022:

    17906dd5

    0    // Returns nullptr if no snapshot has been loaded.
    
  6. in src/validation.cpp:5651 in c4d90167f0 outdated
    5028@@ -4984,3 +5029,33 @@ void ChainstateManager::MaybeRebalanceCaches()
    5029         }
    5030     }
    5031 }
    5032+
    5033+CBlockIndex* ChainstateManager::getSnapshotBaseBlock()
    5034+{
    


    jonatack commented at 10:40 pm on February 1, 2022:

    17906dd5 suggestions

    • add AssertLockHeld(::cs_main);
    • use PascalCase for the function name in new code
    • use const with braced initialization
     0diff --git a/src/validation.cpp b/src/validation.cpp
     1index d81ee4fa03..5c9bf34aad 100644
     2--- a/src/validation.cpp
     3+++ b/src/validation.cpp
     4@@ -5091,16 +5091,18 @@ void ChainstateManager::MaybeRebalanceCaches()
     5     }
     6 }
     7 
     8-CBlockIndex* ChainstateManager::getSnapshotBaseBlock()
     9+CBlockIndex* ChainstateManager::GetSnapshotBaseBlock()
    10 {
    11+    AssertLockHeld(::cs_main);
    12-    auto blockhash_op = SnapshotBlockhash();
    13+    const auto blockhash_op{SnapshotBlockhash()};
    14     if (!blockhash_op) return nullptr;
    15     return m_blockman.LookupBlockIndex(*blockhash_op);
    16 }
    17 
    18-std::optional<int> ChainstateManager::getSnapshotHeight()
    19+std::optional<int> ChainstateManager::GetSnapshotHeight()
    20 {
    21+    AssertLockHeld(::cs_main);
    22-    CBlockIndex* base = getSnapshotBaseBlock();
    23+    const CBlockIndex* base{GetSnapshotBaseBlock()};
    24     return base ? std::make_optional(base->nHeight) : std::nullopt;
    25 }
    26 
    27diff --git a/src/validation.h b/src/validation.h
    28index ddeddbcecc..5bc249c27c 100644
    29--- a/src/validation.h
    30+++ b/src/validation.h
    31@@ -842,8 +842,8 @@ private:
    32     friend CChainState;
    33 
    34     // Returns nullptr if no snapshot ahs been loaded.
    35-    CBlockIndex* getSnapshotBaseBlock() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
    36-    std::optional<int> getSnapshotHeight() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
    37+    CBlockIndex* GetSnapshotBaseBlock() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
    38+    std::optional<int> GetSnapshotHeight() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
    39 
    40 public:
    41     std::thread m_load_block;
    

    jamesob commented at 0:29 am on February 12, 2022:

    use PascalCase for the function name in new code @ryanofsky has pioneered using thisCovention for method names (in order to distinguish methods from functions without using this) and so I’ve been using that here.


    ajtowns commented at 5:29 am on April 12, 2022:

    There isn’t a function with the name GetSnapshotHeight or GetSnapshotBaseBlock – why would you need to distinguish the method from a function that doesn’t exist? I really dislike the “let’s randomly introduce new and different code style conventions” approach, especially when combined with “let’s patch the entire codebase to update to the latest style conventions”.

    The developers-notes say “Class names, function names, and method names are UpperCamelCase (PascalCase). Do not prefix class names with C.”


    ryanofsky commented at 6:10 am on April 12, 2022:

    There isn’t a function with the name GetSnapshotHeight or GetSnapshotBaseBlock – why would you need to distinguish the method from a function that doesn’t exist?

    The reason I like calling non-static member variables m_var instead of var is not to distinguish them from other variables called var, but to make it obvious that they are members, and make code easier to read and understand. Similarly the reason I like calling non-static member functions fun() instead of Fun() is not to distinguish from other functions call Fun(), but to make it obvious that they are members, so code is clearer and easier to understand. It is nice to know, when you are calling a function, if that function will have access to *this.

    I just wanted to respond to your question though, not defend or debate anything. This could easily be a garbage coding convention and I am stupid for liking it.


    ajtowns commented at 6:27 am on April 12, 2022:
    It’s a completely fine convention, it’s just not the one this codebase has adopted. And if we keep changing conventions we’ll never get the benefit of any of them.

    jamesob commented at 2:14 pm on April 12, 2022:

    it’s just not the one this codebase has adopted @ajtowns I encourage you to take a look at the entire src/interfaces and src/node subtrees. Pretty much all of the multiprocess code (and increasingly some validation code) has been using this convention for years.


    jonatack commented at 2:36 pm on April 12, 2022:
    If there’s agreement on this style, it would be helpful and save developer time to update the developer notes.

    jamesob commented at 3:36 pm on April 12, 2022:
    @jonatack agree, but previous attempts have been rejected and locked (?): #14635 (comment)

    ajtowns commented at 5:38 pm on April 12, 2022:
    If I look through the existing codebase, and model my new code on that rather than the developer notes, I’ll be using hungarian notation, just as satoshi intended. Suggesting an alternative convention is fine, but if it gets rejected, you don’t just keep on using it anyway.

    fjahr commented at 4:26 pm on April 18, 2022:
    For the sake of progress here it seems like it would be better to change these methods back to the Uppercase for now and keep that discussion seperate in #24846.

    fjahr commented at 4:31 pm on April 18, 2022:
    Did you consider making the interface of the two methods consistent and getSnapshotBaseBlock return an optional as well?

    jamesob commented at 9:51 pm on April 29, 2022:
    Relented on all counts here.
  7. in src/validation.cpp:5572 in c4d90167f0 outdated
    4959+            // Because we expect to have received the headers for the IBD chain
    4960+            // contents before receiving blocks, this means that any block for
    4961+            // which we don't have headers should go in the snapshot chain.
    4962+            (pblock == nullptr || !m_snapshot_chainstate->m_chain.Contains(pblock))) {
    4963+        return *m_snapshot_chainstate.get();
    4964+    }
    


    jonatack commented at 0:00 am on February 2, 2022:

    75912c5 suggestions

    • add missing AssertLockHeld(::cs_main);
    • place comment outside of conditional
    • const with braced init
     0
     1 CChainState& ChainstateManager::GetChainstateForNewBlock(const uint256& blockhash)
     2 {
     3+    AssertLockHeld(::cs_main);
     4-    auto* pblock = m_blockman.LookupBlockIndex(blockhash);
     5+    const auto* pblock{m_blockman.LookupBlockIndex(blockhash)};
     6     if (m_snapshot_chainstate &&
     7-            // If pblock is null, we haven't seen the header for this block.
     8-            // Because we expect to have received the headers for the IBD chain
     9-            // contents before receiving blocks, this means that any block for
    10-            // which we don't have headers should go in the snapshot chain.
    11-            (pblock == nullptr || !m_snapshot_chainstate->m_chain.Contains(pblock))) {
    12+        (pblock == nullptr || !m_snapshot_chainstate->m_chain.Contains(pblock))) {
    13+        // If pblock is null, we haven't seen the header for this block.
    14+        // Because we expect to have received the headers for the IBD chain
    15+        // contents before receiving blocks, this means that any block for
    16+        // which we don't have headers should go in the snapshot chain.
    17         return *m_snapshot_chainstate.get();
    

    jamesob commented at 9:55 pm on April 29, 2022:
    Fixed.
  8. in src/validation.cpp:3610 in c4d90167f0 outdated
    3602@@ -3603,6 +3603,12 @@ bool ChainstateManager::ProcessNewBlock(const CChainParams& chainparams, const s
    3603 {
    3604     AssertLockNotHeld(cs_main);
    3605 
    3606+    CChainState* chainstate;
    3607+    {
    3608+        LOCK(cs_main);
    3609+        chainstate = &this->GetChainstateForNewBlock(block->GetHash());
    3610+    }
    


    jonatack commented at 0:06 am on February 2, 2022:

    75912c552e43e0bb43e42349164d0aa279927013 suggestions

    • ::cs_main preferred over cs_main to clarify it is a global
    • chainstate can use braced init and an inlined WITH_LOCK macro
    0     AssertLockNotHeld(cs_main);
    1-
    2-    CChainState* chainstate;
    3-    {
    4-        LOCK(cs_main);
    5-        chainstate = &this->GetChainstateForNewBlock(block->GetHash());
    6-    }
    7+    CChainState* chainstate{
    8+        WITH_LOCK(::cs_main, return &this->GetChainstateForNewBlock(block->GetHash()))};
    

    jamesob commented at 9:51 pm on April 29, 2022:
    Fixed.
  9. jonatack commented at 11:09 pm on February 2, 2022: contributor
    Checkpoint review feedback for the first two commits.
  10. in src/validation.cpp:3638 in c4d90167f0 outdated
    3635             return error("%s: AcceptBlock FAILED (%s)", __func__, state.ToString());
    3636         }
    3637     }
    3638 
    3639-    NotifyHeaderTip(ActiveChainstate());
    3640+    if (WITH_LOCK(::cs_main, return chainstate == &this->ActiveChainstate())) {
    


    jonatack commented at 8:18 pm on February 7, 2022:

    75912c552e43e0bb43e42349164d0aa279927013 is the lock needed for chainstate? ChainstateManager::ActiveChainstate() already locks cs_main

    0    if (chainstate == &this->ActiveChainstate()) {
    

    jamesob commented at 9:55 pm on April 29, 2022:
    Fixed.
  11. in src/validation.cpp:4986 in c4d90167f0 outdated
    4604@@ -4596,6 +4605,27 @@ std::vector<CChainState*> ChainstateManager::GetAll()
    4605     return out;
    4606 }
    4607 
    4608+std::vector<CChainState*> ChainstateManager::GetAllForBlockDownload()
    4609+{
    


    jonatack commented at 8:26 pm on February 7, 2022:

    8cafc54 thread safety run-time assertion needed here

    0 std::vector<CChainState*> ChainstateManager::GetAllForBlockDownload()
    1 {
    2+    AssertLockHeld(::cs_main);
    3     std::vector<CChainState*> out;
    

    jamesob commented at 9:58 pm on April 29, 2022:
    Fixed.
  12. in src/validation.cpp:4613 in c4d90167f0 outdated
    4604@@ -4596,6 +4605,27 @@ std::vector<CChainState*> ChainstateManager::GetAll()
    4605     return out;
    4606 }
    4607 
    4608+std::vector<CChainState*> ChainstateManager::GetAllForBlockDownload()
    4609+{
    4610+    std::vector<CChainState*> out;
    4611+
    4612+    bool snapshot_in_ibd =
    4613+        m_snapshot_chainstate && m_snapshot_chainstate->IsInitialBlockDownload();
    


    jonatack commented at 8:28 pm on February 7, 2022:

    8cafc54 style nit

    0-    bool snapshot_in_ibd =
    1-        m_snapshot_chainstate && m_snapshot_chainstate->IsInitialBlockDownload();
    2+    const bool snapshot_in_ibd{
    3+        m_snapshot_chainstate && m_snapshot_chainstate->IsInitialBlockDownload()};
    

    jamesob commented at 9:58 pm on April 29, 2022:
    Fixed.
  13. in src/net_processing.cpp:538 in c4d90167f0 outdated
    537@@ -538,7 +538,12 @@ class PeerManagerImpl final : public PeerManager
    538     /** Update pindexLastCommonBlock and add not-in-flight missing successors to vBlocks, until it has
    


    jonatack commented at 9:04 pm on February 7, 2022:

    0ef00576

    0-    /** Update pindexLastCommonBlock and add not-in-flight missing successors to vBlocks, until it has
    1-     *  at most count entries.
    2+    /** Update `chainstate_to_last_common_block` and add not-in-flight missing
    3+     *  successors to `vBlocks`, until it has at most `count` entries.
    
  14. in src/net_processing.cpp:1345 in c4d90167f0 outdated
    1056         // Guessing wrong in either direction is not a problem.
    1057-        state->pindexLastCommonBlock = m_chainman.ActiveChain()[std::min(state->pindexBestKnownBlock->nHeight, m_chainman.ActiveChain().Height())];
    1058+        //
    1059+        // Namespace this by chainstate so that we can simultaneously sync two
    1060+        // separate chainstates at different heights.
    1061+        state->chainstate_to_last_common_block[chainstate] = our_chain[
    


    jonatack commented at 9:07 pm on February 7, 2022:

    0ef00576 not sure, but maybe safer on first use in this function to raise instead of failing silently/UB

    0        state->chainstate_to_last_common_block.at(chainstate) = our_chain[
    

    ajtowns commented at 11:06 pm on June 2, 2022:
    x[k] for a map isn’t undefined behaviour, it creates the entry (setting it to nullptr here), then the = replaces it. insert_or_assign(chainstate, our_chain[blah]) would arguably be better I guess.

    jamesob commented at 8:38 pm on June 7, 2022:
    Agree that this isn’t UB; not going to address unless there is an observable benefit.
  15. in src/net_processing.cpp:5037 in c4d90167f0 outdated
    5047+        // chainstates.
    5048+        int requests_available = MAX_BLOCKS_IN_TRANSIT_PER_PEER - state.nBlocksInFlight;
    5049+
    5050+        // Service the snapshot chainstate first - more important to get to the
    5051+        // network's tip quickly than do the background validation on the
    5052+        // snapshot.
    


    jonatack commented at 9:22 pm on February 7, 2022:

    0ef00576cc9a4aac27310b9fc29c534ad829aad9 suggestions

     0-        // during IBD, but once it hits a tip capacity will trickle into subsequent
     1-        // chainstates.
     2-        int requests_available = MAX_BLOCKS_IN_TRANSIT_PER_PEER - state.nBlocksInFlight;
     3+        // during IBD, but once it hits a tip, capacity will trickle into
     4+        // subsequent chainstates.
     5+        int requests_available{MAX_BLOCKS_IN_TRANSIT_PER_PEER - state.nBlocksInFlight};
     6 
     7-        // Service the snapshot chainstate first - more important to get to the
     8-        // network's tip quickly than do the background validation on the
     9-        // snapshot.
    10+        // GetAllForBlockDownload() services the snapshot chainstate first.
    11+        // It is more important to get to the network's tip quickly than do the
    12+        // background validation on the snapshot.
    
  16. in src/net_processing.cpp:3317 in c4d90167f0 outdated
    3233@@ -3184,7 +3234,7 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    3234         }
    3235 
    3236         LOCK(cs_main);
    3237-        if (m_chainman.ActiveChainstate().IsInitialBlockDownload() && !pfrom.HasPermission(NetPermissionFlags::Download)) {
    3238+        if (m_chainman.IsAnyChainInIBD() && !pfrom.HasPermission(NetPermissionFlags::Download)) {
    


    jonatack commented at 9:27 pm on February 7, 2022:
    c4d90167 see #24178 that changes this logic
  17. in src/net_processing.cpp:2711 in c4d90167f0 outdated
    2706@@ -2660,6 +2707,9 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    2707 
    2708         // Self advertisement & GETADDR logic
    2709         if (!pfrom.IsInboundConn() && SetupAddressRelay(pfrom, *peer)) {
    2710+            bool is_ibd;
    2711+            WITH_LOCK(::cs_main, is_ibd = m_chainman.IsAnyChainInIBD());
    


    jonatack commented at 9:32 pm on February 7, 2022:

    c4d90167 can simplify here

    0-            bool is_ibd;
    1-            WITH_LOCK(::cs_main, is_ibd = m_chainman.IsAnyChainInIBD());
    2+            const bool is_ibd{WITH_LOCK(::cs_main, return m_chainman.IsAnyChainInIBD())};
    
  18. in src/net_processing.cpp:4465 in c4d90167f0 outdated
    4461@@ -4412,9 +4462,10 @@ void PeerManagerImpl::MaybeSendAddr(CNode& node, Peer& peer, std::chrono::micros
    4462     if (!peer.m_addr_relay_enabled) return;
    4463 
    4464     LOCK(peer.m_addr_send_times_mutex);
    4465+    bool is_ibd = WITH_LOCK(::cs_main, return m_chainman.IsAnyChainInIBD());
    


    jonatack commented at 9:34 pm on February 7, 2022:

    c4d90167f030547373715a44fc020d0da8461565

    0    const bool is_ibd{WITH_LOCK(::cs_main, return m_chainman.IsAnyChainInIBD())};
    
  19. in src/validation.cpp:5843 in c4d90167f0 outdated
    5053+    return this->SnapshotBlockhash() && !m_snapshot_validated;
    5054+}
    5055+
    5056+bool ChainstateManager::IsAnyChainInIBD()
    5057+{
    5058+    return
    


    jonatack commented at 9:41 pm on February 7, 2022:

    0ef00576cc9a4aac27310b9fc29c534ad829aad9

     0 CChainState& ChainstateManager::getChainstateForIndexing()
     1 {
     2+    AssertLockHeld(::cs_main);
     3     return getSnapshotBaseBlock() ? *m_ibd_chainstate : *m_active_chainstate;
     4 }
     5 
     6@@ -5117,6 +5119,7 @@ bool ChainstateManager::hasBgChainstateInUse()
     7 
     8 bool ChainstateManager::IsAnyChainInIBD()
     9 {
    10+    AssertLockHeld(::cs_main);
    11     return
    

    Naming style: PascalCase for getChainstateForIndexing() and getSnapshotHeight()?

  20. jonatack commented at 9:42 pm on February 7, 2022: contributor
    Approach ACK 0ef00576cc9a4aac27310b9fc29c534ad829aad9 the logic seems good AFAICS
  21. in src/net_processing.cpp:5029 in 0ef00576cc outdated
    5039-                    LogPrint(BCLog::NET, "Stall started peer=%d\n", staller);
    5040+
    5041+        // The first chainstate in line for processing will likely exhaust this
    5042+        // during IBD, but once it hits a tip capacity will trickle into subsequent
    5043+        // chainstates.
    5044+        int requests_available = MAX_BLOCKS_IN_TRANSIT_PER_PEER - state.nBlocksInFlight;
    


    naumenkogs commented at 10:36 am on March 11, 2022:
    nit: it takes some effort to see it can’t overflow, but perhaps we should make it uint and make according changes?
  22. naumenkogs commented at 10:41 am on March 11, 2022: member
    ACK modulo nits by me and jonatack
  23. in src/validation.h:827 in 17906dd525 outdated
    821@@ -822,6 +822,10 @@ class ChainstateManager
    822         CBlockIndex** ppindex) EXCLUSIVE_LOCKS_REQUIRED(cs_main);
    823     friend CChainState;
    824 
    825+    // Returns nullptr if no snapshot ahs been loaded.
    826+    CBlockIndex* getSnapshotBaseBlock() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
    827+    std::optional<int> getSnapshotHeight() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
    


    ryanofsky commented at 8:09 pm on March 21, 2022:

    In commit “add ChainstateManager.getSnapshot{Height,BaseBlock}()” (17906dd52543fb75d2c45de884799b35ec5721f4)

    Can methods be const?


    jamesob commented at 9:48 pm on April 29, 2022:
    Done.
  24. in src/validation.cpp:5555 in 75912c552e outdated
    4929@@ -4921,6 +4930,21 @@ bool ChainstateManager::PopulateAndValidateSnapshot(
    4930     return true;
    4931 }
    4932 
    4933+CChainState& ChainstateManager::GetChainstateForNewBlock(const uint256& blockhash)
    4934+{
    


    ryanofsky commented at 8:23 pm on March 21, 2022:

    In commit “validation: introduce ChainstateManager::GetChainstateForNewBlock” (75912c552e43e0bb43e42349164d0aa279927013)

    Note: In my own words I’d describe the logic as: Always prefer to add blocks to the newer snapshot chainstate, not the older background chainstate, but add blocks to the background chainstate if they’re already in in the snapshot chainstate.

  25. in src/validation.h:1005 in c4d90167f0 outdated
    1000+    //!   fully validated chain.
    1001+    CChainState& getChainstateForIndexing() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
    1002+
    1003+    //! @returns true if we have more than one chainstate in use. This means that a
    1004+    //! background validation chainstate is running.
    1005+    bool hasBgChainstateInUse();
    


    ryanofsky commented at 8:53 pm on March 21, 2022:

    In commit “p2p: don’t advertise until we finish all IBDs” (c4d90167f030547373715a44fc020d0da8461565)

    Maybe wait to add these until they are actually used and tested


    fjahr commented at 5:06 pm on April 18, 2022:
    +1

    jamesob commented at 10:00 pm on April 29, 2022:
    Good catch, not sure what happened here. Too many assumeutxo commits getting shuffled around.
  26. in src/net_processing.cpp:3317 in c4d90167f0 outdated
    3233@@ -3231,7 +3234,7 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    3234         }
    3235 
    3236         LOCK(cs_main);
    3237-        if (m_chainman.ActiveChainstate().IsInitialBlockDownload() && !pfrom.HasPermission(NetPermissionFlags::Download)) {
    3238+        if (m_chainman.IsAnyChainInIBD() && !pfrom.HasPermission(NetPermissionFlags::Download)) {
    


    ryanofsky commented at 9:05 pm on March 21, 2022:

    In commit “p2p: don’t advertise until we finish all IBDs” (c4d90167f030547373715a44fc020d0da8461565)

    There are a few other ActiveChainstate().IsInitialBlockDownload() calls in this file that aren’t replaced by m_chainman.IsAnyChainInIBD() calls. Will these need to be changed later to avoid changing network behavior when an assumeutxo snapshot is loaded? If not, maybe it is worth going through the calls in this PR and adding comments that could shed more light on why message processing is affected by IBD status, or why it only depends on active IBD status not full IBD status.


    jamesob commented at 9:42 pm on April 29, 2022:
    Good point; I’ve added some commentary to the other uses of IsInitialBlockDownload().
  27. ryanofsky approved
  28. ryanofsky commented at 9:08 pm on March 21, 2022: contributor
    Looks good! Code review ACK c4d90167f030547373715a44fc020d0da8461565
  29. fjahr commented at 5:18 pm on April 18, 2022: contributor

    Approach ACK

    Only did a light code review pass for now because there are already a lot of open comments to be addressed, I will come back when that is done.

  30. DrahtBot added the label Needs rebase on Apr 28, 2022
  31. jamesob commented at 11:53 am on April 28, 2022: member
    Thanks for all the feedback so far; I’ll get around to fixing this up in the next day or so after I get #24232 in order.
  32. jamesob force-pushed on Apr 29, 2022
  33. jamesob commented at 9:58 pm on April 29, 2022: member
    Rebased, thanks for the feedback all.
  34. DrahtBot removed the label Needs rebase on Apr 29, 2022
  35. in src/validation.cpp:5287 in 2c7595a6f7 outdated
    5282+    return base ? std::make_optional(base->nHeight) : std::nullopt;
    5283+}
    5284+
    5285+CChainState& ChainstateManager::getChainstateForIndexing()
    5286+{
    5287+    return getSnapshotBaseBlock() ? *m_ibd_chainstate : *m_active_chainstate;
    


    fjahr commented at 10:55 pm on April 29, 2022:
    Missed an uppercase here it seems.

    jamesob commented at 4:45 pm on May 2, 2022:
    Ah, thanks.
  36. DrahtBot added the label Needs rebase on Apr 30, 2022
  37. jamesob force-pushed on May 2, 2022
  38. jamesob force-pushed on May 2, 2022
  39. DrahtBot removed the label Needs rebase on May 2, 2022
  40. DrahtBot added the label Needs rebase on May 13, 2022
  41. jamesob force-pushed on May 16, 2022
  42. DrahtBot removed the label Needs rebase on May 16, 2022
  43. DrahtBot added the label Needs rebase on May 31, 2022
  44. jamesob force-pushed on Jun 2, 2022
  45. DrahtBot removed the label Needs rebase on Jun 2, 2022
  46. DrahtBot added the label Needs rebase on Jun 2, 2022
  47. in src/validation.cpp:5270 in df44c76cdb outdated
    5267@@ -5225,3 +5268,30 @@ ChainstateManager::~ChainstateManager()
    5268         i.clear();
    5269     }
    5270 }
    5271+
    5272+std::optional<const CBlockIndex> ChainstateManager::GetSnapshotBaseBlock() const
    


    ajtowns commented at 10:44 pm on June 2, 2022:
    This makes a copy of the CBlockIndex rather than returning a pointer to the existing one?

    jamesob commented at 8:40 pm on June 7, 2022:
    Copy here doesn’t seem like a big deal; smallish data structures and this function is (and always will be) called pretty sparingly. Plus, copy means we don’t have to worry about the lifetime of the underlying CBlockIndex reference.

    maflcko commented at 5:59 am on June 8, 2022:

    This seems dangerous. There are many places in the code base where raw pointers are compared to check CBlockIndex equality. Creating a copy will change the raw pointer and make two “identical” CBlockIndex appear different.

    Also copying a CBlockIndex does not extend its lifetime. It has a pprev member, that will be deleted as well.

    If someone has an easy patch to =delete the copy constructor, I’d be looking forward to review.

    (If you really want to refer to a block after the lifetime of the CBlockIndex, for whatever reason, you could return the hash of the block)


    jamesob commented at 2:03 pm on June 8, 2022:
    Good points, I should think harder. Fixed and pushed.

    jamesob commented at 4:11 pm on June 8, 2022:

    If someone has an easy patch to =delete the copy constructor, I’d be looking forward to review.

    Added in #25311.

  48. in src/net_processing.cpp:399 in 56cae81ed7 outdated
    390+    //!
    391+    //! Nota bene: this is contingent on the ChainstateManager not destructing
    392+    //! any CChainState objects which were in use at any point (e.g. a background
    393+    //! validation chainstate which has completed) until the end of
    394+    //! init.cpp:Shutdown(), else we'll have bad pointers here.
    395+    std::map<const CChainState*, const CBlockIndex*> chainstate_to_last_common_block = {};
    


    ajtowns commented at 11:00 pm on June 2, 2022:

    Having a one or two element map for every peer seems a bit icky.

    Is this even necessary though? If you have a common snapshot block, then your common IBD block will be whatever your IBD tip is, no? And if you do have a snapshot block, but don’t have a common snapshot block with this peer, don’t you want to disconnect?

    I guess I’m suggesting replacing this by a CBlockIndex* to trace the last common block, and a bool is_last_common_block_from_snapshot?


    jamesob commented at 8:35 pm on June 7, 2022:

    Having a one or two element map for every peer seems a bit icky.

    How come? Aren’t the number of peers (in the hundreds at the most, by default ~10?) going to make the memory consumption of this data structure negligible?

    I guess I’m suggesting replacing this by a CBlockIndex* to trace the last common block, and a bool is_last_common_block_from_snapshot?

    The consensus has been that we want as few parts of the system as possible knowing about “snapshots” per se. The current approach also has the advantage that if we were ever to further parallelize validation with more than one background validation chainstate, we could keep this structure.

    I think this suggestion also makes end usage of this data more complicated, because we have to articulate the logic you spell out above within usages. I don’t think the memory parsimony justifies the added complexity.


    ajtowns commented at 3:23 am on June 13, 2022:

    The consensus has been that we want as few parts of the system as possible knowing about “snapshots” per se.

    Sure; but the map already means the code needs to know about snapshots per se.

    if we were ever to further parallelize validation with more than one background validation chainstate, we could keep this structure.

    That seems crazy to me; managing two utxo sets is already a lot of space, increasing that is costly, and I don’t think validating the chain at many points would help with any bottlenecks.

    But you shouldn’t even need the bool; the fact that the snapshots are all at different points in the same chain should let you just say: last_common_block_for_this_chainstate = min(tip, last_common_block) when picking where to start, and if (last_common_block->nHeight < pindex->nHeight) last_common_block = pindex when updating.

      0--- a/src/net_processing.cpp
      1+++ b/src/net_processing.cpp
      2@@ -391,7 +391,7 @@ struct CNodeState {
      3     //! any CChainState objects which were in use at any point (e.g. a background
      4     //! validation chainstate which has completed) until the end of
      5     //! init.cpp:Shutdown(), else we'll have bad pointers here.
      6-    std::map<const CChainState*, const CBlockIndex*> chainstate_to_last_common_block = {};
      7+    const CBlockIndex* last_common_block = nullptr;
      8 
      9     //! The best header we have sent our peer.
     10     const CBlockIndex* pindexBestHeaderSent{nullptr};
     11@@ -757,7 +757,7 @@ private:
     12 
     13     bool TipMayBeStale() EXCLUSIVE_LOCKS_REQUIRED(cs_main);
     14 
     15-    /** Update chainstate_to_last_common_block and add not-in-flight missing successors
     16+    /** Update last_common_block and add not-in-flight missing successors
     17      * to vBlocks, until it has at most count entries.
     18      */
     19     void FindNextBlocksToDownload(
     20@@ -1140,6 +1140,13 @@ void PeerManagerImpl::UpdateBlockAvailability(NodeId nodeid, const uint256 &hash
     21     }
     22 }
     23 
     24+static const CBlockIndex* GetLastCommonBlock(const CBlockIndex* tip, const CBlockIndex* last_common_block) EXCLUSIVE_LOCKS_REQUIRED(::cs_main)
     25+{
     26+    if (last_common_block == nullptr || tip == nullptr) return nullptr;
     27+    if (tip->nHeight <= last_common_block->nHeight) return tip;
     28+    return last_common_block;
     29+}
     30+
     31 void PeerManagerImpl::FindNextBlocksToDownload(
     32     const CChainState* const chainstate,
     33     NodeId nodeid,
     34@@ -1167,27 +1174,22 @@ void PeerManagerImpl::FindNextBlocksToDownload(
     35         return;
     36     }
     37 
     38-    if (!state->chainstate_to_last_common_block.count(chainstate)) {
     39+    if (!state->last_common_block) {
     40         // Bootstrap quickly by guessing a parent of our best tip is the forking point.
     41         // Guessing wrong in either direction is not a problem.
     42-        //
     43-        // Namespace this by chainstate so that we can simultaneously sync two
     44-        // separate chainstates at different heights.
     45-        state->chainstate_to_last_common_block[chainstate] = our_chain[
     46-            std::min(state->pindexBestKnownBlock->nHeight, our_chain.Height())];
     47+        state->last_common_block = our_chain[std::min(state->pindexBestKnownBlock->nHeight, our_chain.Height())];
     48     }
     49 
     50     // If the peer reorganized, our previous chainstate_to_last_common_block may not be an ancestor
     51     // of its current tip anymore. Go back enough to fix that.
     52-    state->chainstate_to_last_common_block[chainstate] = LastCommonAncestor(
     53-        state->chainstate_to_last_common_block[chainstate], state->pindexBestKnownBlock);
     54+    state->last_common_block = LastCommonAncestor(state->last_common_block, state->pindexBestKnownBlock);
     55 
     56-    if (state->chainstate_to_last_common_block[chainstate] == state->pindexBestKnownBlock) {
     57+    if (state->last_common_block == state->pindexBestKnownBlock) {
     58         return;
     59     }
     60 
     61     std::vector<const CBlockIndex*> vToFetch;
     62-    const CBlockIndex *pindexWalk = state->chainstate_to_last_common_block[chainstate];
     63+    const CBlockIndex* pindexWalk = GetLastCommonBlock(our_tip, state->last_common_block);
     64     // Never fetch further than the best block we know the peer has, or more than BLOCK_DOWNLOAD_WINDOW + 1 beyond the last
     65     // linked block we have in common with this peer. The +1 is so we can detect stalling, namely if we would be able to
     66     // download that next block if the window were 1 larger.
     67@@ -1209,7 +1211,7 @@ void PeerManagerImpl::FindNextBlocksToDownload(
     68 
     69         // Iterate over those blocks in vToFetch (in forward direction), adding the ones that
     70         // are not yet downloaded and not in flight to vBlocks. In the meantime, update
     71-        // chainstate_to_last_common_block as long as all ancestors are already downloaded, or if it's
     72+        // last_common_block as long as all ancestors are already downloaded, or if it's
     73         // already part of our chain (and therefore don't need it even if pruned).
     74         for (const CBlockIndex* pindex : vToFetch) {
     75             if (!pindex->IsValid(BLOCK_VALID_TREE)) {
     76@@ -1221,8 +1223,11 @@ void PeerManagerImpl::FindNextBlocksToDownload(
     77                 return;
     78             }
     79             if (pindex->nStatus & BLOCK_HAVE_DATA || our_chain.Contains(pindex)) {
     80-                if (pindex->HaveTxsDownloaded())
     81-                    state->chainstate_to_last_common_block[chainstate] = pindex;
     82+                if (pindex->HaveTxsDownloaded()) {
     83+                    if (state->last_common_block == nullptr || pindex->nHeight > state->last_common_block->nHeight) {
     84+                        state->last_common_block = pindex;
     85+                    }
     86+                }
     87             } else if (!IsBlockRequested(pindex->GetBlockHash())) {
     88                 // The block is not already downloaded, and not yet in flight.
     89                 if (pindex->nHeight > nWindowEnd) {
     90@@ -1434,13 +1439,8 @@ bool PeerManagerImpl::GetNodeStateStats(NodeId nodeid, CNodeStateStats& stats) c
     91         stats.nSyncHeight = state->pindexBestKnownBlock ? state->pindexBestKnownBlock->nHeight : -1;
     92         stats.nCommonHeight = -1;
     93 
     94-        // Take the max common height with this peer across chainstates.
     95-        for (const auto& it : state->chainstate_to_last_common_block) {
     96-            const CBlockIndex* block = it.second;
     97-            if (block && block->nHeight > stats.nCommonHeight) {
     98-                stats.nCommonHeight = block->nHeight;
     99-            }
    100-        }
    101+        if (state->last_common_block) stats.nCommonHeight = state->last_common_block->nHeight;
    102+
    103         for (const QueuedBlock& queue : state->vBlocksInFlight) {
    104             if (queue.pindex)
    105                 stats.vHeightInFlight.push_back(queue.pindex->nHeight);
    

    jamesob commented at 2:41 pm on June 23, 2022:

    Sure; but the map already means the code needs to know about snapshots per se.

    This is not true - knowing about multiple chainstates is not knowing about snapshots, per the definition of “per se!”


    Thanks for the example patch. I have looked at this a few times, and I still haven’t been able to convince myself that it does something equivalent. It likely does - I’m maybe just too tired or demotivated to see it.

    As I just mentioned in #24232#pullrequestreview-1007902467, I am growing very weary of work on assumeutxo after three years. Unless there are substantive safety/correctness concerns, I will not be making substantial changes to the approach in this and other PRs.

  49. in src/validation.cpp:5794 in f61f80a043 outdated
    5287@@ -5288,3 +5288,10 @@ std::optional<int> ChainstateManager::GetSnapshotBaseHeight() const
    5288     auto base{this->GetSnapshotBaseBlock()};
    5289     return base ? std::make_optional(base->nHeight) : std::nullopt;
    5290 }
    5291+
    5292+bool ChainstateManager::IsAnyChainInIBD()
    


    ajtowns commented at 11:12 pm on June 2, 2022:
    Seems weird not to take the lock in the function, rather than have every caller invoke it as WITH_LOCK(cs_main, ...)

    jamesob commented at 8:40 pm on June 7, 2022:
    I thought that generally we preferred lock annotations to inline locking for short functions like these to avoid recursive acquisitions.
  50. in src/validation.h:1192 in f61f80a043 outdated
    1037@@ -1038,6 +1038,9 @@ class ChainstateManager
    1038     /** Produce the necessary coinbase commitment for a block (modifies the hash, don't call for mined blocks). */
    1039     std::vector<unsigned char> GenerateCoinbaseCommitment(CBlock& block, const CBlockIndex* pindexPrev) const;
    1040 
    1041+    //! Returns true if any chainstate in use is in initial block download.
    1042+    bool IsAnyChainInIBD() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
    


    ajtowns commented at 11:16 pm on June 2, 2022:
    Not entirely clear to me that this (“Until we have completed IBD on all chainstates, don’t answer getheaders requests from our peers or advertise ourselves.”) is a good idea? If you’ve caught up on the snapshot chain, is the fact that you got there via assumeutxo any worse than getting there via assumevalid? No objection to keeping it purely for “first, do no harm” reasons, but if we’re already confident there’s no harm possible, better to have more nodes sharing headers etc, than less.

    jamesob commented at 9:38 pm on June 7, 2022:
    Good observation. I definitely agree that we should still be answering getheaders requests, which I think we are per this code: https://github.com/jamesob/bitcoin/blob/39ddd522c37f760c8054d2fa58e026c69a50ce8b/src/net_processing.cpp#L3292-L3321. In terms of advertising ourselves while background-validating, I’ll remain conservative in this changeset and defer to others… should be a pretty easy follow-up if deemed safe, and it’d allow someone besides me to make assumeutxo-related changes :).
  51. jamesob force-pushed on Jun 7, 2022
  52. jamesob commented at 9:40 pm on June 7, 2022: member
    Thanks for the review @ajtowns.
  53. DrahtBot removed the label Needs rebase on Jun 7, 2022
  54. jamesob force-pushed on Jun 8, 2022
  55. DrahtBot added the label Needs rebase on Jul 4, 2022
  56. jamesob force-pushed on Jul 18, 2022
  57. DrahtBot removed the label Needs rebase on Jul 18, 2022
  58. DrahtBot added the label Needs rebase on Jul 19, 2022
  59. in src/validation.h:928 in 84443da54f outdated
    917+    //!
    918+    //! Because the use of UTXO snapshots requires the simultaneous maintenance
    919+    //! of two chainstates, when a new block message arrives we have to decide
    920+    //! which chain we should attempt to append it to.
    921+    //!
    922+    //! If our most-work chain hasn't seen the incoming blockhash, return that.
    


    ariard commented at 0:14 am on August 23, 2022:

    Note, I think there is a discrepancy between the comment and the actual GetChainStateForNewBlock() logic.

    From my understanding, the comment is saying the decision criteria to append a new block should be the PoW accumulated. However, the logic implemented probe the presence of the block in the snapshot chainstate, then if it’s already included select the IBD chainstate as the relevant chainstate ?

    That said, I think this approach is sound in case of a peer deliberately sending non-requested blocks to stale the assumeutxo-syncing, where the snapshot chainstate would be prevented to reach the tip quickly.


    jamesob commented at 5:48 pm on November 8, 2022:
    Yeah there’s nothing to do with PoW here per se - when I say “most-work” that’s just shorthand for “active chain.” I’ll update the comment to be more specific.
  60. in src/validation.cpp:4961 in 38ac3b3bc3 outdated
    4697+{
    4698+    AssertLockHeld(::cs_main);
    4699+    std::vector<CChainState*> out;
    4700+
    4701+    bool snapshot_in_ibd{
    4702+        m_snapshot_chainstate && m_snapshot_chainstate->IsInitialBlockDownload()};
    


    ariard commented at 0:58 am on August 23, 2022:

    I wonder if it’s too early if snapshot chainstate is deprioritized after we get out of IBD. The current definition of network tip is DEFAULT_MAX_TIP_AGE, that’s still 24h of blocks. If we start to download blocks in parallel for both chainstates, the slow down might diminish the “quick deployment effect” for a assumeutxo user.

    I think this concern could only be qualified once the assumeutxo feature is completed and we observe such perf bottleneck. If it’s the case, we could introduce a tighter notion of chain tip, such as 2h of blocks. So this is just a note on potential assumeutxo refinements.


    jamesob commented at 6:14 pm on November 8, 2022:
    Even after IBD, the snapshot chainstate will still take priority for block download due to the ordering here.
  61. in src/net_processing.cpp:5871 in f33a023bc9 outdated
    5294-                    State(staller)->m_stalling_since = current_time;
    5295-                    LogPrint(BCLog::NET, "Stall started peer=%d\n", staller);
    5296+
    5297+        // The first chainstate in line for processing will likely exhaust this
    5298+        // during IBD, but once it hits a tip capacity will trickle into subsequent
    5299+        // chainstates.
    


    ariard commented at 1:41 am on August 23, 2022:
    As we have two chainstates requesting blocks in parallel, at least as long as snapshot has not reached network’s tip, there is a possibility that the bandwidth consumption increase exhaust the monthly ISP data usage caps of the assumeutxo hosts. In some world regions, the Internet access is capped to 5 - 30 GB/month. That’s okay, I think we have few options to do resources control like maxreceivebuffer, however I wonder if there are other interactions where we’re making assumptions on bandwidth / download throughput that would be altered by that change.

    jamesob commented at 5:55 pm on November 8, 2022:
    Huh? We are still sharing the same number of requests_available - we’re just splitting them across two chainstates. It should be identical from the standpoint of bandwidth usage.
  62. in src/net_processing.cpp:2035 in 4d581ddf2c outdated
    1848@@ -1849,6 +1849,10 @@ void PeerManagerImpl::BlockChecked(const CBlock& block, const BlockValidationSta
    1849     // 3. This is currently the best block we're aware of. We haven't updated
    1850     //    the tip yet so we have no way to check this directly here. Instead we
    1851     //    just check that there are currently no other blocks in flight.
    1852+    //
    1853+    // No need to consider the background validation chainstate's IBD status
    1854+    // (if applicable) because any active chainstate not in IBD will allow
    1855+    // us to judge block validity.
    


    ariard commented at 1:43 am on August 23, 2022:
    I think doc/design/assumeutxo.md could be updated with the description of the changes implemented here. Beyond, such documentation would constitute a good reference to review the changes themselves and assert their correctness.
  63. ariard commented at 1:46 am on August 23, 2022: member

    Concept ACK

    Left few high-level comments, I still wonder about the approach itself, w.r.t to all the assumptions (some loosely documented) underpinning block-download that could be altered by those changes. There is also the question if multiple chainstates block download is robust against buggy or deliberately DoSy peers.

  64. in src/net_processing.cpp:399 in 4d581ddf2c outdated
    396+    //!
    397+    //! Nota bene: this is contingent on the ChainstateManager not destructing
    398+    //! any CChainState objects which were in use at any point (e.g. a background
    399+    //! validation chainstate which has completed) until the end of
    400+    //! init.cpp:Shutdown(), else we'll have bad pointers here.
    401+    std::map<const CChainState*, const CBlockIndex*> chainstate_to_last_common_block = {};
    


    dergoegge commented at 4:23 pm on October 7, 2022:

    I think we can do this without adding Chainstate pointers to CNodeState which i would really prefer.

    What I would suggest is that we nuke pindexLastCommonBlock entirely and compute it on demand instead (the performance overhead should be minimal). Then we don’t need to keep track of it per chainstate but rather only compute it on demand for a given chain. I have a branch that should illustrate what i mean: https://github.com/dergoegge/bitcoin/commits/2022-10-assumeutxo-netproc (last commit puts the loop over the chains inside of FindNextBlocksToDownload, not sure if that works out)


    jamesob commented at 5:53 pm on November 8, 2022:

    [rehashing a discussion from CoreDev]

    This is a good idea, but should be considered separately as it will require benchmarking and more thorough analysis to ensure that it e.g. doesn’t open some kind of DoS vector. I’m not familiar with why these values were cached in the first place, and so “un-caching” them is a design choice I’d rather not conflate here with assumeutxo.

  65. jamesob force-pushed on Oct 12, 2022
  66. jamesob force-pushed on Oct 12, 2022
  67. in src/validation.h:937 in 57de47bca9 outdated
    830@@ -831,6 +831,10 @@ class ChainstateManager
    831         CBlockIndex** ppindex) EXCLUSIVE_LOCKS_REQUIRED(cs_main);
    832     friend CChainState;
    833 
    834+    // Returns nullptr if no snapshot has been loaded.
    835+    const CBlockIndex* GetSnapshotBaseBlock() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
    836+    std::optional<int> GetSnapshotBaseHeight() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
    


    mzumsande commented at 10:30 pm on October 26, 2022:
    GetSnapshotBaseHeight() is never used in this PR.

    Sjors commented at 6:58 pm on February 23, 2023:
    Neither are used here, because GetSnapshotBaseBlock is only used by GetSnapshotBaseHeight. But they are used in the final PR, so I’m fine with introducing them here.
  68. in src/net_processing.cpp:2787 in 4d581ddf2c outdated
    2428@@ -2425,6 +2429,10 @@ void PeerManagerImpl::UpdatePeerStateForReceivedHeaders(CNode& pfrom,
    2429 
    2430     // If we're in IBD, we want outbound peers that will serve us a useful
    2431     // chain. Disconnect peers that are on chains with insufficient work.
    2432+    //
    2433+    // Can ignore background validation chainstate IBD status here (if applicable);
    2434+    // as long as the active chainstate is out of IBD, we are able to assess
    2435+    // work in the headers chain.
    


    mzumsande commented at 11:04 pm on October 26, 2022:
    Why would we need a chainstate to be out of IBD to assess work in a headers chain? In my understanding, the reason for this check is that if the peer is on a chain with insufficient work, we disconnect them here because they wouldn’t be able to help us sync. This could also apply to the background validation chain, so I guess we’d have to decide whether syncing that chain is a high enough priority that we would kick a peer for it.
  69. jamesob force-pushed on Nov 8, 2022
  70. jamesob commented at 6:21 pm on November 8, 2022: member
    Rebased here and I think all feedback has been addressed.
  71. DrahtBot removed the label Needs rebase on Nov 8, 2022
  72. in src/validation.cpp:4997 in 42f5b6b03b outdated
    4840+    // priority is getting the active chain to network tip.
    4841+    if (!IsSnapshotValidated() && !snapshot_in_ibd && m_ibd_chainstate) {
    4842+        out.push_back(m_ibd_chainstate.get());
    4843+    }
    4844+
    4845+    assert(out.size() > 0);
    


    naumenkogs commented at 8:00 am on November 9, 2022:

    I’m likely misunderstanding something. Imagine a snapshot doesn’t exist. Then, if m_ibd_chainstate == false, out will be empty, and this would trigger an issue?

    So, if the caller must not call it in this case, this requirement probably should be documented?


    mzumsande commented at 10:15 pm on November 9, 2022:
    I’m new to AssumeUTXO and had a similar question - I came to the conclustion that m_ibd_chainstate always exists. If we are in a situation where the background chainstate catches up with the snapshot, after the next restart there will be only one chain, which is the old snapshot chainstate, but it will now be “rebranded” as m_ibd_chainstate (#25740). If we never used AssumeUTXO, the usual chainstate will be m_ibd_chainstate.

    ryanofsky commented at 1:54 am on November 14, 2022:

    In commit “validation: add ChainstateManager::GetAllForBlockDownload()” (42f5b6b03bba1ef1c1bc6c225606d119c4e27169)

    Yeah I basically repeat this same comment in all my assumeutxo reviews, but I still find the terms “IBD chainstate” and “snapshot chainstate” confusing and think it would be nice to abolish them later (https://github.com/bitcoin/bitcoin/pull/24232#discussion_r835355848).

    Usually when you try to write code that is more general than it needs to be, it makes things more complicated. But sometimes generalizing code makes it simpler. I think all the code using chainstatemanager would be simpler if chainstatemanager just kept a simple vector of chainstates and pointer to the active chainstate, so assumeutxo logic and terminology would be only need to appear in code loading and validating the snapshot, and the rest of validation and net processing code would only have to care about active and inactive chainstates.

  73. mzumsande commented at 11:00 pm on November 9, 2022: contributor

    Concept ACK

    The changes look good to me - I wonder if there is a way to make sure that there aren’t more things in net_processing that would need to be adapted. I guess we’d have to wait for functional tests with that?

    Also, as far as I understand it, AssumeUTXO will mess up the order of the blocks stored on disk - the first blockfiles would have the blocks between the AssumeUTXO block and the tip, and the subsequent blockfiles would have the blocks between Genesis and the AssumeUTXO block. I wonder if that could break or slow down something, have you tried a reindex for example?

  74. in src/validation.cpp:5374 in 3219f1d682 outdated
    5306+const CBlockIndex* ChainstateManager::GetSnapshotBaseBlock() const
    5307+{
    5308+    AssertLockHeld(::cs_main);
    5309+    const auto blockhash_op{this->SnapshotBlockhash()};
    5310+    if (!blockhash_op) return nullptr;
    5311+    return m_blockman.LookupBlockIndex(*blockhash_op);
    


    ryanofsky commented at 1:28 am on November 14, 2022:

    In commit “add ChainstateManager.GetSnapshot{BaseHeight,BaseBlock}()” (3219f1d682378003296b9e8792ae73557c518a0d)

    Maybe it could return Assert(LookupBlockIndex()) to assert that the hash is valid if there is a snapshot.


    Sjors commented at 7:15 pm on February 23, 2023:
    That seems like a good idea, also because GetSnapshotBaseHeight will deference the result to get a height. (Though it won’t crash because it checks if the result is a nullptr.)
  75. in src/net_processing.cpp:1339 in c5bfecff4d outdated
    1323         // This peer has nothing interesting.
    1324         return;
    1325     }
    1326 
    1327-    if (state->pindexLastCommonBlock == nullptr) {
    1328+    if (!state->chainstate_to_last_common_block.count(chainstate)) {
    


    ryanofsky commented at 2:07 am on November 14, 2022:

    In commit “net_processing: work with multiple chainstates” (c5bfecff4d0db726fab893a46cc1dab29591f325)

    There’s a lot of map lookups in this function, and it seems like you could eliminate them all by declaring a local reference

    0const CBlockIndex*& last_common_block =  state->chainstate_to_last_common_block[chainstate];
    

    Note the pointer is const but the reference is non-const, so you could assign to it and make fewer changes to code below, just replacing state->pindexLastCommonBlock with last_common_block and m_chainman.ActiveChain() with our_chain.


    jamesob commented at 3:38 pm on April 19, 2023:
    Great catch, done.
  76. in src/validation.cpp:4968 in 42f5b6b03b outdated
    4835+    if (m_snapshot_chainstate) {
    4836+        out.push_back(m_snapshot_chainstate.get());
    4837+    }
    4838+    // Exclude the ibd chainstate (which is in the background if we have a snapshot
    4839+    // chainstate active) if the snapshot chainstate is in IBD because the main
    4840+    // priority is getting the active chain to network tip.
    


    ryanofsky commented at 2:26 am on November 14, 2022:

    In commit “validation: add ChainstateManager::GetAllForBlockDownload()” (42f5b6b03bba1ef1c1bc6c225606d119c4e27169)

    Is there a reason to exclude the ibd chainstate if the comment in commit “net_processing: work with multiple chainstates” (c5bfecff4d0db726fab893a46cc1dab29591f325) is correct: “The first chainstate in line for processing will likely exhaust this during IBD, but once it hits a tip capacity will trickle into subsequent chainstates.”?

    It seems cleaner if code in net_processing just decide what blocks to download and validation code doesn’t have to be involved in the details.

    EDIT: To be more specific, I’m suggesting dropping this GetAllForBlockDownload method and just having net_processing code call GetAll instead. GetAll method could be changed to return snapshot chainstate before IBD chainstate so it has higher priority to download blocks


    jamesob commented at 3:29 pm on April 19, 2023:
    Yep, that’s a good point. Will do, thanks!
  77. ryanofsky approved
  78. ryanofsky commented at 2:38 am on November 14, 2022: contributor
    Light code review ACK 645418e8b4d76a0c640590024452b4342802286e. All the changes in this PR are very straightforward and clean thanks to the nice work in preceding PRs. I just want to get more familiar with surrounding net_processing code before adding full ACK.
  79. in src/validation.cpp:4057 in 650677dd66 outdated
    3919+        NotifyHeaderTip(*chainstate);
    3920+    }
    3921 
    3922     BlockValidationState state; // Only used to report errors, not invalidity - ignore it
    3923-    if (!ActiveChainstate().ActivateBestChain(state, block)) {
    3924+    if (!chainstate->ActivateBestChain(state, block)) {
    


    ryanofsky commented at 7:50 pm on November 14, 2022:

    In commit “validation: introduce ChainstateManager::GetChainstateForNewBlock” (650677dd66cede06e242975f51c0e5e77b99079f)

    If I revert this line, all tests seem to still pass. I understand why we can’t have functional tests for this code before there is a snapshot loading RPC but it seems like it shouldn’t be that hard to come up with a unit test that would exercise this logic. I don’t think it’s good to be in a state where validation code doesn’t have test coverage, even if it’s a temporary state.

  80. in src/validation.cpp:4963 in 42f5b6b03b outdated
    4830+    std::vector<Chainstate*> out;
    4831+
    4832+    bool snapshot_in_ibd{
    4833+        m_snapshot_chainstate && m_snapshot_chainstate->IsInitialBlockDownload()};
    4834+
    4835+    if (m_snapshot_chainstate) {
    


    ryanofsky commented at 7:53 pm on November 14, 2022:

    In commit “validation: add ChainstateManager::GetAllForBlockDownload()” (42f5b6b03bba1ef1c1bc6c225606d119c4e27169)

    After #24008 will this need to change to m_snapshot_chainstate && IsUsable(m_snapshot_chainstate)?


    jamesob commented at 3:30 pm on April 19, 2023:
    Done.
  81. in src/net_processing.cpp:5876 in 645418e8b4 outdated
    5869+                static_cast<unsigned int>(MAX_BLOCKS_IN_TRANSIT_PER_PEER - state.nBlocksInFlight)};
    5870+
    5871+        // GetAllForBlockDownload() ensures we get blocks for the snapshot
    5872+        // chainstate first. It is more important to get to the network's tip
    5873+        // quickly than do the background validation on the snapshot.
    5874+        for (Chainstate* chainstate : m_chainman.GetAllForBlockDownload()) {
    


    aureleoules commented at 11:44 am on November 15, 2022:

    c5bfecff4d0db726fab893a46cc1dab29591f325

    0        for (const auto* chainstate : m_chainman.GetAllForBlockDownload()) {
    

    jamesob commented at 3:41 pm on April 19, 2023:
    Fixed.
  82. in src/validation.cpp:4955 in 645418e8b4 outdated
    4823@@ -4819,6 +4824,28 @@ std::vector<Chainstate*> ChainstateManager::GetAll()
    4824     return out;
    4825 }
    4826 
    4827+std::vector<Chainstate*> ChainstateManager::GetAllForBlockDownload()
    


    aureleoules commented at 11:45 am on November 15, 2022:

    42f5b6b03bba1ef1c1bc6c225606d119c4e27169

    0std::vector<Chainstate*> ChainstateManager::GetAllForBlockDownload() const
    

    jamesob commented at 3:30 pm on April 19, 2023:
    Done.
  83. in src/validation.cpp:5794 in 645418e8b4 outdated
    5359+    AssertLockHeld(::cs_main);
    5360+    auto base{this->GetSnapshotBaseBlock()};
    5361+    return base ? std::make_optional(base->nHeight) : std::nullopt;
    5362+}
    5363+
    5364+bool ChainstateManager::IsAnyChainInIBD()
    


    aureleoules commented at 11:46 am on November 15, 2022:

    03abde6150e9950e18edc00cb0541a33f714a231

    0bool ChainstateManager::IsAnyChainInIBD() const
    

    jamesob commented at 4:00 pm on April 19, 2023:
    Fixed.
  84. DrahtBot added the label Needs rebase on Dec 5, 2022
  85. jamesob force-pushed on Dec 7, 2022
  86. DrahtBot removed the label Needs rebase on Dec 7, 2022
  87. DrahtBot added the label Needs rebase on Dec 12, 2022
  88. jamesob force-pushed on Dec 14, 2022
  89. DrahtBot removed the label Needs rebase on Dec 14, 2022
  90. fanquake referenced this in commit 65f5cfda65 on Dec 19, 2022
  91. in src/validation.h:1055 in bf99340ede outdated
    942+    //!
    943+    //! Because the use of UTXO snapshots requires the simultaneous maintenance
    944+    //! of two chainstates, when a new block message arrives we have to decide
    945+    //! which chain we should attempt to append it to.
    946+    //!
    947+    //! If our active chainstate hasn't seen the incoming blockhash, return that.
    


    Sjors commented at 7:32 pm on February 23, 2023:

    I find the explanation in the function body more clear. Maybe say:

    0// Because we expect to have received the headers for the IBD chain
    1// contents before receiving blocks, when the (shared) block manager
    2// doesn't include a header for this block, it must go to
    3// to the snapshot chain.
    

    jamesob commented at 3:58 pm on April 19, 2023:
    Reworded.
  92. in src/validation.cpp:5556 in bf99340ede outdated
    5216+    // If pblock is null, we haven't seen the header for this block.
    5217+    // Because we expect to have received the headers for the IBD chain
    5218+    // contents before receiving blocks, this means that any block for
    5219+    // which we don't have headers should go in the snapshot chain.
    5220+    if (m_snapshot_chainstate &&
    5221+            (pblock == nullptr || !m_snapshot_chainstate->m_chain.Contains(pblock))) {
    


    Sjors commented at 7:42 pm on February 23, 2023:
    bf99340edefc83913116f2ef972988e016deba3d IIUC: maybe clarify here that m_chain.Contains searches both the snapshot headers and the IBD headers it’s built on top of. When a pblock exists but is not contained in the chainstate it’s a fork, and it’s up to the snapshot chainstate to explore those.

    jamesob commented at 3:58 pm on April 19, 2023:
    Good point, done.
  93. Sjors commented at 7:47 pm on February 23, 2023: member

    Reviewed the first two commits.

    They brought up a question in me: what happens if during snapshot validation we find a fork with more PoW that does not contain the assumed UTXO block? Do we finish the background validation and then reorg away, do we abort it, or is this unhandled?

    The second commit is also not really net_processing, but the PR already has a Validation tag, so hopefully that’s clear to everyone.

  94. DrahtBot added the label Needs rebase on Mar 8, 2023
  95. jamesob force-pushed on Mar 8, 2023
  96. DrahtBot removed the label Needs rebase on Mar 8, 2023
  97. jamesob commented at 4:15 pm on March 9, 2023: member

    Also, as far as I understand it, AssumeUTXO will mess up the order of the blocks stored on disk - the first blockfiles would have the blocks between the AssumeUTXO block and the tip, and the subsequent blockfiles would have the blocks between Genesis and the AssumeUTXO block. I wonder if that could break or slow down something, have you tried a reindex for example?

    This is certainly worth addressing. My inclination is to say that given there are only 6925 separate block files on my mainnet node at time of writing, I wouldn’t expect a significant slowdown to intermingle block storage locality. But that said this should certainly be benchmarked. Over the next week I’ll do an IBD with assumeutxo, and then compare reindexing it to reindexing a “traditionally” IBD’d datadir.

  98. in src/net_processing.cpp:906 in 07ab98ecc6 outdated
    904+    /** Update chainstate_to_last_common_block and add not-in-flight missing successors
    905+     * to vBlocks, until it has at most count entries.
    906      */
    907-    void FindNextBlocksToDownload(const Peer& peer, unsigned int count, std::vector<const CBlockIndex*>& vBlocks, NodeId& nodeStaller) EXCLUSIVE_LOCKS_REQUIRED(cs_main);
    908+    void FindNextBlocksToDownload(
    909+        const Chainstate* const chainstate,
    


    maflcko commented at 12:02 pm on March 10, 2023:
    nit (feel free to ignore): What about passing a const& ref here? Otherwise someone could pass in a nullptr and run into UB at runtime?

    jamesob commented at 3:39 pm on April 19, 2023:
    Yup, good idea. Done.
  99. in src/validation.cpp:5528 in 07ab98ecc6 outdated
    5523+    if (m_snapshot_chainstate &&
    5524+            (pblock == nullptr || !m_snapshot_chainstate->m_chain.Contains(pblock))) {
    5525+        return *m_snapshot_chainstate.get();
    5526+    }
    5527+    assert(m_ibd_chainstate);
    5528+    return *m_ibd_chainstate.get();
    


    maflcko commented at 12:08 pm on March 10, 2023:

    nit: No need to call get if you call * later on. Also, can inline the assert:

    0
    1    return *Assert(m_ibd_chainstate);
    

    jamesob commented at 3:58 pm on April 19, 2023:
    Thanks, done.
  100. maflcko commented at 12:10 pm on March 10, 2023: member
    (two nits, feel free to ignore)
  101. in src/validation.cpp:5562 in 8e48137362 outdated
    5489@@ -5485,6 +5490,22 @@ SnapshotCompletionResult ChainstateManager::MaybeCompleteSnapshotValidation(
    5490     return SnapshotCompletionResult::SUCCESS;
    5491 }
    5492 
    5493+Chainstate& ChainstateManager::GetChainstateForNewBlock(const uint256& blockhash)
    5494+{
    5495+    AssertLockHeld(::cs_main);
    5496+    const auto* pblock{m_blockman.LookupBlockIndex(blockhash)};
    


    fjahr commented at 5:15 pm on March 12, 2023:
    This function could have an early return of m_ibd_chainstate if there is no m_snapshot_chainstate which should be the base case, i.e. when assumeutxo is not used?

    jamesob commented at 3:57 pm on April 19, 2023:
    Good point, added.
  102. in src/net_processing.cpp:423 in ed8f55b87c outdated
    420+    //!
    421+    //! Nota bene: this is contingent on the ChainstateManager not destructing
    422+    //! any Chainstate objects which were in use at any point (e.g. a background
    423+    //! validation chainstate which has completed) until the end of
    424+    //! init.cpp:Shutdown(), else we'll have bad pointers here.
    425+    std::map<const Chainstate*, const CBlockIndex*> chainstate_to_last_common_block = {};
    


    fjahr commented at 5:32 pm on March 12, 2023:
    nit: m_chainstate_to_last_common_block

    jamesob commented at 3:39 pm on April 19, 2023:
    Done.
  103. fjahr commented at 5:52 pm on March 12, 2023: contributor

    Light code review ACK 07ab98ecc64a39b377e20dcdd571a750ab62fd97

    Will do another pass after the open comments are addressed.

  104. DrahtBot requested review from ryanofsky on Mar 12, 2023
  105. jonatack commented at 8:03 pm on April 14, 2023: contributor

    Approach ACK 0ef0057 the logic seems good AFAICS

    Will re-review after my last review 14 months age.

    Finishing the 2 remaining PRs in the parent #15606 after v25 branch-off, to target early in the v26 release cycle might be a reasonable goal?

  106. jamesob force-pushed on Apr 19, 2023
  107. jamesob commented at 4:05 pm on April 19, 2023: member
    Thanks to all the dedicated reviewers and my apologies for the delay. Lot of good feedback addressed.
  108. DrahtBot added the label CI failed on Apr 19, 2023
  109. jamesob force-pushed on Apr 19, 2023
  110. DrahtBot removed the label CI failed on Apr 19, 2023
  111. instagibbs commented at 8:14 pm on May 4, 2023: member
    @sdaftuar did you have Current Thoughts about this approach? Trying to bridge IRC discussions to the blocking PR
  112. in src/validation.cpp:5539 in 426ebe830b outdated
    5534+    // contents before receiving blocks, this means that any block for
    5535+    // which we don't have headers should go in the snapshot chain.
    5536+    //
    5537+    // Note that searching the snapshot chain (as below) implicitly searches the ibd
    5538+    // chain, since the former includes the latter.
    5539+    if (m_snapshot_chainstate &&
    


    Sjors commented at 10:33 am on May 9, 2023:
    426ebe830b3bb076182505b60233e8c8f6d35069: the m_snapshot_chainstate check is now redundant: https://github.com/bitcoin/bitcoin/pull/24008/commits/426ebe830b3bb076182505b60233e8c8f6d35069#r1171549876

    jamesob commented at 7:19 pm on May 9, 2023:
    Fixed.
  113. Sjors commented at 11:30 am on May 9, 2023: member

    The first commit tends to confuses me, so I’ll just note a clarification here. I’m not fully convinced that blocks near the tip can’t end up getting processed by the background chainstate. And if they do, I’m not sure if that’s bad.

    When a block comes in near the tip, we know the header so LookupBlockIndex will return a pblock. This header is not contained in m_chain, because headers are only added to m_chain by ConnectTip(). So a potential tip block is sent to the snapshot chainstate.

    m_chain grows sequentially as the tip moves from the snapshot height onwards. Therefore is we only receive each block once, you would expect !m_snapshot_chainstate->m_chain.Contains(pblock) to be true for candidate tip blocks, and false for blocks from before the snapshot.

    But blocks can be received multiple times (and out of order, but that doesn’t seem to be a problem). Each time a block is received it triggers ProcessNewBlock(). Can this never be called with a block in m_chain that is between the snapshot and the tip? What if we receive a block that we saw before and already added to m_chain?

    net_processing calls chainman.ProcessNewBlock from ProcessBlock(), which is called two different sites: when receiving a compact block, a BLOCKTXN message and when receiving a BLOCK message. I don’t think these call sites check that we already processed this block.

  114. in src/net_processing.cpp:2785 in c7cdc7ca83 outdated
    2780@@ -2777,7 +2781,7 @@ void PeerManagerImpl::UpdatePeerStateForReceivedHeaders(CNode& pfrom, Peer& peer
    2781 
    2782     // If we're in IBD, we want outbound peers that will serve us a useful
    2783     // chain. Disconnect peers that are on chains with insufficient work.
    2784-    if (m_chainman.ActiveChainstate().IsInitialBlockDownload() && !may_have_more_headers) {
    2785+    if (m_chainman.IsAnyChainInIBD() && !may_have_more_headers) {
    


    Sjors commented at 12:29 pm on May 9, 2023:

    c7cdc7ca833ce1b5ce9a696fb4ef633c55528023: this line does more than “add commentary”. Maybe move this to 3a7a714721a593e271ce70d4612c995968dbb0da or its own commit directly after that one?

    The code comment could clarify why we care about this even for the background sync: “with insufficient work” -> “with less than the minimum chain work.”


    jamesob commented at 7:15 pm on May 9, 2023:
    Fixed, thanks. Though I haven’t updated the comment; having trouble figuring out what to say there.
  115. Sjors commented at 12:51 pm on May 9, 2023: member
    c7cdc7ca833ce1b5ce9a696fb4ef633c55528023 is missing a comment for the use of ActiveChainstate in ProcessGetBlockData and after receiving a GETBLOCKS message.
  116. sdaftuar commented at 1:05 pm on May 9, 2023: member

    @sdaftuar did you have Current Thoughts about this approach? Trying to bridge IRC discussions to the blocking PR

    I was trying to come up with my own proof-of-concept to demonstrate an alternate approach before commenting, but just to get everyone up to speed on what I’m thinking:

    • The first commit “validation: introduce ChainstateManager::GetChainstateForNewBlock” seems like something we don’t conceptually need. I think the underlying issue is that AcceptBlock should be chainstate-agnostic – the decision to store a block on disk isn’t fundamentally based on a chain tip or the utxos we have. (The reason it comes into play in the current logic is as a heuristic for anti-DoS checks.) I have a branch that refactors AcceptBlock and some related code into ChainstateManager to show what I mean. https://github.com/bitcoin/bitcoin/compare/master...sdaftuar:bitcoin:2023-04-blockstorage
    • I think we can avoid parametrizing data in net_processing by chainstate, and thus avoid storing chainstate-pointers or making the existing download logic more complex by having it support two different download strategies at the same time. My thought is that a lot of the complexity in FindNextBlocksToDownload comes from being able to download blocks towards any tip that our peer has which has more work than our own. However, in the background sync case, we are only interested in download blocks towards the snapshot base block hash. So all we need to know is whether our peer’s tip contains the base block hash, and if it does, we just walk from our background chainstate tip towards that base block and look for blocks that we haven’t requested yet.

    I don’t have a branch to share yet that implements that second bullet, but I’m hoping to get to it soon.

  117. in src/net_processing.cpp:5851 in 8e1d9a4bed outdated
    5861+
    5862+        // The first chainstate in line for processing will likely exhaust this
    5863+        // during IBD, but once it hits a tip capacity will trickle into subsequent
    5864+        // chainstates.
    5865+        unsigned int requests_available{
    5866+            state.nBlocksInFlight > MAX_BLOCKS_IN_TRANSIT_PER_PEER ?
    


    Sjors commented at 1:25 pm on May 9, 2023:
    8e1d9a4bedc73b1ce2124a8583df8ed78d0c48ed: nit >= seems more intuitive, so the else case can’t be 0.

    jamesob commented at 7:18 pm on May 9, 2023:
    Fixed.
  118. in src/net_processing.cpp:5859 in 8e1d9a4bed outdated
    5869+
    5870+        // GetAllForBlockDownload() ensures we get blocks for the snapshot
    5871+        // chainstate first. It is more important to get to the network's tip
    5872+        // quickly than do the background validation on the snapshot.
    5873+        for (const auto* const chainstate : m_chainman.GetAllForBlockDownload()) {
    5874+            if (CanServeBlocks(*peer) &&
    


    Sjors commented at 1:41 pm on May 9, 2023:

    8e1d9a4bedc73b1ce2124a8583df8ed78d0c48ed: Maybe move the CanServeBlocks(*peer) check out of this loop (and check < MAX_BLOCKS_IN_TRANSIT_PER_PEER first) to make the if statement more readable.

    You should also clarify that || !chainstate->IsInitialBlockDownload() is always true for the background validation chainstate, so we never use a pruned peer for this.

    I’m somewhat inclined with @ryanofsky’s observation that this abstraction is a bit overkill. You could consider dropping ac00102e4712af7b04fb59b0035df32d6d662d13 and just explicitly handling the background and tip behavior here.


    jamesob commented at 7:18 pm on May 9, 2023:
    Fixed, and I’ve dropped the commit in question.
  119. in src/net_processing.cpp:5875 in 8e1d9a4bed outdated
    5885+                    BlockRequested(pto->GetId(), *pindex);
    5886+                    LogPrint(BCLog::NET, "Requesting block %s (%d) peer=%d\n", pindex->GetBlockHash().ToString(),
    5887+                        pindex->nHeight, pto->GetId());
    5888+                }
    5889+
    5890+                requests_available -= vToDownload.size();
    


    Sjors commented at 1:47 pm on May 9, 2023:
    8e1d9a4bedc73b1ce2124a8583df8ed78d0c48ed: light preference for --requests_available in the for loop above.

    jamesob commented at 7:18 pm on May 9, 2023:
    Fixed.
  120. in src/net_processing.cpp:5889 in 8e1d9a4bed outdated
    5875+                    ((sync_blocks_and_headers_from_peer && !IsLimitedPeer(*peer))
    5876+                     || !chainstate->IsInitialBlockDownload())
    5877+                    && state.nBlocksInFlight < MAX_BLOCKS_IN_TRANSIT_PER_PEER) {
    5878+                std::vector<const CBlockIndex*> vToDownload;
    5879+                NodeId staller = -1;
    5880+                FindNextBlocksToDownload(*chainstate, *peer, requests_available, vToDownload, staller);
    


    Sjors commented at 1:47 pm on May 9, 2023:
    assert(requests_available >= vToDownload.size()) ? FindNextBlocksToDownload itself doesn’t guarantee that, because its vBlocks param doesn’t have to be empty.

    jamesob commented at 7:18 pm on May 9, 2023:
    Fixed, but using Assume() because that seems safer.
  121. in src/net_processing.cpp:1341 in 8e1d9a4bed outdated
    1336+    const CChain& our_chain = chainstate.m_chain;
    1337+    const CBlockIndex* our_tip = our_chain.Tip();
    1338+
    1339+    if (state->pindexBestKnownBlock == nullptr
    1340+            || state->pindexBestKnownBlock->nChainWork < our_tip->nChainWork
    1341+            || state->pindexBestKnownBlock->nChainWork < m_chainman.MinimumChainWork()) {
    


    Sjors commented at 1:59 pm on May 9, 2023:
    Could add a TODO here to relax this for background validation. And/or explain that we expect a node with less than the minimum chainwork to still be in IBD and not willing to serve us blocks? Or that it’s too much hassle to figure out which blocks they have in common with us (e.g. it’s a fork-coin)?

    jamesob commented at 7:14 pm on May 9, 2023:
    Hmm, would we relax this for background validation? I don’t really see why. Recall that “our_tip” is referring to the tip of the chainstate we’re working on (could be background) and not our network tip.

    Sjors commented at 9:11 am on May 10, 2023:
    I only meant the comparison to MinimumChainWork(). Though in practice the assume UTXO height and minimum chainwork are probably not that far apart.
  122. Sjors commented at 2:09 pm on May 9, 2023: member
    Other than my remarks I’m happy with c7cdc7ca833ce1b5ce9a696fb4ef633c55528023.
  123. validation: introduce ChainstateManager::GetChainstateForNewBlock
    and use it in ProcessNewBlock().
    
    This is needed for an upcoming change to `net_processing`.
    c14ae132c5
  124. doc: add docstring for CanDirectFetch()
    Unrelated to assumeutxo changes, but while I was in the
    neighborhood...
    defddae090
  125. net_processing: work with multiple chainstates
    Most easily reviewed with `--ignore-space-change`
    
    - Modify FindNextBlocksToDownload() to parameterize the chainstate
      being worked on.
    
    - Change GetNodeStateStats to take the max nCommonHeight per peer across
      all chainstates.
    
    - Add CNodeState::chainstate_to_last_common_block
      * we need this to allow handling for a single peer to distinguish
        between separate chainstates we're simultaneously downloading blocks for
    
    - Share `requests_available` across chainstates when finding the next blocks
      to download (call to `FindNextBlocksToDownload()`).
    058fecdc7d
  126. p2p: don't advertise until we finish all IBDs
    Until we have completed IBD on all chainstates, don't answer getheaders
    requests from our peers or advertise ourselves. This preserves the
    existing behavior of not performing these actions until IBD of the
    whole chain completes.
    46a6fe7f42
  127. net_processing: add commentary describing IsIBD use
    For all remaining usages of IsInitialBlockDownload() where
    the status of any background validation chainstate that might
    be in use isn't relevant.
    ac9adf0129
  128. jamesob force-pushed on May 9, 2023
  129. jamesob commented at 7:21 pm on May 9, 2023: member
    I’ve udpated this PR to remove the new ChainstateManager::GetAllForBlockDownload(), instead adding a parameter assumed_first to the existing GetAll(). I think this is clearer at the net_processing call site and avoids another unnecessary method.
  130. jamesob commented at 7:22 pm on May 9, 2023: member
    I’ve updated this PR to remove the new ChainstateManager::GetAllForBlockDownload(), instead adding a parameter assumed_first to the existing GetAll(). I think this is clearer at the net_processing call site and avoids another unnecessary method.
  131. jamesob commented at 7:24 pm on May 9, 2023: member

    Also, as far as I understand it, AssumeUTXO will mess up the order of the blocks stored on disk - the first blockfiles would have the blocks between the AssumeUTXO block and the tip, and the subsequent blockfiles would have the blocks between Genesis and the AssumeUTXO block. I wonder if that could break or slow down something, have you tried a reindex for example?

    It’s worth noting that there’s a proposed remedy for this in #27596 - see https://github.com/bitcoin/bitcoin/pull/27596/commits/b29f39ecd9affdfa90e273c6e6422b5c2c04cd01. cc @mzumsande

  132. in src/validation.cpp:5556 in c14ae132c5 outdated
    5547@@ -5543,6 +5548,28 @@ SnapshotCompletionResult ChainstateManager::MaybeCompleteSnapshotValidation(
    5548     return SnapshotCompletionResult::SUCCESS;
    5549 }
    5550 
    5551+Chainstate& ChainstateManager::GetChainstateForNewBlock(const uint256& blockhash)
    5552+{
    5553+    AssertLockHeld(::cs_main);
    5554+    // Early return to avoid unnecessary blockindex lookup when assumeutxo is not in use.
    5555+    if (!m_snapshot_chainstate) {
    5556+        return *Assert(m_ibd_chainstate);
    


    fjahr commented at 9:11 am on May 16, 2023:
    somewhat nit: Why not use ActiveChainstate() here? From my understanding, it doesn’t functionally make a difference but it makes this a cleaner review since the result of this is replacing ActiveChainstate().
  133. in src/validation.cpp:5005 in 058fecdc7d outdated
    4997@@ -4998,12 +4998,15 @@ std::optional<uint256> ChainstateManager::SnapshotBlockhash() const
    4998     return std::nullopt;
    4999 }
    5000 
    5001-std::vector<Chainstate*> ChainstateManager::GetAll()
    5002+std::vector<Chainstate*> ChainstateManager::GetAll(bool assumed_first)
    5003 {
    5004     LOCK(::cs_main);
    5005     std::vector<Chainstate*> out;
    5006+    const auto chainstates = assumed_first ?
    


    fjahr commented at 9:50 am on May 16, 2023:

    This parameter/api seems kind of unnecessary. Do we actually need/want assumed_first=false anywhere explicitly? Otherwise, I would suggest always returning assumed first and adding a comment that this specific ordering is important.

    Alternatively, if it is better to have a default behavior of assumed_first=false then I would still find it simpler to reverse the order in the one place where it’s needed rather than adding the param.


    Sjors commented at 5:27 pm on May 16, 2023:
    This seems fairly easy to refactor later.
  134. in src/net_processing.cpp:5227 in 46a6fe7f42 outdated
    5223@@ -5224,9 +5224,10 @@ void PeerManagerImpl::MaybeSendAddr(CNode& node, Peer& peer, std::chrono::micros
    5224     if (!peer.m_addr_relay_enabled) return;
    5225 
    5226     LOCK(peer.m_addr_send_times_mutex);
    5227+    const bool is_ibd{WITH_LOCK(::cs_main, return m_chainman.IsAnyChainInIBD())};
    


    fjahr commented at 12:21 pm on May 16, 2023:
    nit: I would prefer in_ibd here
  135. fjahr commented at 12:59 pm on May 16, 2023: contributor

    Code review ACK ac9adf012925c770dfe523c5b57451f313cc8be5

    This is alright, I am just not loving the new assumed_first parameter to be honest. But curious to hear what you say @jamesob . If you change it, I promise to re-review it swiftly.

  136. jamesob commented at 3:51 pm on May 16, 2023: member
    Thanks for the review! I’m done with nits on this one, and assumeutxo in general.
  137. in src/net_processing.cpp:5867 in 058fecdc7d outdated
    5877+        const bool is_limited = IsLimitedPeer(*peer);
    5878+
    5879+        // It is more important to get to the network's tip
    5880+        // quickly than do the background validation on the snapshot.
    5881+        for (const auto* const chainstate : m_chainman.GetAll(/*assumed_first=*/true)) {
    5882+            if (can_serve &&
    


    Sjors commented at 5:17 pm on May 16, 2023:
    058fecdc7d: could still move if (can_serve out of the loop to reduce brackets.
  138. Sjors commented at 5:35 pm on May 16, 2023: member

    Happy with the improvements in ac9adf012925c770dfe523c5b57451f313cc8be5.

    I wrote:

    The first commit tends to confuses me

    Could use some thoughts on that comment (from anyone really). @sdaftuar wrote:

    The first commit “validation: introduce ChainstateManager::GetChainstateForNewBlock" seems like something we don’t conceptually need.

    I’m pretty close to wrapping my head around it, so would be OK with merging it once I do and then refactor it. Avoiding the complexity in the first place sounds nice too, though someone else may need to take over the rebase baton from @jamesob in that case.

    Also, as far as I understand it, AssumeUTXO will mess up the order of the blocks stored on disk

    I’m fine dealing with that in later PRs.

  139. fjahr commented at 10:16 pm on May 16, 2023: contributor

    I wrote:

    The first commit tends to confuses me

    Could use some thoughts on that comment (from anyone really). @Sjors I found it hard to follow that message, to be honest :D Could you maybe try to formulate more clearly what is bothering you in particular and what questions you need to be answered? It might help if you reference the specific phases you are thinking of based on doc/design/assumeutxo.md. I think this is the most confusing part when discussing this.

  140. Sjors commented at 8:09 am on May 17, 2023: member
    I guess a simpler way to phrase it is: what can go wrong if c14ae132c5a5204a9a755c84c6de05fb30459221 picks the incorrect chainstate for a block? What are all the different ways in which we can receive a block? What do we expect to happen in each of these cases? Tests would be a really nice way to illustrate that, but are apparently a pain to write for net_processing.
  141. fjahr commented at 2:47 pm on May 17, 2023: contributor

    I guess a simpler way to phrase it is: what can go wrong if c14ae13 picks the incorrect chainstate for a block? What are all the different ways in which we can receive a block? What do we expect to happen in each of these cases? Tests would be a really nice way to illustrate that, but are apparently a pain to write for net_processing. @Sjors I have tried to create an overview of the different cases and added my understanding of what is happening, I hope this helps move the discussion along and clear up the confusion. Let me know if it helps or if I missed the point. Rather than copying it in here, it’s probably better to keep it in the gist, so it can be updated and referenced easier and GitHub doesn’t swallow it like the other comments. And I have thrown this together rather quickly so it’s probably more likely that I have made a mistake in the overview rather than that the code is wrong if something seems weird :)

    Here it is: https://gist.github.com/fjahr/4b90adf2c24b39b84d5e6f4822f6a14f

    So let’s sink some ships… *insert boating accident joke*

    The one thing that did not realize before is that syncing the ibd chain seems to work rather inefficiently unless I am missing something. From my understanding now (see A2, A3, A4) it seems that blocks would need to be received a second time in order to be included in ibd because they are first included in snapshot? I guess the naive way to fix this would be to look at the snapshot height and decide based on that which chainstate to use. But putting even more logic like this into net_processing seems wrong and makes me agree more with @sdaftuar that this may be better handled differently instead.

    Scattered thoughts that are probably ok but might be considered for cleanup later:

    • What are the implications of the snapshot being “inactive” while loading per doc/design/assumeutxo.md (Row 2)? There isn’t really a status like m_disabled for that. So I guess it’s is just theoretically inactive but it can already be used. Maybe note that in assumeutxo.md.
    • Still hitting the disabled chain also seems kind of wasteful but it’s probably not a big deal and shouldn’t happen too often in normal operation (Row 5).
    • We do potentially save many blocks twice if we receive them multiple times. I don’t think this is a strong attack vector, but in the extreme case a node could have the full chain on disk twice, so use 2x the storage until it is restarted.
  142. in src/validation.cpp:5555 in c14ae132c5 outdated
    5547@@ -5543,6 +5548,28 @@ SnapshotCompletionResult ChainstateManager::MaybeCompleteSnapshotValidation(
    5548     return SnapshotCompletionResult::SUCCESS;
    5549 }
    5550 
    5551+Chainstate& ChainstateManager::GetChainstateForNewBlock(const uint256& blockhash)
    5552+{
    5553+    AssertLockHeld(::cs_main);
    5554+    // Early return to avoid unnecessary blockindex lookup when assumeutxo is not in use.
    5555+    if (!m_snapshot_chainstate) {
    


    brunoerg commented at 2:19 pm on May 22, 2023:
    In c14ae132c5a5204a9a755c84c6de05fb30459221, is there a reason for not using IsSnapshotActive() here?
  143. in src/net_processing.cpp:1319 in 058fecdc7d outdated
    1315@@ -1301,7 +1316,12 @@ void PeerManagerImpl::UpdateBlockAvailability(NodeId nodeid, const uint256 &hash
    1316     }
    1317 }
    1318 
    1319-void PeerManagerImpl::FindNextBlocksToDownload(const Peer& peer, unsigned int count, std::vector<const CBlockIndex*>& vBlocks, NodeId& nodeStaller)
    1320+void PeerManagerImpl::FindNextBlocksToDownload(
    


    brunoerg commented at 3:09 pm on May 22, 2023:
    In 058fecdc7d2c99af9ae6aedc68c3600321e7c565: just a suggestion but I’d change FindNextBlocksToDownload to return std::vector<const CBlockIndex*> instead of passing it in the parameters.
  144. DrahtBot added the label Needs rebase on May 24, 2023
  145. DrahtBot commented at 10:41 am on May 24, 2023: contributor

    🐙 This pull request conflicts with the target branch and needs rebase.

  146. jamesob commented at 9:26 pm on May 26, 2023: member
    Closing in lieu of some simpler changes that @sdaftuar has written, currently integrated in #27596.
  147. jamesob closed this on May 26, 2023

  148. Sjors commented at 1:14 pm on May 27, 2023: member
    @fjahr thanks for the write up! It’s on my reading list. Will also look at @sdaftuar’s PR.
  149. bitcoin locked this on Jun 24, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-01 10:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me