p2p: Erlay support signaling #23443

pull naumenkogs wants to merge 9 commits into bitcoin:master from naumenkogs:2021-11-erlay1 changing 17 files +696 −5
  1. naumenkogs commented at 12:51 pm on November 5, 2021: member

    This is a part of the Erlay project:


    This PR adds a new p2p message sendtxrcncl signaling for reconciliation support. Before sending that message, a node is supposed to “pre-register” the peer by generating and storing an associated reconciliation salt component. Once the salts are exchanged within this new message, nodes “register” each other for future reconciliations by computing and storing the aggregate salt, along with the reconciliation parameters based on the connection direction.

  2. DrahtBot added the label Build system on Nov 5, 2021
  3. DrahtBot added the label P2P on Nov 5, 2021
  4. naumenkogs force-pushed on Nov 5, 2021
  5. DrahtBot commented at 8:10 pm on November 5, 2021: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #26151 (refactor: Guard TxRequestTracker by its own lock instead of cs_main by dergoegge)
    • #25957 (wallet: fast rescan with BIP157 block filters for descriptor wallets by theStack)
    • #25572 (refactor: Introduce EvictionManager and use it for the inbound eviction logic by dergoegge)
    • #24545 (BIP324: Enable v2 P2P encrypted transport by dhruv)
    • #23233 (BIP324: Add encrypted p2p transport {de}serializer by dhruv)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    Coverage

    Coverage Change (pull 23443, 4dc7bac65e41a1f2bbbae084b8bbaeee70cec700) Reference (master, ed4eeafbb6e2e73f)
    Lines +0.0276 % 83.7655 %
    Functions +0.0294 % 80.7326 %
    Branches +0.0133 % 51.3151 %

    Updated at: 2022-05-10T14:16:22.523123.

  6. fanquake removed the label Build system on Nov 8, 2021
  7. naumenkogs force-pushed on Nov 8, 2021
  8. naumenkogs force-pushed on Nov 11, 2021
  9. naumenkogs force-pushed on Nov 17, 2021
  10. in src/net_processing.cpp:2619 in a4400b4ffc outdated
    2575@@ -2574,6 +2576,15 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    2576 
    2577         if (greatest_common_version >= WTXID_RELAY_VERSION) {
    2578             m_connman.PushMessage(&pfrom, msg_maker.Make(NetMsgType::WTXIDRELAY));
    2579+
    2580+            // Reconciliation is supported only when wtxid relay is supported for only
    


    ariard commented at 1:53 am on November 23, 2021:

    Note, I think BIP330 should make the dependency on BIP339 explicit.

    Our implementation makes a requirement to speak a protocol version superior to WTXID_RELAY_VERSION. Further, it also relies on MSG_WTX for the fallback inv announcement. Those 2 elements are defined in BIP339. I think a client implementing straightly the BIP won’t be able to tx-announce with Erlay-supported Core nodes.

    IIRC, this is mostly a historical artifact because BIP330 was submitted before BIP339.

    If you agree, please stage this comment until there are multiple updates worthy to do to the BIP.


    naumenkogs commented at 10:54 am on November 23, 2021:
    In BIP already: Since sketches are based on the WTXIDs, the negotiation and support of Erlay should be enabled only if both peers signal BIP-339 support.

    ariard commented at 0:09 am on November 24, 2021:
    I’ve grepped the phrase in https://github.com/bitcoin/bips/blob/master/bip-0330.mediawiki and I can’t find it ? Are we sure we’re looking on the same version of the BIP?

    naumenkogs commented at 8:15 am on November 24, 2021:

    An up-to-date BIP version is linked in the first post of this PR. This link: https://github.com/naumenkogs/bips/blob/bip_0330_updates/bip-0330.mediawiki

    At some point we want to merge it to the bip repo, but probably later.


    ariard commented at 3:24 am on November 26, 2021:
    Thanks for the pointer. Well the update warning could be in bold :p
  11. in src/protocol.h:267 in a4400b4ffc outdated
    261@@ -262,6 +262,13 @@ extern const char* CFCHECKPT;
    262  * @since protocol version 70016 as described by BIP 339.
    263  */
    264 extern const char* WTXIDRELAY;
    265+/**
    266+ * Contains 2 1-byte bools, a 4-byte version number and an 8-byte salt.
    267+ * Indicates that a node is willing to participate in transaction reconciliation,
    


    ariard commented at 1:55 am on November 23, 2021:
    nit: “The boolean indicates”
  12. in src/txreconciliation.cpp:86 in a4400b4ffc outdated
    31+
    32+    std::tuple<bool, bool, uint32_t, uint64_t> PreRegisterPeer(NodeId peer_id, bool peer_inbound)
    33+    {
    34+        bool we_initiate_recon, we_respond_recon;
    35+        // Currently reconciliation roles are defined by the connection direction: only the inbound
    36+        // peer initiate reconciliations and the outbound peer is supposed to only respond.
    


    ariard commented at 2:24 am on November 23, 2021:

    If the reconciliation roles are updated, do you need to bump the RECON_VERSION ?

    I would say so as a old-reconciliation outbound peer could assume a new-reconciliation inbound peer is the initiator when the new-reconciliation inbound peer would estimate the reverse, thus halting reconciliation ?


    naumenkogs commented at 10:37 am on November 23, 2021:
    Yeah I think you are probably right. What should i do with your comment though?

    ariard commented at 0:14 am on November 24, 2021:

    I would say add a “If the reconciliation roles are updated, RECON_VERSION should be bumped” comment ?

    I think it’s nice to ease future reconciliation development or extension by laying out clearly what should be updated or taken care of by the ones doing the work. A good chunk of today p2p mechanisms aren’t that much documented in that regard.


    naumenkogs commented at 8:19 am on November 24, 2021:
    ack
  13. in src/txreconciliation.cpp:23 in 0e9d0dc852 outdated
    18+ * to compute transaction short IDs, which are then used to construct a sketch representing a set
    19+ * of transactions we want to announce to the peer.
    20+ */
    21+static uint256 ComputeSalt(uint64_t local_salt, uint64_t remote_salt)
    22+{
    23+    // Accoring to BIP-330, salts should be combined in ascending order.
    


    ariard commented at 2:32 am on November 23, 2021:
    nit: s/Accoring/According/g
  14. in src/net_processing.cpp:2827 in 85992f3f2c outdated
    2821@@ -2822,6 +2822,48 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    2822         return;
    2823     }
    2824 
    2825+    // Received from a peer demonstrating readiness to announce transactions via reconciliations.
    2826+    // This feature negotiation should happen:
    2827+    // - between VERSION and VERACK to avoid relay problems from switching annoyncement protocols
    


    ariard commented at 2:38 am on November 23, 2021:
    nit: s/annoyncement/announcement/g
  15. in src/net_processing.cpp:2839 in 85992f3f2c outdated
    2834+            pfrom.fDisconnect = true;
    2835+            return;
    2836+        }
    2837+
    2838+        if (!pfrom.m_tx_relay) {
    2839+            // If we chose a peer to not send us transactions, disconnect if they want to reconcile.
    


    ariard commented at 2:43 am on November 23, 2021:
    nit: s/chose/choose/g
  16. in src/txreconciliation.cpp:112 in 85992f3f2c outdated
    104+        bool they_may_initiate, bool they_may_respond, uint32_t recon_version, uint64_t remote_salt)
    105+    {
    106+        // We do not support reconciliation salt/version updates. We treat an attempt to update
    107+        // after a successful registration as a protocol violation.
    108+        LOCK(m_mutex);
    109+        if (m_states.find(peer_id) != m_states.end()) return false;
    


    ariard commented at 3:04 am on November 23, 2021:

    This check means that duplicated SENDRECON are processed as protocol violation, correct ?

    I wonder if we have Bitcoin nodes widely deployed over non-TCP stack, where the packet dedup is not guaranteed. I guess that should be so rare a case, though if we learn some and the duplication rate is high enough to trigger this reconciliation failure we can still relax this check.


    naumenkogs commented at 10:45 am on November 23, 2021:
    Yeah good point, keeping it open for now so that maybe others comment if they want to relax the check.

    sipa commented at 2:49 pm on November 25, 2021:
    Bitcoin P2P connections are stateful, so you can’t generally run them over say a UDP stack anyway.
  17. in src/txreconciliation.cpp:120 in 85992f3f2c outdated
    112+        // they support. For now, this only guarantees that nodes with future reconciliation
    113+        // versions have the choice of reconciling with this current version. However, they also
    114+        // have the choice to refuse supporting reconciliations if the common version is not
    115+        // satisfactory (e.g. too low).
    116+        recon_version = std::min(recon_version, RECON_VERSION);
    117+        // v1 is the lowest version, so suggesting something below must be a protocol violation.
    


    ariard commented at 3:05 am on November 23, 2021:
    Note, the BIP330 says version != 1 should be ignored, making the 0 case a protocol violation should be reflected in the BIP, I think.

    naumenkogs commented at 10:51 am on November 23, 2021:
    Done in 38e66d31f7eb47cd2566003e8d0d8808b129eabb

  18. in src/txreconciliation.cpp:155 in 85992f3f2c outdated
    148     {
    149         LOCK(m_mutex);
    150-        if (m_local_salts.erase(peer_id)) {
    151+        auto salt_erased = m_local_salts.erase(peer_id);
    152+        auto state_erased = m_states.erase(peer_id);
    153+        if (salt_erased || state_erased) {
    


    ariard commented at 3:16 am on November 23, 2021:
    Shouldn’t be a &&, otherwise we might have staling state for the peer ?

    naumenkogs commented at 10:53 am on November 23, 2021:

    This line only defines whether we print logs or not, so I don’t understand how does it affect the actual behavior. The ideas is that it logs deleting the state for both pre-registered and registered peers.

    The previous 2 lines taking care of deletion are indeed &&.


    ariard commented at 0:18 am on November 24, 2021:

    The ideas is that it logs deleting the state for both pre-registered and registered peers.

    I understand. Maybe maybe the log comment could have a ternary and print “salt_erased” or “state_erased” to be more meaningful. Though that’s really low-level information likely only relevant for debug/hacking.

  19. ariard commented at 3:17 am on November 23, 2021: member
    Reviewed until 85992f3. So far only minor comments.
  20. naumenkogs force-pushed on Nov 23, 2021
  21. in src/test/txreconciliation_tests.cpp:23 in 4c0122e06b outdated
    19+    assert(we_respond_recon);
    20+    assert(recon_version == 1); // RECON_VERSION in src/txreconciliation.cpp
    21+
    22+    std::tie(we_initiate_recon, we_respond_recon, recon_version, recon_salt) = tracker.PreRegisterPeer(1, false);
    23+    assert(we_initiate_recon);
    24+    assert(!we_respond_recon);
    


    ariard commented at 11:52 pm on November 23, 2021:
    I don’t remember if the boost framework has an equivalent of the rust’s should_panic, if so maybe test duplicated registration ? Should hit the assert.

    naumenkogs commented at 8:24 am on November 24, 2021:
    Yeah that was my problem as well, not sure how to test assertions failed.

    maflcko commented at 8:02 am on November 29, 2021:
    Assertions can’t be tested in unit tests. If this was an exception, it could be tested.
  22. ariard commented at 0:06 am on November 24, 2021: member
    Finished a first parse, including verifying unit test coverage of the new TxReconciliationTracker class. LGTM
  23. rebroad commented at 8:12 pm on November 25, 2021: contributor
    0txreconciliation.cpp:43:20: warning: private field 'm_k0' is not used [-Wunused-private-field]
    1    const uint64_t m_k0, m_k1;
    2                   ^
    3txreconciliation.cpp:43:26: warning: private field 'm_k1' is not used [-Wunused-private-field]
    4    const uint64_t m_k0, m_k1;
    
  24. naumenkogs force-pushed on Nov 29, 2021
  25. in src/Makefile.am:371 in 6a79aae095 outdated
    367@@ -367,6 +368,7 @@ libbitcoin_server_a_SOURCES = \
    368   txdb.cpp \
    369   txmempool.cpp \
    370   txorphanage.cpp \
    371+  txreconciliation.cpp \
    


    maflcko commented at 7:58 am on November 29, 2021:

    would be nice to put this into the node directory. There is really no way to do anything tx-relay related without a full node.

    Also, as you are putting this into libserver, for clarity the file should also be in the right directory.

  26. in src/protocol.h:271 in 6a79aae095 outdated
    266+ * Contains 2 1-byte bools, a 4-byte version number and an 8-byte salt.
    267+ * The boolean indicates that a node is willing to participate in transaction
    268+ * reconciliation, either as a sender or a receiver.
    269+ * The salt is used to compute short txids needed for efficient reconciliation.
    270+ */
    271+extern const char *SENDRECON;
    


    maflcko commented at 7:59 am on November 29, 2021:

    nit:

    0extern const char* SENDRECON;
    
  27. maflcko commented at 8:05 am on November 29, 2021: member
    left some nits
  28. maflcko commented at 3:24 pm on November 29, 2021: member
    Run process_message with args ['/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz', '-runs=1', '/tmp/cirrus-ci-build/ci/scratch/qa-assets/fuzz_seed_corpus/process_message']fuzz: test/fuzz/process_message.cpp:57: auto initialize_process_message()::(anonymous class)::operator()() const: Assertion “GetNumMsgTypes() == getAllNetMessageTypes().size()” && check’ failed.
  29. in src/net_processing.cpp:2842 in 04ef6f7ccd outdated
    2833+            LogPrint(BCLog::NET, "sendrecon received after verack from peer=%d; disconnecting\n", pfrom.GetId());
    2834+            pfrom.fDisconnect = true;
    2835+            return;
    2836+        }
    2837+
    2838+        if (!pfrom.m_tx_relay) {
    


    mzumsande commented at 0:52 am on November 30, 2021:
    Isn’t this a problem in the case of a block-relay-only connection? If I initiate an outbound block-relay-only connection, I set m_tx_relay=nullptr for this peer, don’t send SENDRECON and set fRelay=0 in my version msg. My peer gets an inbound connection, will set pfrom.m_tx_relay->fRelayTxes to false, but will still send out SENDRECON because m_tx_relay!=nullptr. So wouldn’t I then receive a SENDRECON msg and immediately disconnect the peer?

    naumenkogs commented at 11:11 am on December 2, 2021:

    I think you are right. This case shouldn’t get us disconnected from the peer.

    What would be the best way to detect peers which clearly do something meaningless? I mean, this check could be just dropped I guess, but ideally, it should handle misbehavior.


    mzumsande commented at 6:49 pm on December 5, 2021:

    I don’t really think of this as misbehaviour: I think of the meaning of SENDRECON as “I understand reconciliation and would like to participate in it on this connection” - if just on of the two sides signal that - ok, no reconciliation on this connection, but no harm done. So I think it might make sense to drop the check.

    That being said, I think it might still make sense to not send SENDRECON when our peer has signified fRelay=false in their version message. Obviously, this would also mean no reconciliation with BIP37 peers, but that seems also like a good thing?! This has the additional benefit that no reconciliations would happen when one side is in blocksonly mode (i.e. has m_tx_relay!=nullptr but still doesn’t wish to receive transactions). I think that currently, both sides would send SENDRECON in this case. One could also prevent this on the other side, by not sending out SENDRECON ourselves when in -blocksonly mode (m_ignore_incoming_txs set)


    naumenkogs commented at 9:10 am on December 6, 2021:
    I agree with most of what you say except the BIP37 part. Are you implying BIP37 peers won’t want reconciliations?

    mzumsande commented at 11:12 pm on December 6, 2021:

    Not sure. I don’t know too much about BIP37 and its usage, but I thought it unlikely that SPVs using it would go through the efforts of implementing it and get significant gains from it when they are interested in a limited number of transactions anyway. Do you think participating in recons would make sense for them?

    The issue is that BIP37 peers initially set fRelay=false in their version and later activate tx relay by sending a filter. So, if BIP37 peers should use reconciliation, disabling SENDRECON based on receiving fRelay=false is no longer possible, and all the logic needs to be on the side of the peer that sends fRelay=false (although this work just fine).


    naumenkogs commented at 11:16 am on December 7, 2021:

    First of all, I’d note BIP37 user != SPV. You can think of another kind of implementation of BIP37, which filters, for example, 25% of all the addr space (not sure it’s practical currently w.r.t filter size), doing that over N=20 connections with different filters to receive all transactions overall. Given how unrealistic that is today, we can probably implement Erlay support for that once we know about this scenario?

    Secondly, let’s consider a regular small-filter (say, 1000 of addrs for redundancy) SPV node. If it’s connected to X serving nodes, it’d still get X times more tx announcements than needed. But considering how little is that bandwidth, reducing this redundancy might be useless, so yeah.

    At the same time, I don’t think this justification is particularly strong, but yeah, in the end we could just suggest SPV nodes to not send SENDRECON if they don’t want. In that case, the outcome is that regular nodes would send SENDRECON even though ‘fRelay=false’, and in some cases that would be useless, but that’s not a big deal.

    Do you agree with this thinking?


    mzumsande commented at 1:04 am on December 8, 2021:

    Yes, I agree it makes sense. To sum up, the suggestion would be

    1. to remove the pfrom.m_tx_relay check when receiving SENDRECON in order to not disconnect all block-relay-only peers, and to not disconnect if we receive a SENDRECON message but haven’t sent one.
    2. to add a check for m_ignore_incoming_txs here in order to opt out of recons when in -blocksonly mode

    glozow commented at 6:43 pm on April 19, 2022:
    Catching up to this discussion (sorry for missing it earlier). I’m not convinced that we care enough about doing tx reconciliation with bloom filtered peers to justify creating reconciliation-related state for them. If a peer is filtering transactions, obviously you don’t care reconciling mempools: they clearly don’t want all of your mempool’s transactions, and most likely you won’t learn much from them. Bloom filters are also disabled by default.
  30. naumenkogs force-pushed on Dec 2, 2021
  31. naumenkogs force-pushed on Dec 2, 2021
  32. in src/node/txreconciliation.h:23 in ea3b87af90 outdated
    18+ * 0. Reconciliation protocol handshake.
    19+ * 1. Once we receive a new transaction, add it to the set instead of announcing immediately
    20+ * 2. When the time comes, a reconciliation initiator requests a sketch from the peer, where a sketch
    21+ *    is a compressed representation of their set
    22+ * 3. Once the initiator received a sketch from the peer, the initiator computes a local sketch,
    23+ *    and combines the two skethes to find the difference in *sets*.
    


    stickies-v commented at 0:51 am on December 5, 2021:

    typo:

    0 *    and combines the two sketches to find the difference in *sets*.
    
  33. in src/node/txreconciliation.h:20 in ea3b87af90 outdated
    15+ * Transaction reconciliation is a way for nodes to efficiently announce transactions.
    16+ * This object keeps track of all reconciliation-related communications with the peers.
    17+ * The high-level protocol is:
    18+ * 0. Reconciliation protocol handshake.
    19+ * 1. Once we receive a new transaction, add it to the set instead of announcing immediately
    20+ * 2. When the time comes, a reconciliation initiator requests a sketch from the peer, where a sketch
    


    stickies-v commented at 0:54 am on December 5, 2021:
    nit: “When the time comes” doesn’t really say much. I would either remove or replace with something more meaningful, like “At repeated intervals”
  34. in src/node/txreconciliation.h:21 in ea3b87af90 outdated
    16+ * This object keeps track of all reconciliation-related communications with the peers.
    17+ * The high-level protocol is:
    18+ * 0. Reconciliation protocol handshake.
    19+ * 1. Once we receive a new transaction, add it to the set instead of announcing immediately
    20+ * 2. When the time comes, a reconciliation initiator requests a sketch from the peer, where a sketch
    21+ *    is a compressed representation of their set
    


    stickies-v commented at 0:59 am on December 5, 2021:

    nit: since we refer to short IDs later, I would change introduce this here:

    0 *    is a compressed representation of short form IDs of the transactions in their set
    
  35. in src/node/txreconciliation.h:28 in ea3b87af90 outdated
    23+ *    and combines the two skethes to find the difference in *sets*.
    24+ * 4. Now the initiator knows full symmetrical difference and can request what the initiator is
    25+ *    missing and announce to the peer what the peer is missing. For the former, an extra round is
    26+ *    required because the initiator knows only short IDs of those transactions.
    27+ * 5. Sometimes reconciliation fails if the difference is larger than the parties estimated,
    28+ *    then there is one sketch extension round, in which the initiator requests for extra data.
    


    stickies-v commented at 1:06 am on December 5, 2021:

    I think reconciliation always fails if capacity is too low? And some suggested general rewording

    0 * 5. If the difference is larger than estimated, reconciliation fails. Via an extension
    1 *    round (allowed only once), the initiator can request a larger sketch.
    
  36. in src/node/txreconciliation.h:32 in ea3b87af90 outdated
    27+ * 5. Sometimes reconciliation fails if the difference is larger than the parties estimated,
    28+ *    then there is one sketch extension round, in which the initiator requests for extra data.
    29+ * 6. If extension succeeds, go to step 4.
    30+ * 7. If extension fails, the initiator notifies the peer and announces all transactions from the
    31+ *    corresponding set. Once the peer received the failure notification, the peer announces all
    32+ *    transactions from the corresponding set.
    


    stickies-v commented at 1:12 am on December 5, 2021:

    nit: I think extension is optional, so maybe reconciliation is true in a more general sense?

    0 * 7. If reconciliation fails, the initiator notifies the peer and announces all transactions from its
    1 *    own set. Once the peer received the failure notification, the peer announces all
    2 *    transactions from their set.
    
  37. in src/node/txreconciliation.cpp:96 in ea3b87af90 outdated
    91+        } else {
    92+            we_initiate_recon = true;
    93+            we_respond_recon = false;
    94+        }
    95+
    96+        uint64_t m_local_recon_salt(GetRand(UINT64_MAX));
    


    stickies-v commented at 1:27 am on December 5, 2021:

    nit: don’t think that’s a member var?

    0        uint64_t local_recon_salt(GetRand(UINT64_MAX));
    

    stickies-v commented at 2:52 pm on December 8, 2021:
    nit: Also, there may be a slight resource wastage by calculating the local salt in PreRegisterPeer already, since the peer may not make it to the next step. Have you considered doing this in the RegisterPeer step? More extremely, it could also be delayed until an actual reconciliation request is sent/received, but that extra code complexity of checking if the local salt exists for every request probably isn’t worth it.

    jnewbery commented at 4:40 pm on December 8, 2021:
    0        uint64_t m_local_recon_salt{GetRand(UINT64_MAX)};
    

    naumenkogs commented at 12:16 pm on December 9, 2021:
    The reason for computing it here is that we need to return the salt from PreRegisterPeer, because we send the salt to the peer after pre-registering it.

    stickies-v commented at 12:24 pm on December 9, 2021:
    Right, didn’t think of that, thanks for clearing up!
  38. in src/node/txreconciliation.cpp:24 in ea3b87af90 outdated
    19+ * of transactions we want to announce to the peer.
    20+ */
    21+static uint256 ComputeSalt(uint64_t local_salt, uint64_t remote_salt)
    22+{
    23+    // According to BIP-330, salts should be combined in ascending order.
    24+    uint64_t salt1 = local_salt, salt2 = remote_salt;
    


    stickies-v commented at 1:51 am on December 5, 2021:

    Since ComputeSalt doesn’t care about which is local and which is remote and you’re passing by value, I think you could just skip the entire reassignment step

    0static uint256 ComputeSalt(uint64_t salt1, uint64_t salt2)
    1{
    2    // According to BIP-330, salts should be combined in ascending order.
    

    naumenkogs commented at 12:20 pm on December 9, 2021:

    I think without reassignment, swapping them for ordering would be unclear. You swap local with remote, and they loose the meaning.

    Maybe you can suggest a full code fragment for this function, which doesn’t have this problem?


    stickies-v commented at 12:29 pm on December 9, 2021:

    I agree, that would be confusing. That’s why in the above suggested change, I also updated the function signature. There is no more reference to salts being local or remote.

    The Suggested change should be complete (i.e. just applying the diff should be enough, but for completeness the entire function would be:

    0static uint256 ComputeSalt(uint64_t salt1, uint64_t salt2)
    1{
    2    // According to BIP-330, salts should be combined in ascending order.
    3    if (salt1 > salt2) std::swap(salt1, salt2);
    4
    5    static auto RECON_SALT_HASHER = TaggedHash(RECON_STATIC_SALT);
    6    return (RECON_SALT_HASHER << salt1 << salt2).GetSHA256();
    7}
    
  39. in src/node/txreconciliation.cpp:128 in 4ac0de088d outdated
    123+        auto local_salt = m_local_salts.find(peer_id);
    124+
    125+        // This function should be called only after generating the local salt.
    126+        if (local_salt == m_local_salts.end()) return false;
    127+
    128+        // Must match SuggestReconciling logic.
    


    mzumsande commented at 0:23 am on December 6, 2021:
    SuggestReconciling sounds like a function name, but there is no such function.
  40. in src/node/txreconciliation.h:23 in b55cbf63e1 outdated
    18+ * 0. Reconciliation protocol handshake.
    19+ * 1. Once we receive a new transaction, add it to the set instead of announcing immediately
    20+ * 2. When the time comes, a reconciliation initiator requests a sketch from the peer, where a sketch
    21+ *    is a compressed representation of their set
    22+ * 3. Once the initiator received a sketch from the peer, the initiator computes a local sketch,
    23+ *    and combines the two skethes to find the difference in *sets*.
    


    mzumsande commented at 1:09 am on December 6, 2021:
    nit:sketches
  41. in src/protocol.h:268 in ea3b87af90 outdated
    261@@ -262,6 +262,13 @@ extern const char* CFCHECKPT;
    262  * @since protocol version 70016 as described by BIP 339.
    263  */
    264 extern const char* WTXIDRELAY;
    265+/**
    266+ * Contains 2 1-byte bools, a 4-byte version number and an 8-byte salt.
    267+ * The boolean indicates that a node is willing to participate in transaction
    268+ * reconciliation, either as a sender or a receiver.
    


    stickies-v commented at 3:47 pm on December 6, 2021:

    nit: I think this can be slightly more precise without increasing complexity, like in:

    0 * The 2 booleans indicate that a node is willing to participate in transaction
    1 * reconciliation, respectively as an initiator or as a receiver.
    
  42. mzumsande commented at 10:47 pm on December 6, 2021: contributor
    Did you think about a -disablerecon command or something similar, to not activate Erlay until all the parts have been merged? Providing the opportunity to opt in/out would probably make sense anyway, but it would also make the review process of its parts independent from releases - e.g. I also don’t think it makes sense to have nodes running master exchange SENDRECON before Erlay has been fully merged.
  43. in src/node/txreconciliation.h:32 in b55cbf63e1 outdated
    27+ * 5. Sometimes reconciliation fails if the difference is larger than the parties estimated,
    28+ *    then there is one sketch extension round, in which the initiator requests for extra data.
    29+ * 6. If extension succeeds, go to step 4.
    30+ * 7. If extension fails, the initiator notifies the peer and announces all transactions from the
    31+ *    corresponding set. Once the peer received the failure notification, the peer announces all
    32+ *    transactions from the corresponding set.
    


    glozow commented at 10:05 am on December 7, 2021:

    Nit in b55cbf63e15766bdabcac1c08b4cbfea0badeb04:

    This ordering was really confusing for me, “Step 4: now we know the full symmetrical difference… Step 5: sometimes we don’t…. Step 6: go back to Step 4.” It seems more intuitive if Step 4 is at the end, and other steps can “Skip to Step n if successful.”

  44. in src/net_processing.cpp:3504 in 4ac0de088d outdated
    2847+        if (!State(pfrom.GetId())->m_wtxid_relay) {
    2848+            // Disconnect peers that send a SENDRECON message before/without WTXIDRELAY.
    2849+            LogPrint(BCLog::NET, "sendrecon received before wtxidrelay peer=%d; disconnecting\n", pfrom.GetId());
    2850+            pfrom.fDisconnect = true;
    2851+            return;
    2852+        }
    


    glozow commented at 10:12 am on December 7, 2021:

    In 4ac0de088db9b1d3926daaf4f55dc780a77a7f2c:

    BIP330 sendrecon specification says:

    The sendrecon message […] Should be sent only after sending “wtxidrelay”, and should be ignored if received before receiving “wtxidrelay” from the peer.

    So it seems incorrect to disconnect. We should just ignore, i.e. return, no?

  45. in src/node/txreconciliation.cpp:113 in 4ac0de088d outdated
    121+        if (recon_version < 1) return false;
    122+
    123+        auto local_salt = m_local_salts.find(peer_id);
    124+
    125+        // This function should be called only after generating the local salt.
    126+        if (local_salt == m_local_salts.end()) return false;
    


    glozow commented at 10:16 am on December 7, 2021:

    In 4ac0de088db9b1d3926daaf4f55dc780a77a7f2c:

    This doesn’t seem like it would be a peer protocol violation, but a bug in our node/implementation. If we didn’t PreRegisterPeer() for some reason, we would disconnect the peer even though it’s our fault. Is this safe? Would it be better to assert() or Assume() instead?

  46. in src/node/txreconciliation.h:72 in 1e83fda9fe outdated
    56@@ -57,6 +57,14 @@ class TxReconciliationTracker {
    57      * This function must be called only once per peer.
    58      */
    59     std::tuple<bool, bool, uint32_t, uint64_t> PreRegisterPeer(NodeId peer_id, bool peer_inbound);
    60+
    61+    // Helpers
    62+
    63+    /**
    64+     * Attempts to forget reconciliation-related state of the peer (if we previously stored any).
    65+     * After this, we won't be able to reconcile with the peer.
    


    glozow commented at 10:18 am on December 7, 2021:

    In 1e83fda9fe1be2ad17d57b287cb672595e3173a4:

    0     * Tear down all reconciliation-related state for this peer, if any exists.
    

    I don’t think this captures the purpose of this function quite accurately; we’re not banning the peer from reconciliation, we’re just tearing down state for a disconnected peer. Perhaps it would be more clear to just call this function DisconnectedPeer() (similar to txrequest)


    naumenkogs commented at 9:59 am on December 21, 2021:
    With the latest code, we also call it when a peer forgot to send us wtxid (although being pre-registered by sending us SENDRECON). So DisconenctedPeer() would be quite inaccurate.

    glozow commented at 12:46 pm on February 18, 2022:
    Makes sense - can you update the commit message of 182d984136066fd4f09b3e1a1bbd9b50c6cb87be to say “Forget peer’s reconciliation state on disconnect and when wtxidrelay is not sent” so that it’s documented there as well?
  47. in src/node/txreconciliation.cpp:45 in 4ac0de088d outdated
    48+
    49+    /**
    50+     * Reconciliation protocol assumes using one role consistently: either a reconciliation
    51+     * initiator (requesting sketches), or responder (sending sketches). This defines our role.
    52+     * */
    53+    const bool m_we_initiate;
    


    glozow commented at 10:29 am on December 7, 2021:

    In 4ac0de088db9b1d3926daaf4f55dc780a77a7f2c

    Small food for thought - the many assertions between bools to ensure one peer is initiator and the other peer is responder suggests to me that ReconciliationState is an appropriate use case for a std::variant.


    naumenkogs commented at 10:00 am on December 21, 2021:
    Yeah my initial intention was to make it good for future where both could be true or false, and avoid having a 4-way variant of {YESYES NONO YESNO NOYES}. But hey, since this future might be very far ahead, you’re probably right. Will do.

    naumenkogs commented at 10:20 am on December 21, 2021:

    Actually, I think, it would be very useful if you take a look at the final Erlay code fragment?

    So, the structure there is:

    1. class ReconciliationState, which has 2 fields: ReconciliationInitByUs and ReconciliationInitByUs.
    2. Those two fields are structs, corresponding to Initiator and Responder roles. They encapsulate specific stuff.
    3. The rest of ReconciliationState is stuff which is not specific to a role (salt, for example).

    I guess I just used another pattern for that.

    If I understand you correctly, you suggesting using std::variant<Initiator, Responder> instead of ReconciliationState.

    Is std::variant still a good candidate in that case? What would be the right way for them to have many fields in common? I guess Initiator and Responder could be inherited from one shared parent with all that stuff.


    glozow commented at 9:54 am on January 12, 2022:
    Sorry, I have realized I was missing some context. Right now, one peer is Initiator and the other is Responder, but do you envision it being more dynamic in future versions of the protocol? If so, perhaps it doesn’t make sense to use a variant. But perhaps, since you only have 2 possibilities right now, you only need 1 bool instead of 2?

    naumenkogs commented at 12:12 pm on January 26, 2022:
    Based on more discussions with Gloria, I think we keep this field, but in latter commits we switch to std::variant<ReconciliationInitByUs, ReconciliationInitByThem> m_current_recon; which would dictate us the role of the peer.
  48. in src/test/txreconciliation_tests.cpp:18 in 76c6e557e8 outdated
    13+BOOST_AUTO_TEST_CASE(PreRegisterPeerTest)
    14+{
    15+    TxReconciliationTracker tracker;
    16+
    17+    auto [we_initiate_recon, we_respond_recon, recon_version, recon_salt] = tracker.PreRegisterPeer(0, true);
    18+    assert(!we_initiate_recon);
    


    glozow commented at 10:34 am on December 7, 2021:

    In 76c6e557e8fe843d08b388cd2e541a1f9abed141:

    jw, is there a reason you’re using asserts instead of BOOST_CHECK?

  49. in src/test/txreconciliation_tests.cpp:55 in 76c6e557e8 outdated
    64+    NodeId peer_id0 = 0;
    65+
    66+    // Removing peer after pre-registring works and does not let to register the peer.
    67+    tracker.PreRegisterPeer(peer_id0, true);
    68+    tracker.ForgetPeer(peer_id0);
    69+    assert(!tracker.RegisterPeer(peer_id0, true, true, false, 1, 1));
    


    glozow commented at 10:38 am on December 7, 2021:

    In 76c6e557e8fe843d08b388cd2e541a1f9abed141:

    I’m wondering if there’s a way of checking that the m_local_salts are also properly torn down for peers when we Forget them. If IsPeerRegistered() is used primarily for testing, perhaps it would be more useful to have a “do we know about this peer” function.

  50. naumenkogs commented at 10:39 am on December 7, 2021: member

    Did you think about a -disablerecon command or something similar, to not activate Erlay until all the parts have been merged?

    Yeah I was also thinking about how it’s weird to send SENDRECON messages while they don’t mean anything, but couldn’t come up with any good solution. For starters, I could comment out sending the message at least I guess.

    I guess a flag is better if someone wants to test this stuff via functional tests (without modifying the source code), so yeah, that’s probably an ultimate solution. I’ll add a commit for that.

  51. in test/functional/test_framework/messages.py:1821 in ea3b87af90 outdated
    1813@@ -1814,3 +1814,31 @@ def serialize(self):
    1814     def __repr__(self):
    1815         return "msg_cfcheckpt(filter_type={:#x}, stop_hash={:x})".format(
    1816             self.filter_type, self.stop_hash)
    1817+
    1818+class msg_sendrecon:
    


    glozow commented at 10:43 am on December 7, 2021:

    In ea3b87af90e1649428cabac83012d6faa8df61f5:

    I think it would be helpful to add functional test coverage for sendrecon logic between 2 peers and assert ignore/disconnection/response. Specifically, test cases:

    • peer {1, 2} is {WTXID relay, TXID relay}
    • peer 1 creates a {outbound-full-relay, outbound-block-relay-only, manual} and peer 2 receives inbound connection
    • peer 2 puts fRelay=false
    • peer {1,2} sends sendrecon
    • peer 1 sends sendrecon {before, after} wtxidrelay
    • the sendrecon has version={0,1,2}

    jnewbery commented at 6:22 pm on December 8, 2021:
    I agree. If you’re adding support for these messages to the test framework, then there should be test cases that use those messages.

    mzumsande commented at 8:44 pm on December 8, 2021:
    Agreed, not just for completeness - that would have also caught the issue in #23443 (review) - it is really easy to get the logic wrong in one of the many combinations, leading to erroneous disconnections.
  52. in src/net_processing.cpp:2859 in 4ac0de088d outdated
    2854+        bool they_initiator, they_responder;
    2855+        uint32_t recon_version;
    2856+        uint64_t remote_salt;
    2857+        vRecv >> they_initiator >> they_responder >> recon_version >> remote_salt;
    2858+
    2859+        if (!m_reconciliation.RegisterPeer(pfrom.GetId(), pfrom.IsInboundConn(),
    


    glozow commented at 10:55 am on December 7, 2021:

    In 4ac0de088db9b1d3926daaf4f55dc780a77a7f2c:

    BIP330 sendrecon specification says:

    Sender must set this to 1 currently, otherwise receiver should ignore the message. v1 is the lowest protocol version, everything below that is a protocol violation.

    Shouldn’t we be checking recon_version and return early (ignore) if the version is higher than RECON_VERSION? I suppose it’s not possible to do from net_processing because that constant is inside of txreconciliation module.

    This suggests to me that RegisterPeer() can’t just return a bool, but an enum between {success, protocol violation, skipped for benign reason}, see my other comment about the fact that RegisterPeer() returns false for non-protocol-violation reasons as well #23443 (review)


    naumenkogs commented at 10:38 am on December 21, 2021:

    For this particular comment, I think we should just update the bip. HIgher protocol version shouldn’t be ignored, we just take the minimum between the two and use that. So, will update the bip.

    In general, I hear you about returning from RegisterPeer. Will address that in the context where it needs to be addressed (not in this one).

  53. glozow commented at 11:03 am on December 7, 2021: member
    Concept ACK, thanks for splitting it up. I did a first pass, mostly comparing to the BIP330 specification, looking at test coverage, and a little bit of implementation.
  54. GeneFerneau commented at 8:38 pm on December 7, 2021: none

    utACK ea3b87a

    Code review, everything looks good to me outside nits raised by others.

    Will run the fuzzer for some cycles, and report back results.

  55. in src/node/txreconciliation.h:64 in ea3b87af90 outdated
    59+    std::tuple<bool, bool, uint32_t, uint64_t> PreRegisterPeer(NodeId peer_id, bool peer_inbound);
    60+
    61+    /**
    62+     * Step 0. Once the peer agreed to reconcile with us, generate the state required to track
    63+     * ongoing reconciliations. Should be called only after pre-registering the peer and only once.
    64+     * Does nothing and returns false if the peer violates the protocol.
    


    stickies-v commented at 2:58 pm on December 8, 2021:

    nit: suggested improvement for readability

    0     * If the peer violates the protocol, no state is changed and false is returned.
    
  56. in src/node/txreconciliation.cpp:28 in ea3b87af90 outdated
    23+    // According to BIP-330, salts should be combined in ascending order.
    24+    uint64_t salt1 = local_salt, salt2 = remote_salt;
    25+    if (salt1 > salt2) std::swap(salt1, salt2);
    26+
    27+    static const auto RECON_SALT_HASHER = TaggedHash(RECON_STATIC_SALT);
    28+    return (CHashWriter(RECON_SALT_HASHER) << salt1 << salt2).GetSHA256();
    


    stickies-v commented at 3:21 pm on December 8, 2021:

    I think you can avoid the copy constructor by making RECON_SALT_HASHER not const?

    0    static auto RECON_SALT_HASHER = TaggedHash(RECON_STATIC_SALT);
    1    return (RECON_SALT_HASHER << salt1 << salt2).GetSHA256();
    

    naumenkogs commented at 12:30 pm on December 9, 2021:
    Is this an improvement though?

    stickies-v commented at 12:39 pm on December 9, 2021:
    Well I’m just not sure why you would call the CHashWriter constructor in the first place when RECON_SALT_HASHER already is a CHashWriter? I don’t see the point, hence my suggestion to leave it out for simplicity. If you’re worried about visual feedback re type, I would suggest replacing auto with CHashWriter.
  57. in src/node/txreconciliation.cpp:114 in ea3b87af90 outdated
    109+        // We do not support reconciliation salt/version updates. We treat an attempt to update
    110+        // after a successful registration as a protocol violation.
    111+        LOCK(m_mutex);
    112+        if (m_states.find(peer_id) != m_states.end()) return false;
    113+
    114+        // If the peer supports the version which is lower than our, we downgrade to the version
    


    stickies-v commented at 3:53 pm on December 8, 2021:

    typo:

    0        // If the peer supports the version which is lower than ours, we downgrade to the version
    
  58. in src/node/txreconciliation.cpp:132 in ea3b87af90 outdated
    127+
    128+        // Must match SuggestReconciling logic.
    129+        bool we_may_initiate = !peer_inbound, we_may_respond = peer_inbound;
    130+
    131+        bool they_initiate = they_may_initiate && we_may_respond;
    132+        bool we_initiate = we_may_initiate && they_may_respond;
    


    stickies-v commented at 4:08 pm on December 8, 2021:

    nit: I’m all in favour for using helpful variable names. In this case, I’m not convinced it makes the code easier to understand. Personally, I find the below easier to understand and keep in memory:

    0        bool they_initiate = they_may_initiate && peer_inbound;
    1        bool we_initiate = !peer_inbound && they_may_respond;
    

    stickies-v commented at 6:48 pm on December 13, 2021:
    Thanks for changing this @naumenkogs. There is still a reference to we_may_respond in the comments that you probably want to take out too?
  59. in src/node/txreconciliation.cpp:128 in ea3b87af90 outdated
    133+        // If we ever announce we_initiate && we_may_respond, this will need tie-breaking. For now,
    134+        // this is mutually exclusive because both are based on the inbound flag.
    135+        assert(!(they_initiate && we_initiate));
    136+
    137+        // The peer set both flags to false, we treat it as a protocol violation.
    138+        if (!(they_initiate || we_initiate)) return false;
    


    stickies-v commented at 4:09 pm on December 8, 2021:
    Since the peer can trigger both paths, I think it would be prudent to add a log entry in both of these cases?
  60. in src/node/txreconciliation.cpp:88 in b55cbf63e1 outdated
    47+        bool added = WITH_LOCK(m_mutex, return m_local_salts.emplace(peer_id, m_local_recon_salt).second);
    48+        // We do this exactly once per peer (which are unique by NodeId, see GetNewNodeId) so it's
    49+        // safe to assume we don't have this record yet.
    50+        assert(added);
    51+
    52+        LogPrint(BCLog::NET, "Pre-register peer=%d for reconciling.\n", peer_id);
    


    stickies-v commented at 4:12 pm on December 8, 2021:
    This log should probably come before any state is changed? So I’d move it up to before calling m_local_salts.emplace
  61. in src/node/txreconciliation.h:39 in ea3b87af90 outdated
    34+ * This is a modification of the Erlay protocol (https://arxiv.org/abs/1905.10518) with two
    35+ * changes (sketch extensions instead of bisections, and an extra INV exchange round), both
    36+ * are motivated in BIP-330.
    37+ */
    38+class TxReconciliationTracker {
    39+    // Avoid littering this header file with implementation details.
    


    jnewbery commented at 4:13 pm on December 8, 2021:
    This comment is unnecessary. No need to add comments about language features or common idioms like pimpl.
  62. in src/node/txreconciliation.h:43 in ea3b87af90 outdated
    38+class TxReconciliationTracker {
    39+    // Avoid littering this header file with implementation details.
    40+    class Impl;
    41+    const std::unique_ptr<Impl> m_impl;
    42+
    43+    public:
    


    jnewbery commented at 4:13 pm on December 8, 2021:
    0public:
    
  63. in src/node/txreconciliation.h:45 in ea3b87af90 outdated
    40+    class Impl;
    41+    const std::unique_ptr<Impl> m_impl;
    42+
    43+    public:
    44+
    45+    explicit TxReconciliationTracker();
    


    jnewbery commented at 4:15 pm on December 8, 2021:
    explicit prevents implicit conversions in the ctor arguments. This ctor has no arguments, so the explicit doesn’t do anything. I’d suggest removing it.
  64. in src/node/txreconciliation.cpp:38 in ea3b87af90 outdated
    33+ * peer when next transaction reconciliation happens, and also all parameters required to perform
    34+ * reconciliations.
    35+ */
    36+class ReconciliationState {
    37+
    38+    public:
    


    jnewbery commented at 4:22 pm on December 8, 2021:
    0public:
    
  65. in src/node/txreconciliation.cpp:5 in ea3b87af90 outdated
    0@@ -0,0 +1,192 @@
    1+// Copyright (c) 2021 The Bitcoin Core developers
    2+// Distributed under the MIT software license, see the accompanying
    3+// file COPYING or http://www.opensource.org/licenses/mit-license.php.
    4+
    5+#include <node/txreconciliation.h>
    


    jnewbery commented at 4:29 pm on December 8, 2021:
    There are quite a lot of modules that are used but not included, such as sync.h (for Mutex) and cstdint (for uint64_t, etc).

    glozow commented at 0:43 am on April 23, 2022:
    include-what-you-use might help? see #24831
  66. in src/node/txreconciliation.cpp:103 in ea3b87af90 outdated
    107+        bool they_may_initiate, bool they_may_respond, uint32_t recon_version, uint64_t remote_salt)
    108+    {
    109+        // We do not support reconciliation salt/version updates. We treat an attempt to update
    110+        // after a successful registration as a protocol violation.
    111+        LOCK(m_mutex);
    112+        if (m_states.find(peer_id) != m_states.end()) return false;
    


    stickies-v commented at 4:30 pm on December 8, 2021:
    Since 103e52dcd316a5b3e939f3880da566430820e567, I think this can be replace with IsPeerRegistered?

    naumenkogs commented at 2:05 pm on December 10, 2021:
    I thought it’s better to use internal access internally. Loose coupling & high cohesion i guess.

    stickies-v commented at 7:42 pm on December 13, 2021:
    I don’t see how this helps with reducing coupling/increasing cohesion, when IsPeerRegistered is a public member of the same class? This just looks like a regular case of DRY imo. Plus, readability improves.

    naumenkogs commented at 9:17 am on December 17, 2021:
    Probably you are right
  67. in src/node/txreconciliation.cpp:42 in ea3b87af90 outdated
    37+
    38+    public:
    39+
    40+    /**
    41+     * Reconciliation involves exchanging sketches, which efficiently represent transactions each
    42+     * peer wants to announce. Sketches are computed over transaction short IDs.
    


    jnewbery commented at 4:31 pm on December 8, 2021:
    This seems like too high-level commentary on the salt ids.
  68. in src/node/txreconciliation.cpp:88 in ea3b87af90 outdated
     97+        bool added = WITH_LOCK(m_mutex, return m_local_salts.emplace(peer_id, m_local_recon_salt).second);
     98+        // We do this exactly once per peer (which are unique by NodeId, see GetNewNodeId) so it's
     99+        // safe to assume we don't have this record yet.
    100+        assert(added);
    101+
    102+        LogPrint(BCLog::NET, "Pre-register peer=%d for reconciling.\n", peer_id);
    


    jnewbery commented at 4:32 pm on December 8, 2021:
    What do you think about adding a dedicated BCLog::ERLAY log category?
  69. in src/node/txreconciliation.cpp:94 in ea3b87af90 outdated
    89+            we_initiate_recon = false;
    90+            we_respond_recon = true;
    91+        } else {
    92+            we_initiate_recon = true;
    93+            we_respond_recon = false;
    94+        }
    


    jnewbery commented at 4:38 pm on December 8, 2021:

    This seems unnecessary. The caller passes a bool argument peer_inbound into the function:

    • if peer_inbound is true, the function returns <false, true, ...>
    • if peer_inbound is false, the function returns <true, false, ...>

    and peer_inbound is not used for anything else. I suggest just removing the argument and return values.

  70. in src/node/txreconciliation.cpp:103 in ea3b87af90 outdated
     98+        // We do this exactly once per peer (which are unique by NodeId, see GetNewNodeId) so it's
     99+        // safe to assume we don't have this record yet.
    100+        assert(added);
    101+
    102+        LogPrint(BCLog::NET, "Pre-register peer=%d for reconciling.\n", peer_id);
    103+        return std::make_tuple(we_initiate_recon, we_respond_recon, RECON_VERSION, m_local_recon_salt);
    


    jnewbery commented at 4:41 pm on December 8, 2021:
    Seems strange to return a constant in every function call. Perhaps PeerManager can construct TxReconciliationTracker with the recon_version as a parameter, and always be aware of what version reconciliation tracker it owns?
  71. stickies-v commented at 4:56 pm on December 8, 2021: contributor

    Approach ACK, code review until 103e52dcd316a5b3e939f3880da566430820e567 - didn’t look at tests yet.

    This PR introduces the logic to register newly connected peers as peers that support transaction reconciliation, and initialize the necessary data structures required to do the reconciliation later on.

    I think everything is well laid out and implemented, so most of my comments are quite nitty and focus on readability etc. They’re subjective, feel free to ignore.

    Some general thoughts/suggestions:

    • “SENDRECON” does not hint that it corresponds to transactions, would it make sense to rename this to “SENDTXRECON” so people can quickly understand what it’s about without know the Erlay code?
    • I understand you’re renaming the two roles to “initiator” & “responder”, there are still quite a few instances of “requestor”, “sender”, “receiver” throughout the code and in BIP300. Would suggest to replace them all in both documentation and variable names, consistent naming makes it much easier to follow.
  72. in src/node/txreconciliation.cpp:67 in ea3b87af90 outdated
    68+     * construct reconciliation sketches.
    69+     * Salt is generated randomly per-peer to prevent:
    70+     * - linking of network nodes belonging to the same physical node
    71+     * - halting of relay of particular transactions due to short ID collisions (DoS)
    72+     */
    73+    std::unordered_map<NodeId, uint64_t> m_local_salts GUARDED_BY(m_mutex);
    


    jnewbery commented at 4:58 pm on December 8, 2021:
    I don’t think it makes sense to store two maps from NodeId, where m_local_salts is only really needed between the PreRegister and Register steps. Why not add a uint64_t member to ReconciliationState that contains the local salt?

    naumenkogs commented at 12:42 pm on December 10, 2021:
    1. A new member would sit there with no use most of the ReconciliationState lifetime
    2. ReconciliationState would be initialized but not ready to use for reconciliations (while there is no full salt)

    I thought the latter might be particularly confusing in terms of understanding the protocol flow


    naumenkogs commented at 2:12 pm on December 21, 2021:

    I started integrating m_local_salt into ReconciliationState, and realized that I’m not sure what’s the best way to check the state of the peer (pre-registered or registered)?

    1. A new m_registrered flag in ReconciliationState
    2. Make m_local_salt in ReconciliationState optional, and if it’s nullptr – means it’s registered already (more implicit approach)
    3. Something even crazier is having std::variant <std::pair<NodeId, uint64_t>, ReconciliationState> m_states, so that the value could be either a node<->salt mapping (pre-registered) and ReconciliationState (registered) @glozow do you have an opinion?

    glozow commented at 10:18 am on January 4, 2022:
    I agree with having the salt inside of ReconciliationState, and it seems like (2) would be most idiomatic - std::optional<ReconciliationState> where nullopt means not registered yet.
  73. in src/net_processing.cpp:2833 in ea3b87af90 outdated
    2824@@ -2810,6 +2825,49 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    2825         return;
    2826     }
    2827 
    2828+    // Received from a peer demonstrating readiness to announce transactions via reconciliations.
    2829+    // This feature negotiation should happen:
    2830+    // - between VERSION and VERACK to avoid relay problems from switching announcement protocols
    2831+    //   after the connection is up
    2832+    // - after WTXID because we reconcile only with peers supporting wtxid-relay
    


    jnewbery commented at 5:57 pm on December 8, 2021:
    I’m not sure I like this restriction. As far as I’m aware, this is the first feature negotiation message that has a strict ordering dependency on a different feature negotiation message.

    naumenkogs commented at 12:19 pm on December 10, 2021:

    I think you are right, this is the first time. I assume that we still want to have a dependency on WTXID, but don’t have ordering between WTXID and SENDRECON.

    If it’s WTXID and then SENDRECON it’s clear (same as now), but what if SENDRECON comes first? It’s kinda too early to “Register”, but also we need to remember we received this message once we receive WTXID and can register. I see the following options:

    1. Optimistically register at the receiving of SENDRECON (even if came first). Then, unregister if at VERACK we haven’t received wtxid.
    2. Track receiving SENDRECON in a yet-another-variable like, and register only when both are set (could be also at verack, or not).

    I think (1) is better.

  74. in src/node/txreconciliation.h:69 in ea3b87af90 outdated
    64+     * Does nothing and returns false if the peer violates the protocol.
    65+     */
    66+    bool RegisterPeer(NodeId peer_id, bool peer_inbound,
    67+        bool recon_requestor, bool recon_responder, uint32_t recon_version, uint64_t remote_salt);
    68+
    69+    // Helpers
    


    jnewbery commented at 6:01 pm on December 8, 2021:
    What makes this a “helper”?
  75. in src/node/txreconciliation.cpp:25 in ea3b87af90 outdated
    20+ */
    21+static uint256 ComputeSalt(uint64_t local_salt, uint64_t remote_salt)
    22+{
    23+    // According to BIP-330, salts should be combined in ascending order.
    24+    uint64_t salt1 = local_salt, salt2 = remote_salt;
    25+    if (salt1 > salt2) std::swap(salt1, salt2);
    


    jnewbery commented at 6:04 pm on December 8, 2021:

    I think it’s best to avoid multiple declarations on one line.

    Also, I think these local variables can be avoided with:

    0    // According to BIP-330, salts should be combined in ascending order.
    1    return (CHashWriter(RECON_SALT_HASHER) << std::min(salt1, salt2) << std::max(salt1, salt2)).GetSHA256();
    
  76. in src/node/txreconciliation.cpp:21 in ea3b87af90 outdated
    16+/**
    17+ * Salt (specified by BIP-330) constructed from contributions from both peers. It is used
    18+ * to compute transaction short IDs, which are then used to construct a sketch representing a set
    19+ * of transactions we want to announce to the peer.
    20+ */
    21+static uint256 ComputeSalt(uint64_t local_salt, uint64_t remote_salt)
    


    jnewbery commented at 6:05 pm on December 8, 2021:
    Up to you, but static can be omitted since this is in the unnamed namespace.
  77. in src/net_processing.cpp:3042 in ea3b87af90 outdated
    2852+            LogPrint(BCLog::NET, "sendrecon received before wtxidrelay peer=%d; disconnecting\n", pfrom.GetId());
    2853+            pfrom.fDisconnect = true;
    2854+            return;
    2855+        }
    2856+
    2857+        bool they_initiator, they_responder;
    


    jnewbery commented at 6:07 pm on December 8, 2021:
    Prefer not to do multiple declarations on one line.
  78. in src/node/txreconciliation.h:67 in ea3b87af90 outdated
    62+     * Step 0. Once the peer agreed to reconcile with us, generate the state required to track
    63+     * ongoing reconciliations. Should be called only after pre-registering the peer and only once.
    64+     * Does nothing and returns false if the peer violates the protocol.
    65+     */
    66+    bool RegisterPeer(NodeId peer_id, bool peer_inbound,
    67+        bool recon_requestor, bool recon_responder, uint32_t recon_version, uint64_t remote_salt);
    


    jnewbery commented at 6:11 pm on December 8, 2021:
    nit: Do you mind using AlignAfterOpenBracket as specified in the clang-format file? (here and elsewhere)
  79. in src/net_processing.cpp:2864 in ea3b87af90 outdated
    2862+        if (!m_reconciliation.RegisterPeer(pfrom.GetId(), pfrom.IsInboundConn(),
    2863+            they_initiator, they_responder, recon_version, remote_salt)) {
    2864+                LogPrint(BCLog::NET, "reconciliation protocol violation from peer=%d; disconnecting\n", pfrom.GetId());
    2865+                pfrom.fDisconnect = true;
    2866+                return;
    2867+            }
    


    jnewbery commented at 6:11 pm on December 8, 2021:
    This is over-indented
  80. in src/node/txreconciliation.cpp:129 in ea3b87af90 outdated
    124+
    125+        // This function should be called only after generating the local salt.
    126+        if (local_salt == m_local_salts.end()) return false;
    127+
    128+        // Must match SuggestReconciling logic.
    129+        bool we_may_initiate = !peer_inbound, we_may_respond = peer_inbound;
    


    jnewbery commented at 6:16 pm on December 8, 2021:
    This is unnecessarily difficult to read. Please don’t put multiple declarations/assignments on the same line.
  81. in src/net_processing.cpp:1225 in ea3b87af90 outdated
    1219@@ -1218,6 +1220,10 @@ void PeerManagerImpl::FinalizeNode(const CNode& node)
    1220     }
    1221     WITH_LOCK(g_cs_orphans, m_orphanage.EraseForPeer(nodeid));
    1222     m_txrequest.DisconnectedPeer(nodeid);
    1223+    // Do not check whether peer is registered for reconciliation here, but rather delegate checks
    1224+    // to the module. Otherwise it's easy to skip deleting an intermediate state (e.g., we store
    1225+    // salt for peers before we register them for reconciliation).
    


    jnewbery commented at 6:20 pm on December 8, 2021:
    This comment seems unnecessary. We’re simply telling the TxReconciliationTracker to clear all state for this peer. That seems straightforward enough to not require a comment (similarly for the orphanage and txrequest modules).
  82. jnewbery commented at 6:23 pm on December 8, 2021: contributor

    Did you think about a -disablerecon command or something similar, to not activate Erlay until all the parts have been merged?

    I agree that this should be added, although to bikeshed I’d call it -erlay and default to false.

  83. naumenkogs force-pushed on Dec 9, 2021
  84. naumenkogs force-pushed on Dec 10, 2021
  85. naumenkogs commented at 2:14 pm on December 10, 2021: member
    Still going through comments, beware it’s not the final shape from my side.
  86. in src/test/txreconciliation_tests.cpp:11 in 07b9357c8a outdated
     6+
     7+#include <test/util/setup_common.h>
     8+
     9+#include <boost/test/unit_test.hpp>
    10+
    11+BOOST_FIXTURE_TEST_SUITE(txreconciliation_tests, BasicTestingSetup)
    


    maflcko commented at 2:15 pm on December 10, 2021:
    is this file even compiled? I can’t find it in the makefile
  87. naumenkogs force-pushed on Dec 13, 2021
  88. naumenkogs force-pushed on Dec 17, 2021
  89. naumenkogs force-pushed on Dec 17, 2021
  90. bitcoin deleted a comment on Jan 22, 2022
  91. naumenkogs force-pushed on Jan 26, 2022
  92. naumenkogs commented at 4:22 pm on January 26, 2022: member

    Rebased and addressed most of the comments.

    The only remaining thing is to make functional tests for this. I’m working on that.

  93. naumenkogs force-pushed on Jan 26, 2022
  94. naumenkogs force-pushed on Feb 4, 2022
  95. naumenkogs force-pushed on Feb 4, 2022
  96. naumenkogs force-pushed on Feb 5, 2022
  97. naumenkogs force-pushed on Feb 7, 2022
  98. in test/functional/p2p_sendrecon.py:77 in d3cc218415 outdated
    66+        assert_equal(peer.sendrecon_msg_received.version, 1)
    67+
    68+        self.log.info('SENDRECON should not be sent if block-relay-only')
    69+        peer = self.nodes[0].add_outbound_p2p_connection(SendReconSender(), p2p_idx=2, connection_type="block-relay-only")
    70+        time.sleep(1)
    71+        assert not peer.sendrecon_msg_received
    


    glozow commented at 12:57 pm on February 17, 2022:
    you want to sync_with_ping() here, and then assert not received. otherwise the message could be unreceived due to race condition

    naumenkogs commented at 10:44 am on February 23, 2022:
    This doesn’t work prior to verack? Where it’s possible, i add it. Here, i can increase a timer… or what would you suggest?

    glozow commented at 3:10 pm on February 23, 2022:

    ah my bad! well, could you wait until verack handshake is done, and then assert that sendrecon was never recieved?

    0peer.wait_for_verack()
    1assert not peer.sendrecon_msg_received
    
  99. in test/functional/p2p_sendrecon.py:94 in d3cc218415 outdated
    89+        time.sleep(1)
    90+        assert peer.is_connected
    91+        self.log.info('second SENDRECON triggers a disconnect')
    92+        peer.send_message(create_sendrecon_msg())
    93+        time.sleep(1)
    94+        assert not peer.is_connected
    


    glozow commented at 12:58 pm on February 17, 2022:

    you can use wait_for_disconnect(). sleeping is insufficient; it could take longer to disconnect especially if other functional tests are being run in parallel

    (same with the rest of the disconnection tests)

  100. in src/test/txreconciliation_tests.cpp:22 in d3cc218415 outdated
    17+
    18+    // Prepare a peer for reconciliation.
    19+    tracker.PreRegisterPeer(0);
    20+
    21+    // Both roles are false, don't register.
    22+    assert(!tracker.RegisterPeer(0, true, false, false, 1, salt).value());
    


    glozow commented at 12:59 pm on February 17, 2022:
    use BOOST_CHECK()

    glozow commented at 12:15 pm on February 18, 2022:

    in 38cd00572de8540138eb0140553e9ab669b457b2:

    nit: Maybe add some argument labels to make a future dev’s life a bit easier? Even just commenting the first one would be helpful.

    0    assert(!tracker.RegisterPeer(/*peer_id=*/ 0, /*peer_inbound=*/ true, /*they_may_initiate=*/ false, /*they_may_respond=*/ false, /*peer_recon_version=*/ 1, salt).value());
    
  101. glozow commented at 12:59 pm on February 17, 2022: member
    Thanks for adding the functional tests! A few initial comments below.
  102. in src/logging.h:65 in 812ea2a7dd outdated
    61@@ -62,6 +62,7 @@ namespace BCLog {
    62         LOCK        = (1 << 24),
    63         UTIL        = (1 << 25),
    64         BLOCKSTORE  = (1 << 26),
    65+        TXRECON  = (1 << 27),
    


    glozow commented at 11:56 am on February 18, 2022:

    in 812ea2a7ddc71ed0ea7602b278a5b2d1a53b4101

    nit: misaligned

  103. in src/init.cpp:472 in d53b4bc036 outdated
    468@@ -469,6 +469,7 @@ void SetupServerArgs(ArgsManager& argsman)
    469     argsman.AddArg("-seednode=<ip>", "Connect to a node to retrieve peer addresses, and disconnect. This option can be specified multiple times to connect to multiple nodes.", ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    470     argsman.AddArg("-networkactive", "Enable all P2P network activity (default: 1). Can be changed by the setnetworkactive RPC command", ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    471     argsman.AddArg("-timeout=<n>", strprintf("Specify socket connection timeout in milliseconds. If an initial attempt to connect is unsuccessful after this amount of time, drop it (minimum: 1, default: %d)", DEFAULT_CONNECT_TIMEOUT), ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    472+    argsman.AddArg("-txrecon", strprintf("Enable transaction reconciliations per BIP-330 (default: %d)", DEFAULT_TXRECON_ENABLE), ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    


    glozow commented at 11:59 am on February 18, 2022:

    in d53b4bc0360760737b250b81e82c4d8e6e6b1144:

    0    argsman.AddArg("-txrecon", strprintf("Enable transaction reconciliations per BIP-330 (default: %d)", DEFAULT_TXRECON_ENABLE), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::CONNECTION);
    

    I like this approach of using -txrecon config to turn it on. Imo we should make it debug-only as well, otherwise users might be confused and think their node can use Erlay even though the rest of the functionality isn’t there yet?

  104. in src/net_processing.cpp:3425 in 77d1aaab3e outdated
    2845+            // registered upon receiving SENDRECON afterwards.
    2846+            // Now we are sure they won't announce WTXIDRELAY (can't happen after VERACK).
    2847+            // We should clear the reconciliation state of the peer (because we support
    2848+            // reconciliations only based on wtxids).
    2849+            m_reconciliation.ForgetPeer(peer_id);
    2850+        }
    


    glozow commented at 12:12 pm on February 18, 2022:

    in 77d1aaab3e62aae2319860049781f4e9f0087521:

    I like that there is no longer an ordering requirement between wtxidrelay and sendrecon :+1:

    Question: shouldn’t there be a gate for whether or not we’re doing reconciliations at all?

  105. in test/functional/p2p_sendrecon.py:36 in d3cc218415 outdated
    31+        if not self.manual_verack:
    32+            super().on_version(message)
    33+        self.version_received = True
    34+
    35+    def on_sendrecon(self, message):
    36+        print(message)
    


    glozow commented at 12:17 pm on February 18, 2022:

    in d3cc218415f3a5f9a7170cda48a5e3938900832c:

    Remove this print? Or at least use self.log

  106. in test/functional/p2p_sendrecon.py:141 in d3cc218415 outdated
    137+        peer.wait_for_verack()
    138+        peer.send_message(msg_verack())
    139+        peer.send_message(create_sendrecon_msg())
    140+        time.sleep(1)
    141+        assert not peer.is_connected
    142+
    


    glozow commented at 12:24 pm on February 18, 2022:

    in d3cc218415f3a5f9a7170cda48a5e3938900832c:

    Nice, I like the new tests. Are you missing a test case for when the peer sends more than 1 sendrecon ?


    naumenkogs commented at 12:21 pm on February 23, 2022:
    self.log.info('second SENDRECON triggers a disconnect'), it’s there

    glozow commented at 2:19 pm on February 23, 2022:
    oops, thanks!
  107. in src/net_processing.cpp:431 in d53b4bc036 outdated
    427@@ -425,6 +428,7 @@ class PeerManagerImpl final : public PeerManager
    428     ChainstateManager& m_chainman;
    429     CTxMemPool& m_mempool;
    430     TxRequestTracker m_txrequest GUARDED_BY(::cs_main);
    431+    TxReconciliationTracker m_reconciliation;
    


    glozow commented at 12:32 pm on February 18, 2022:

    in d53b4bc0360760737b250b81e82c4d8e6e6b1144:

    Perhaps it would make more sense to have PeerManagerImpl keep a std::unique_ptr<TxReconciliationTracker> and only initialize the recon tracker if -txrecon is on? And further, check whether m_reconciliation points to something to figure out whether we have txrecon enabled, instead of querying the ArgsManager every time.

  108. in src/net_processing.cpp:177 in d53b4bc036 outdated
    172@@ -172,6 +173,8 @@ static constexpr double MAX_ADDR_RATE_PER_SECOND{0.1};
    173  *  based increments won't go above this, but the MAX_ADDR_TO_SEND increment following GETADDR
    174  *  is exempt from this limit). */
    175 static constexpr size_t MAX_ADDR_PROCESSING_TOKEN_BUCKET{MAX_ADDR_TO_SEND};
    176+/** Supported transaction reconciliation protocol version */
    177+static constexpr uint32_t RECON_VERSION = 1;
    


    glozow commented at 12:33 pm on February 18, 2022:

    in d53b4bc0360760737b250b81e82c4d8e6e6b1144:

    nit

    0static constexpr uint32_t RECON_VERSION{1};
    
  109. in src/protocol.h:269 in d3cc218415 outdated
    261@@ -262,6 +262,13 @@ extern const char* CFCHECKPT;
    262  * @since protocol version 70016 as described by BIP 339.
    263  */
    264 extern const char* WTXIDRELAY;
    265+/**
    266+ * Contains 2 1-byte bools, a 4-byte version number and an 8-byte salt.
    267+ * The 2 booleans indicate that a node is willing to participate in transaction
    268+ * reconciliation, respectively as an initiator or as a receiver.
    269+ * The salt is used to compute short txids needed for efficient reconciliation.
    


    glozow commented at 12:35 pm on February 18, 2022:
    Should mention BIP330 here
  110. in src/node/txreconciliation.h:72 in d3cc218415 outdated
    67+    std::optional<bool> RegisterPeer(NodeId peer_id, bool peer_inbound, bool recon_requestor,
    68+                                     bool recon_responder, uint32_t peer_recon_version, uint64_t remote_salt);
    69+
    70+    /**
    71+     * Attempts to forget reconciliation-related state of the peer (if we previously stored any).
    72+     * After this, we won't be able to reconcile with the peer.
    


    glozow commented at 12:42 pm on February 18, 2022:

    in 182d984136066fd4f09b3e1a1bbd9b50c6cb87be

    Nice doxygen comments. Can you specify here whether it’s safe/not to call ForgetPeer multiple times for the same peer?


    naumenkogs commented at 11:05 am on February 23, 2022:
    What makes this function different from others, for which we don’t mention this?

    glozow commented at 6:45 pm on April 18, 2022:
    It’s not different, I think all functions should be well-documented. Just thought this would be good information to include, but it’s fine if it isn’t.
  111. glozow commented at 12:48 pm on February 18, 2022: member
    Looking a lot better
  112. naumenkogs force-pushed on Feb 23, 2022
  113. in src/net_processing.cpp:1256 in 88e5b0f8ad outdated
    1252@@ -1251,6 +1253,7 @@ void PeerManagerImpl::FinalizeNode(const CNode& node)
    1253     }
    1254     WITH_LOCK(g_cs_orphans, m_orphanage.EraseForPeer(nodeid));
    1255     m_txrequest.DisconnectedPeer(nodeid);
    1256+    m_txreconciliation->ForgetPeer(nodeid);
    


    glozow commented at 2:06 pm on February 23, 2022:

    Seems like CI is failing on this:

    0    if (m_txreconciliation) m_txreconciliation->ForgetPeer(nodeid);
    
  114. naumenkogs force-pushed on Mar 10, 2022
  115. in src/protocol.cpp:47 in 8da53ba582 outdated
    43@@ -44,7 +44,7 @@ const char *CFHEADERS="cfheaders";
    44 const char *GETCFCHECKPT="getcfcheckpt";
    45 const char *CFCHECKPT="cfcheckpt";
    46 const char *WTXIDRELAY="wtxidrelay";
    47-const char *SENDRECON = "sendrecon";
    48+const char* SENDRECON = "sendrecon";
    


    dergoegge commented at 7:18 pm on March 11, 2022:

    In 8da53ba5824fc7e367dce07dc14da6b8c0916f0a

    nit: This looks like an accidental change

  116. in src/node/txreconciliation.cpp:67 in bc9f223c35 outdated
    51@@ -37,6 +52,9 @@ class TxReconciliationTracker::Impl
    52 {
    53     mutable Mutex m_mutex;
    54 
    55+    // Local protocol version
    56+    uint32_t m_recon_version;
    


    dergoegge commented at 7:25 pm on March 11, 2022:

    In bc9f223c359f3fddd3d66b4b0cecd2fbda31778b

    I don’t see the need to store the version on the TxReconciliationTracker since the only value this will ever have is the same as RECON_VERSION.

    nit: If you want to keep this i would suggest doing this in the commit (0b7fb42952f2e41812648d305139279b3ccae629) that initially adds the TxReconciliationTracker, since m_recon_version is not set based on what a peer sent to us.

    0    const uint32_t m_recon_version;
    
  117. in src/net_processing.cpp:2661 in ab27d1d51b outdated
    2650@@ -2645,6 +2651,23 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    2651 
    2652         if (greatest_common_version >= WTXID_RELAY_VERSION) {
    2653             m_connman.PushMessage(&pfrom, msg_maker.Make(NetMsgType::WTXIDRELAY));
    2654+
    2655+            if (m_txreconciliation) {
    2656+                // Reconciliation is supported only when wtxid relay is supported for only
    2657+                // those connections which (at least might) support transaction relay.
    2658+                if (pfrom.m_tx_relay && !m_ignore_incoming_txs) {
    


    ariard commented at 10:39 pm on March 22, 2022:
    nit: Alternatively clearer imo “If the connection is marked as block-relay-only or we’re running as block-only, reconciliation is not supported”.

    naumenkogs commented at 11:49 am on March 28, 2022:
    I agree the comment is too complicated. I will simplify it, but in a different way.
  118. in src/net_processing.cpp:2858 in ab27d1d51b outdated
    2850+                // We could have optimistically pre-registered the peer on sending SENDRECON, or
    2851+                // registered upon receiving SENDRECON afterwards.
    2852+                // Now we are sure they won't announce WTXIDRELAY (can't happen after VERACK).
    2853+                // We should clear the reconciliation state of the peer (because we support
    2854+                // reconciliations only based on wtxids).
    2855+                m_txreconciliation->ForgetPeer(peer_id);
    


    ariard commented at 11:04 pm on March 22, 2022:
    Note the BIP still says “Should be ignored if received before receiving “wtxidrelay” from the peer”. I think spec should be aligned on code to reflect the reception order dependency has been removed.
  119. in src/net_processing.cpp:3015 in ab27d1d51b outdated
    2925@@ -2889,6 +2926,49 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    2926         return;
    2927     }
    2928 
    2929+    // Received from a peer demonstrating readiness to announce transactions via reconciliations.
    2930+    // This feature negotiation should happen: between VERSION and VERACK to avoid relay problems
    


    ariard commented at 11:11 pm on March 22, 2022:
    Could be reflected in the BIP that SENDRECON must be sent before VERACK.
  120. in src/node/txreconciliation.cpp:1 in ab27d1d51b outdated
    0@@ -0,0 +1,167 @@
    1+// Copyright (c) 2021 The Bitcoin Core developers
    


    ariard commented at 11:21 pm on March 22, 2022:
    I’ve been told we’re in 2022 in legacy time.
  121. in src/node/txreconciliation.cpp:61 in ab27d1d51b outdated
    56+    uint32_t m_recon_version;
    57+
    58+    /**
    59+     * Keeps track of reconciliation states of eligible peers.
    60+     * For pre-registered peers, the locally generated salt is stored.
    61+     * For registered peers, the salt is forgotten and the state is stored.
    


    ariard commented at 11:26 pm on March 22, 2022:
    I think the locally generated salt is forgotten though the full salt is made part of the reconciliation state.
  122. in test/functional/p2p_sendrecon.py:140 in ab27d1d51b outdated
    129+        self.log.info('SENDRECON after VERACK triggers a disconnect')
    130+        peer = self.nodes[0].add_p2p_connection(SendReconSender(manual_verack=True), send_version=True, wait_for_verack=False)
    131+        peer.wait_for_verack()
    132+        peer.send_message(msg_verack())
    133+        peer.send_message(create_sendrecon_msg())
    134+        peer.wait_for_disconnect()
    


    ariard commented at 11:39 pm on March 22, 2022:
    Do you have coverage for the case the peer does not send a WTXIDRELAY before VERACK, thus forgetting the reconciliation registration (L2855, in src/net_processing.cpp) ?
  123. ariard commented at 11:40 pm on March 22, 2022: member
    Reviewed both code/test coverage.
  124. naumenkogs force-pushed on Mar 28, 2022
  125. naumenkogs force-pushed on Mar 29, 2022
  126. in test/functional/p2p_sendrecon.py:105 in 0ed41c3761 outdated
     95+        sendrecon_no_role = create_sendrecon_msg()
     96+        sendrecon_no_role.initiator = False
     97+        sendrecon_no_role.responder = False
     98+        peer = self.nodes[0].add_p2p_connection(
     99+            SendReconSender(sendrecon_msg_to_send=sendrecon_no_role),
    100+            send_version=True, wait_for_verack=False)
    


    glozow commented at 2:43 pm on March 29, 2022:
    CI fails here, my guess is it’s getting disconnected for the sendrecon violation before the test framework checks peer.is_connected?
  127. naumenkogs force-pushed on Mar 30, 2022
  128. naumenkogs force-pushed on Mar 30, 2022
  129. DrahtBot added the label Needs rebase on Apr 8, 2022
  130. naumenkogs force-pushed on Apr 10, 2022
  131. DrahtBot removed the label Needs rebase on Apr 10, 2022
  132. in src/net_processing.cpp:3496 in b4457577fc outdated
    2977+    // from switching announcement protocols after the connection is up.
    2978+    if (msg_type == NetMsgType::SENDRECON) {
    2979+        // For now, transaction reconciliation is an experimental feature that could be
    2980+        // enabled via CLI.
    2981+        if (!m_txreconciliation) {
    2982+            return;
    


    ariard commented at 8:32 pm on April 10, 2022:
    To ease debugging, a LogPrint can be added to inform the operator than lack of reconciliation is due to the node config.
  133. in src/net_processing.cpp:2895 in b4457577fc outdated
    2889@@ -2864,6 +2890,19 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    2890             nCMPCTBLOCKVersion = 1;
    2891             m_connman.PushMessage(&pfrom, msgMaker.Make(NetMsgType::SENDCMPCT, fAnnounceUsingCMPCTBLOCK, nCMPCTBLOCKVersion));
    2892         }
    2893+
    2894+        if (m_txreconciliation) {
    2895+            LOCK(cs_main);
    


    ariard commented at 8:38 pm on April 10, 2022:

    What’s the purpose of the lock tacking here ?

    From what I understand, TxReconciliationTracker’s m_states is protected by m_mutex and m_wtxid_relay other callsites doesn’t seem grieved by a lock tacking (note it might be under-sights there).

  134. naumenkogs force-pushed on Apr 12, 2022
  135. maflcko commented at 7:46 am on April 13, 2022: member

    There is a tidy error, which can be fixed by either removing the = or by adjusting the names.

     0test/txreconciliation_tests.cpp:23:39: error: argument name 'they_may_initiate' in comment does not match parameter name 'recon_requestor' [bugprone-argument-comment,-warnings-as-errors]
     1                                      /*they_may_initiate=*/false, /*they_may_respond=*/false,
     2                                      ^
     3./node/txreconciliation.h:67:78: note: 'recon_requestor' declared here
     4    std::optional<bool> RegisterPeer(NodeId peer_id, bool peer_inbound, bool recon_requestor,
     5                                                                             ^
     6test/txreconciliation_tests.cpp:23:68: error: argument name 'they_may_respond' in comment does not match parameter name 'recon_responder' [bugprone-argument-comment,-warnings-as-errors]
     7                                      /*they_may_initiate=*/false, /*they_may_respond=*/false,
     8                                                                   ^
     9./node/txreconciliation.h:68:43: note: 'recon_responder' declared here
    10                                     bool recon_responder, uint32_t peer_recon_version, uint64_t remote_salt);
    11                                          ^
    
  136. naumenkogs force-pushed on Apr 14, 2022
  137. in src/net_processing.cpp:2703 in 137bdf86f5 outdated
    2698+                // We announce reconciliation support only if this connection supports relaying
    2699+                // transactions when they arrive (which is not the case for blocks-only nodes,
    2700+                // block-relay-only connections, etc.).
    2701+                // Also, we announce reconciliation support only for protocol versions above
    2702+                // WTXID_RELAY per BIP-330.
    2703+                if (peer->m_tx_relay && !m_ignore_incoming_txs) {
    


    ariard commented at 3:45 pm on April 15, 2022:

    The following diff doesn’t break the functional test locally, shouldn’t this hit -blocksonly ?

     0diff --git a/src/net_processing.cpp b/src/net_processing.cpp
     1index 1e83c4ee2..3fbc9f617 100644
     2--- a/src/net_processing.cpp
     3+++ b/src/net_processing.cpp
     4@@ -2700,7 +2700,7 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
     5                 // block-relay-only connections, etc.).
     6                 // Also, we announce reconciliation support only for protocol versions above
     7                 // WTXID_RELAY per BIP-330.
     8-                if (peer->m_tx_relay && !m_ignore_incoming_txs) {
     9+                if (peer->m_tx_relay) {
    10                     const uint64_t recon_salt = m_txreconciliation->PreRegisterPeer(pfrom.GetId());
    11                     // Per the protocol, only the inbound peer initiate reconciliations and the
    12                     // outbound peer is supposed to only respond. Here we suggest our roles in
    
  138. in src/init.cpp:1285 in 137bdf86f5 outdated
    1278@@ -1277,9 +1279,16 @@ bool AppInitMain(NodeContext& node, interfaces::BlockAndHeaderTipInfo* tip_info)
    1279     node.chainman = std::make_unique<ChainstateManager>();
    1280     ChainstateManager& chainman = *node.chainman;
    1281 
    1282+    std::unique_ptr<TxReconciliationTracker> txreconciliation;
    1283+    // For now, transaction reconciliation is an experimental feature that could be
    1284+    // enabled via CLI.
    1285+    if (gArgs.GetBoolArg("-txrecon", DEFAULT_TXRECON_ENABLE)) {
    


    ariard commented at 3:49 pm on April 15, 2022:

    The following diff doesn’t break the functional test locally, shouldn’t CLI setting be covered ?

     0diff --git a/src/init.cpp b/src/init.cpp
     1index 7c7e29760..b0af005ce 100644
     2--- a/src/init.cpp
     3+++ b/src/init.cpp
     4@@ -1282,7 +1282,7 @@ bool AppInitMain(NodeContext& node, interfaces::BlockAndHeaderTipInfo* tip_info)
     5     std::unique_ptr<TxReconciliationTracker> txreconciliation;
     6     // For now, transaction reconciliation is an experimental feature that could be
     7     // enabled via CLI.
     8-    if (gArgs.GetBoolArg("-txrecon", DEFAULT_TXRECON_ENABLE)) {
     9+    if (gArgs.GetBoolArg("-txrecon", true)) {
    10         txreconciliation = std::make_unique<TxReconciliationTracker>(RECON_VERSION);
    11     }
    12 
    

    glozow commented at 5:54 pm on April 18, 2022:
    Yeah, a simple test verifying that the default is false would be nice
  139. ariard commented at 4:13 pm on April 15, 2022: member
    I did a swap of the functional test coverage. Sounds to me there are issues with some. Otherwise I think it’s mature.
  140. in src/test/denialofservice_tests.cpp:51 in f3023ce853 outdated
    52@@ -53,7 +53,7 @@ BOOST_AUTO_TEST_CASE(outbound_slow_chain_eviction)
    53     // Disable inactivity checks for this test to avoid interference
    54     static_cast<ConnmanTestMsg*>(connman.get())->SetPeerConnectTimeout(99999s);
    55     auto peerLogic = PeerManager::make(chainparams, *connman, *m_node.addrman, nullptr,
    


    glozow commented at 6:04 pm on April 18, 2022:

    nit in f3023ce8534b7ef988a4ab3e6680f02ece83718d

    (here and in setup_common)

    For people looking at the tests, it would be helpful to know what the nullptr is - either creating a std::unique_ptr<TxReconciliationTracker> m_txreconciliation = nullptr; or labeling with a comment would be good.

  141. in src/net_processing.cpp:3019 in 1de9d30a4a outdated
    2964+    // from switching announcement protocols after the connection is up.
    2965+    if (msg_type == NetMsgType::SENDRECON) {
    2966+        // For now, transaction reconciliation is an experimental feature that could be
    2967+        // enabled via CLI.
    2968+        if (!m_txreconciliation) {
    2969+            LogPrint(BCLog::NET, "sendrecon from peer=%d ignored according to our config\n", pfrom.GetId());
    


    glozow commented at 6:46 pm on April 18, 2022:

    in 1de9d30a4aae95d532f249866be45e56bb6766a3

    Why is this type BCLog::NET instead of BCLog::TXRECON?


    naumenkogs commented at 7:32 am on April 19, 2022:

    This is not an accident, I was thinking about this as well. I think stuff in net_processing.cpp makes sense to log with BCLog::NET, as it’s not really recon-specific.

    I won’t die on this hill though, if you insist i can change


    glozow commented at 6:59 pm on April 19, 2022:
    Ok yeah makes sense! Understood
  142. glozow commented at 7:40 pm on April 18, 2022: member

    I did another pass. My only remaining question in terms of specification is Should we announce sendrecon when the peer has explicitly turned transaction relay off (i.e. version.fRelay==false)? I would argue no (especially with #22778, it seems nonsensical to send messages about transaction reconciliation if we aren’t even going to make a TxRelay struct for the peer). Likewise, if we’ve specified fRelay=false and they send a sendrecon or any kind of transaction reconciliation message, perhaps we should disconnect.

    I implemented this (based on top of #22778) here.

  143. naumenkogs force-pushed on Apr 19, 2022
  144. naumenkogs commented at 3:15 pm on April 19, 2022: member

    @glozow in the discussion above we were not sure what to do with nodes signaling fRelay=0, because they might later enable tx relay via bloom filters (SPV etc.).

    Disconnecting for sendrecon after fRelay=0 would enforce the following rule: “even if tx relay is enabled via a bloom filter later, for these nodes reconciliation is prohibited”.

    I don’t know what’s the right answer.

    I’d suggest getting back to this after #22778 and have a broader discussion possibly? It would be easier to reason about the code at least. I think the discussion i reference is already difficult to grasp.

  145. glozow commented at 6:53 pm on April 19, 2022: member

    @naumenkogs Ah thanks for linking me! Sorry for missing that. Fwiw, I believe that “even if tx relay is enabled via a bloom filter later, for these nodes reconciliation is prohibited” is the correct behavior. As said above, if a peer is filtering transactions, obviously you don’t care reconciling mempools: they clearly don’t want all of your mempool’s transactions, and most likely you won’t learn much from them. Bloom filters are also disabled by default, so it’s unlikely that the tx reconciliation state that we initialize will be of any use. If anything, maybe you could have a special case for peers with bloom filter permissions, but I still don’t see much utility in reconciliation there.

    I’d suggest getting back to this after #22778 and have a broader discussion possibly?

    I think it’s better to specify this now rather than later. Sorry for any confusion caused by basing it on #22778, here is the branch without that part (just based on master + this PR). If I make a block-relay-only connection to a node, I expect to not receive any transaction-related messages from them. I would argue that a receiving a sendrecon (especially since I didn’t send one, which means we’ll never do reconciliations anyway) would fall under the transaction relay-related category.

  146. naumenkogs force-pushed on Apr 20, 2022
  147. naumenkogs commented at 11:44 am on April 20, 2022: member
    @glozow took your suggestion, updated the bip
  148. in src/net_processing.cpp:3510 in b3bd5325fe outdated
    2990+
    2991+        if (!peer->m_tx_relay) {
    2992+            // Disconnect peers that send a SENDRECON message even though we indicated we don't
    2993+            // support transaction relay.
    2994+            LogPrint(BCLog::NET, "sendrecon received from peer=%d to which we indicated no tx relay; disconnecting\n", pfrom.GetId());
    2995+            pfrom.fDisconnect = true;
    


    ariard commented at 8:37 pm on April 21, 2022:
    Mutating this value doesn’t trigger a functional test failure. If you can add one more test to check that disconnection is enforced in case we receive a SENDRECON even if we signaled we don’t support transaction relay.
  149. ariard commented at 8:43 pm on April 21, 2022: member
    If #20726 goes first, I think the no-sendrecon/bip330 messages can be scoped by bip338. See conv here : https://github.com/bitcoin/bitcoin/pull/20726/files#r780842134
  150. in src/init.cpp:1284 in b568d28f6a outdated
    1278@@ -1277,9 +1279,16 @@ bool AppInitMain(NodeContext& node, interfaces::BlockAndHeaderTipInfo* tip_info)
    1279     node.chainman = std::make_unique<ChainstateManager>();
    1280     ChainstateManager& chainman = *node.chainman;
    1281 
    1282+    std::unique_ptr<TxReconciliationTracker> txreconciliation;
    1283+    // For now, transaction reconciliation is an experimental feature that could be
    1284+    // enabled via CLI.
    


    glozow commented at 0:12 am on April 22, 2022:
    in b568d28f6a7e26d02c973245d987843d8588a025 It may be more helpful to replace this with “TODO: remove this option when Erlay is fully supported” or something

    sipa commented at 7:23 pm on April 28, 2022:

    In “p2p: Announce reconciliation support”

    Perhaps formulate it as “While Erlay support is incomplete, it must be enabled explicitly via -txrecon. This argument can go away after Erlay support is complete.”

  151. naumenkogs force-pushed on Apr 22, 2022
  152. in src/test/util/setup_common.cpp:232 in b568d28f6a outdated
    227@@ -228,8 +228,10 @@ TestingSetup::TestingSetup(const std::string& chainName, const std::vector<const
    228     m_node.banman = std::make_unique<BanMan>(m_args.GetDataDirBase() / "banlist", nullptr, DEFAULT_MISBEHAVING_BANTIME);
    229     m_node.connman = std::make_unique<CConnman>(0x1337, 0x1337, *m_node.addrman); // Deterministic randomness for tests.
    230     m_node.peerman = PeerManager::make(chainparams, *m_node.connman, *m_node.addrman,
    231-                                       m_node.banman.get(), *m_node.chainman,
    232-                                       *m_node.mempool, false);
    233+                                       m_node.banman.get(), *m_node.chainman, *m_node.mempool,
    234+                                       /*std::unique_ptr<TxReconciliationTracker> txreconciliation=*/nullptr,
    


    glozow commented at 0:44 am on April 23, 2022:

    in b568d28f6a7e26d02c973245d987843d8588a025

    Thanks for taking the suggestion! you only need the name btw

    0                                       /*txreconciliation=*/nullptr,
    
  153. ariard commented at 3:36 pm on April 27, 2022: member

    ACK 21bbd5e5

    Reviewed the code-vs-bip-consistency, the test coverage exhaustiveness, the TxReconciliationTracker interface sanity, the disconnection and message ordering requirements. Running out of ideas on how to break this PR.

  154. in test/functional/p2p_sendrecon.py:117 in 6ec8628181 outdated
    112+
    113+        self.log.info('sending SENDRECON after sending VERACK triggers a disconnect')
    114+        # We use PeerNoVerack even though verack is sent right after, to make sure it was actually
    115+        # sent before sendrecon is sent.
    116+        peer = self.nodes[0].add_p2p_connection(PeerNoVerack(), send_version=True, wait_for_verack=False)
    117+        peer.send_message(msg_verack())
    


    glozow commented at 10:33 pm on April 27, 2022:

    in 6ec8628181bb9342c338d180acc96f5a48185527: I think it would be good to sync with ping to rule out the possibility that something else triggered the disconnect. sync with ping doesn’t work before verack because they’re not really connected yet, but it’s doable here.

    0        peer.send_and_ping(msg_verack())
    
  155. in test/functional/p2p_sendrecon.py:70 in 6ec8628181 outdated
    56+        self.log.info('SENDRECON sent to an inbound')
    57+        peer = self.nodes[0].add_p2p_connection(SendReconReceiver(), send_version=True, wait_for_verack=True)
    58+        assert peer.sendrecon_msg_received
    59+        assert not peer.sendrecon_msg_received.initiator
    60+        assert peer.sendrecon_msg_received.responder
    61+        assert_equal(peer.sendrecon_msg_received.version, 1)
    


    glozow commented at 10:43 pm on April 27, 2022:

    6ec8628181bb9342c338d180acc96f5a48185527:

    I think it’s good practice to disconnect these peers after you’re done with them, to minimize the risk of subtests affecting each other. (And for the others below)

    0        assert_equal(peer.sendrecon_msg_received.version, 1)
    1        peer.disconnect_peer()
    
  156. in test/functional/p2p_sendrecon.py:107 in 6ec8628181 outdated
    82+        peer.wait_for_verack()
    83+        assert not peer.sendrecon_msg_received
    84+
    85+        self.log.info('valid SENDRECON received')
    86+        peer = self.nodes[0].add_p2p_connection(PeerNoVerack(), send_version=True, wait_for_verack=False)
    87+        peer.send_message(create_sendrecon_msg())
    


    glozow commented at 10:45 pm on April 27, 2022:

    6ec8628181bb9342c338d180acc96f5a48185527 It may be helpful to sync here to make sure the disconnect isn’t caused by the first sendrecon

    self.wait_until(lambda : self.nodes[0].getpeerinfo()[-1]["bytesrecv_per_msg"]["sendrecon"])

  157. in test/functional/p2p_sendrecon.py:122 in 21bbd5e5dd outdated
    117+        peer.send_message(msg_verack())
    118+        peer.send_message(create_sendrecon_msg())
    119+        peer.wait_for_disconnect()
    120+
    121+        self.log.info('SENDRECON without WTXIDRELAY is ignored (recon state is erased after VERACK)')
    122+        with self.nodes[0].assert_debug_log(['Forget reconciliation state of peer=8']):
    


    glozow commented at 10:51 pm on April 27, 2022:

    I’d suggest leaving out the peer id, since that isn’t necessarily a correctness issue and could change if we add more tests

    0        with self.nodes[0].assert_debug_log(['Forget reconciliation state of peer']):
    
  158. glozow commented at 10:58 pm on April 27, 2022: member

    code review ACK 21bbd5e5dda4f40debcf2b05fa03817a5be506e6

    I only have test nits which can be done in a followup.

  159. DrahtBot added the label Needs rebase on Apr 28, 2022
  160. in src/net_processing.cpp:2739 in b568d28f6a outdated
    2734+                if (pfrom.m_relays_txs && !m_ignore_incoming_txs) {
    2735+                    const uint64_t recon_salt = m_txreconciliation->PreRegisterPeer(pfrom.GetId());
    2736+                    // Only the inbound peer initiate reconciliations and the outbound peer is
    2737+                    // supposed to only respond. Here we suggest our roles in reconciliations
    2738+                    // (initiator/responder) based on the connection direction.
    2739+                    // If the way we assign the roles is updated, reconciliation protocol
    


    sipa commented at 7:27 pm on April 28, 2022:

    In “p2p: Announce reconciliation support”

    This seems inconsistent to me. The BIP330 document defines the reconciliation protocol version 1, but I don’t think it says anything about which side is supposed to initiate (and the presence of the initiator/responder bools seems to be designed to explicitly support both).

    Is there a reason for requiring a recon protocol version update when the role assignment changes?

    • If not, just drop the comment.
    • If there is, it seems cleaner to have BIP330 just hardcode v1 = outbound-initiated, and possibly even forego the two booleans. Later versions (in potential future BIPs) can change that.
  161. in src/test/denialofservice_tests.cpp:57 in b568d28f6a outdated
    52@@ -53,7 +53,9 @@ BOOST_AUTO_TEST_CASE(outbound_slow_chain_eviction)
    53     // Disable inactivity checks for this test to avoid interference
    54     static_cast<ConnmanTestMsg*>(connman.get())->SetPeerConnectTimeout(99999s);
    55     auto peerLogic = PeerManager::make(chainparams, *connman, *m_node.addrman, nullptr,
    56-                                       *m_node.chainman, *m_node.mempool, false);
    57+                                       *m_node.chainman, *m_node.mempool,
    58+                                       /*std::unique_ptr<TxReconciliationTracker> txreconciliation=*/nullptr,
    


    sipa commented at 7:34 pm on April 28, 2022:

    In “p2p: Announce reconciliation support”

    Nit: listing the type as well seems a bit excessive to me, and not something we do elsewhere either (here and further in the commit).

  162. in src/node/txreconciliation.cpp:23 in 8ed5ef61a9 outdated
    18+ */
    19+uint256 ComputeSalt(uint64_t salt1, uint64_t salt2)
    20+{
    21+    // According to BIP-330, salts should be combined in ascending order.
    22+    static auto RECON_SALT_HASHER = TaggedHash(RECON_STATIC_SALT);
    23+    return (RECON_SALT_HASHER << std::min(salt1, salt2) << std::max(salt1, salt2)).GetSHA256();
    


    sipa commented at 7:40 pm on April 28, 2022:

    In commit “Add helper to compute reconciliation salt”

    I don’t understand how this can work. I believe it’ll keep updating the shared global RECON_SALT_HASHER object (across multiple peers, and possibly even in a non-thread-safe way).

    You need CHashWriter(RECON_SALT_HASHER) << ... instead, so each salt computation uses its own copy. The RECON_SALT_HASHER object can then also become const.

    If you do that, it may also be desirable to move the “RECON_SALT_HASHER” out of the function (that’s slightly more efficient, as otherwise an atomic check is issued in every function execution to see if the static is already initialized).

  163. in src/net_processing.cpp:2965 in ac0865abf5 outdated
    2957@@ -2958,6 +2958,59 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    2958         return;
    2959     }
    2960 
    2961+    // Received from a peer demonstrating readiness to announce transactions via reconciliations.
    2962+    // This feature negotiation should happen: between VERSION and VERACK to avoid relay problems
    2963+    // from switching announcement protocols after the connection is up.
    2964+    if (msg_type == NetMsgType::SENDRECON) {
    2965+        // For now, transaction reconciliation is an experimental feature that could be
    


    sipa commented at 7:43 pm on April 28, 2022:

    In “p2p: Finish negotiating reconciliation support”

    Nit: no need to repeat this multiple times (it’s already mentioned in AppInit).

  164. in src/net_processing.cpp:2894 in 07b68d2790 outdated
    2888@@ -2889,6 +2889,18 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    2889             nCMPCTBLOCKVersion = 1;
    2890             m_connman.PushMessage(&pfrom, msgMaker.Make(NetMsgType::SENDCMPCT, fAnnounceUsingCMPCTBLOCK, nCMPCTBLOCKVersion));
    2891         }
    2892+
    2893+        if (m_txreconciliation) {
    2894+            if (!peer->m_wtxid_relay) {
    


    mzumsande commented at 6:01 pm on April 29, 2022:
    Commit “p2p: clear reconciliation state for no-wtxid peers” Could this be more general so that we also check for !IsPeerRegistered() here, forgetting about any pre-registered peers that chose not to send us a SENDRECON before verack - this will be a very frequent case for peers that haven’t implemented Erlay or aren’t interested in it.
  165. naumenkogs force-pushed on May 3, 2022
  166. DrahtBot removed the label Needs rebase on May 3, 2022
  167. in src/init.cpp:479 in 6fd947b9bf outdated
    476@@ -476,6 +477,7 @@ void SetupServerArgs(ArgsManager& argsman)
    477     argsman.AddArg("-seednode=<ip>", "Connect to a node to retrieve peer addresses, and disconnect. This option can be specified multiple times to connect to multiple nodes.", ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    478     argsman.AddArg("-networkactive", "Enable all P2P network activity (default: 1). Can be changed by the setnetworkactive RPC command", ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    479     argsman.AddArg("-timeout=<n>", strprintf("Specify socket connection timeout in milliseconds. If an initial attempt to connect is unsuccessful after this amount of time, drop it (minimum: 1, default: %d)", DEFAULT_CONNECT_TIMEOUT), ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    480+    argsman.AddArg("-txrecon", strprintf("Enable transaction reconciliations per BIP-330 (default: %d)", DEFAULT_TXRECON_ENABLE), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::CONNECTION);
    


    jonatack commented at 7:17 am on May 8, 2022:
    • “recon” has a different meaning than reconcile: https://www.merriam-webster.com/dictionary/recon …maybe -txreconcile?

    • Is -txrecon supposed to be testable/usable starting with the second commit where it is added? Would it be better to introduce the user-facing config option (maybe with a release note), after that part of the implementation is complete?

    I’m only starting to review here and haven’t yet read through all the previous review feedback, so feel free to ignore if I’m missing something or this was already discussed.


    laanwj commented at 4:00 pm on June 20, 2022:
    Agree with @jonatack that -txreconcile would be a better name (edit: then again, it’s called txrecon everywhere, maybe too late to change it)
  168. jonatack commented at 7:28 am on May 8, 2022: contributor

    Starting review here after looking through the parent PR #21515, and at first glance it looks quite a bit changed compared to the parent.

    Seeing this build warning (error for me with --enable-werror) starting from “Add helper to compute reconciliation salt” and in “Add helper to see if a peer is registered for reconciliations” until commit “Finish negotiating reconciliation support”. Adding the function when it is first used would fix that (and facilitate review).

    0node/txreconciliation.cpp:20:9: error: unused function 'ComputeSalt' [-Werror,-Wunused-function]
    1uint256 ComputeSalt(uint64_t salt1, uint64_t salt2)
    2        ^
    31 error generated.
    
  169. naumenkogs force-pushed on May 10, 2022
  170. maflcko commented at 2:18 pm on May 10, 2022: member
    Drahty generated a coverage report in #23443 (comment) . Feel free to ping @DrahtBot when it needs to be updated.
  171. DrahtBot added the label Needs rebase on May 20, 2022
  172. naumenkogs force-pushed on May 23, 2022
  173. naumenkogs force-pushed on May 23, 2022
  174. DrahtBot removed the label Needs rebase on May 23, 2022
  175. laanwj commented at 6:29 am on June 2, 2022: member

    Looks like the new test fails in some of the CI runs:

     02022-05-23T09:36:46.423000Z TestFramework (INFO): SENDRECON sent to an inbound
     12022-05-23T09:36:46.527000Z TestFramework (INFO): SENDRECON should be sent before VERACK
     22022-05-23T09:36:46.631000Z TestFramework (INFO): SENDRECON on pre-WTXID version should not be sent
     32022-05-23T09:36:46.733000Z TestFramework (INFO): SENDRECON for fRelay=false should not be sent
     42022-05-23T09:36:46.955000Z TestFramework (INFO): valid SENDRECON received
     52022-05-23T09:36:47.098000Z TestFramework (ERROR): Key error
     6Traceback (most recent call last):
     7  File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 133, in main
     8    self.run_test()
     9  File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/p2p_sendrecon.py", line 108, in run_test
    10    self.wait_until(lambda : self.nodes[0].getpeerinfo()[-1]["bytesrecv_per_msg"]["sendrecon"])
    11  File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 715, in wait_until
    12    return wait_until_helper(test_function, timeout=timeout, timeout_factor=self.options.timeout_factor)
    13  File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/util.py", line 265, in wait_until_helper
    14    if predicate():
    15  File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/p2p_sendrecon.py", line 108, in <lambda>
    16    self.wait_until(lambda : self.nodes[0].getpeerinfo()[-1]["bytesrecv_per_msg"]["sendrecon"])
    17KeyError: 'sendrecon'
    
  176. naumenkogs force-pushed on Jun 3, 2022
  177. naumenkogs force-pushed on Jun 4, 2022
  178. naumenkogs force-pushed on Jun 4, 2022
  179. DrahtBot added the label Needs rebase on Jun 15, 2022
  180. naumenkogs force-pushed on Jun 17, 2022
  181. DrahtBot removed the label Needs rebase on Jun 17, 2022
  182. naumenkogs force-pushed on Jun 17, 2022
  183. naumenkogs force-pushed on Jun 20, 2022
  184. in src/node/txreconciliation.h:67 in ff65eda016 outdated
    62+     * Returns:
    63+     * - true if the peer was registered
    64+     * - false if the peer violates the protocol
    65+     * - nullopt if nothing was done (e.g., we haven't pre-registered this peer)
    66+     */
    67+    std::optional<bool> RegisterPeer(NodeId peer_id, bool peer_inbound, bool recon_initiator,
    


    laanwj commented at 3:56 pm on June 20, 2022:
    I’m not sure I like using a std::optional<bool> as a tri-state value here. Let’s define an enum?
  185. laanwj commented at 4:08 pm on June 20, 2022: member
    Code review ACK ff65eda01675f853aa2b4c430e77767471b8d8bd
  186. in src/net_processing.cpp:1768 in ff65eda016 outdated
    1594@@ -1592,20 +1595,23 @@ std::optional<std::string> PeerManagerImpl::FetchBlock(NodeId peer_id, const CBl
    1595 
    1596 std::unique_ptr<PeerManager> PeerManager::make(CConnman& connman, AddrMan& addrman,
    1597                                                BanMan* banman, ChainstateManager& chainman,
    1598-                                               CTxMemPool& pool, bool ignore_incoming_txs)
    1599+                                               CTxMemPool& pool, std::unique_ptr<TxReconciliationTracker> txreconciliation,
    


    jonatack commented at 10:03 am on June 21, 2022:

    0d7823a20650ef3b5f2c2da6b33f6dd142636c38 I wondered why txreconciliation wasn’t being passed by X&&, but per https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#f18-for-will-move-from-parameters-pass-by-x-and-stdmove-the-parameter, this is recommended, so all good:

    “Exception: Unique owner types that are move-only and cheap-to-move, such as unique_ptr, can also be passed by value which is simpler to write and achieves the same effect. Passing by value does generate one extra (cheap) move operation, but prefer simplicity and clarity first.”

  187. in src/net_processing.cpp:2782 in ff65eda016 outdated
    2776@@ -2773,6 +2777,25 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    2777             if (fRelay) pfrom.m_relays_txs = true;
    2778         }
    2779 
    2780+        if (greatest_common_version >= WTXID_RELAY_VERSION && m_txreconciliation) {
    2781+            // Per BIP-330, we announce reconciliation support if:
    2782+            // - protocol version per the VERSION message supports for WTXID_RELAY
    


    jonatack commented at 10:04 am on June 21, 2022:

    0d7823a20650ef3b5f2c2da6b33f6dd142636c38

    0            // - protocol version per the VERSION message supports WTXID_RELAY
    
  188. in src/net_processing.cpp:2793 in ff65eda016 outdated
    2788+                // We suggest our reconciliation role (initiator/responder) based on
    2789+                // the connection direction.
    2790+                m_connman.PushMessage(&pfrom, msg_maker.Make(NetMsgType::SENDRECON,
    2791+                                                                !pfrom.IsInboundConn(),
    2792+                                                                pfrom.IsInboundConn(),
    2793+                                                                RECON_VERSION, recon_salt));
    


    jonatack commented at 10:07 am on June 21, 2022:

    0d7823a20650ef3b5f2c2da6b33f6dd142636c38 nit, indentation/clang format

    0                 m_connman.PushMessage(&pfrom, msg_maker.Make(NetMsgType::SENDRECON,
    1-                                                                !pfrom.IsInboundConn(),
    2-                                                                pfrom.IsInboundConn(),
    3-                                                                RECON_VERSION, recon_salt));
    4+                                                             !pfrom.IsInboundConn(),
    5+                                                             pfrom.IsInboundConn(),
    6+                                                             RECON_VERSION, recon_salt));
    
  189. jonatack commented at 10:40 am on June 21, 2022: contributor

    Reviewed, debug-built, and ran unit tests the first two commits up to 0d7823a20650ef3b5f2c2da6b33f6dd142636c38.

    In this PR it might be clearer to standardize on reconcile and reconciliation naming in the codebase and user-facing config option, both for simplicity and to reduce confusion, as “recon” (e.g. NetMsgType::SENDRECON, -txrecon) is a different word with a different meaning (as I mentioned in an earlier comment). It would be easier to do it now while introducing the naming, than later on.

  190. in src/node/txreconciliation.cpp:40 in 0d7823a206 outdated
    35+    std::unordered_map<NodeId, std::variant<uint64_t, ReconciliationState>> m_states GUARDED_BY(m_mutex);
    36+
    37+public:
    38+    // Local protocol version
    39+    // Made public to supress -Wunused-private-field. Should be made private when becomes used.
    40+    const uint32_t m_recon_version;
    


    jonatack commented at 10:41 am on June 21, 2022:

    https://github.com/bitcoin/bitcoin/commit/0d7823a20650ef3b5f2c2da6b33f6dd142636c38 It might be simpler to add this data member when it is first used, which avoids needing to move it and delete the comments at that point.

    Edit: the following diff in this commit builds just fine, m_recon_version is used in the ctor.

     0@@ -24,6 +24,8 @@ class ReconciliationState
     1 /** Actual implementation for TxReconciliationTracker's data structure. */
     2 class TxReconciliationTracker::Impl
     3 {
     4+private:
     5+    const uint32_t m_recon_version;
     6     mutable Mutex m_mutex;
     7 
     8     /**
     9@@ -35,10 +37,6 @@ class TxReconciliationTracker::Impl
    10     std::unordered_map<NodeId, std::variant<uint64_t, ReconciliationState>> m_states GUARDED_BY(m_mutex);
    11 
    12 public:
    13-    // Local protocol version
    14-    // Made public to supress -Wunused-private-field. Should be made private when becomes used.
    15-    const uint32_t m_recon_version;
    16-
    17     explicit Impl(uint32_t recon_version) : m_recon_version(recon_version) {}
    

    naumenkogs commented at 9:49 am on June 22, 2022:

    My compiler gives this:

    0node/txreconciliation.cpp:29:20: warning: private field 'm_recon_version' is not used [-Wunused-private-field]
    1    const uint32_t m_recon_version;
    

    naumenkogs commented at 9:49 am on June 22, 2022:
    So i’ll stick to the first option you suggest i guess.
  191. in src/node/txreconciliation.cpp:85 in 0d7823a206 outdated
    43+
    44+    uint64_t PreRegisterPeer(NodeId peer_id) EXCLUSIVE_LOCKS_REQUIRED(!m_mutex)
    45+    {
    46+        LOCK(m_mutex);
    47+        // We do not support reconciliation salt/version updates.
    48+        assert(m_states.find(peer_id) == m_states.end());
    


    jonatack commented at 10:47 am on June 21, 2022:
    0d7823a20650ef3b5f2c2da6b33f6dd142636c38 This lookup could have complexity up to linear size of the unordered map, if you see a way now or later on to perform this check more cheaply. Edit: though for now PreRegisterPeer doesn’t seem to be a hotspot yet.

    naumenkogs commented at 10:01 am on June 22, 2022:
    isn’t it o(1)?

    jonatack commented at 7:11 pm on June 23, 2022:
    Worst case linear in the size of the container. https://en.cppreference.com/w/cpp/container/unordered_map/find

    sipa commented at 6:35 pm on June 24, 2022:
    If you use a salted hasher for the unordered map, the worst-case behavior becomes irrelevant.

    jonatack commented at 6:47 pm on June 24, 2022:

    If you use a salted hasher for the unordered map, the worst-case behavior becomes irrelevant.

    Ah! Thank you.

  192. in src/node/txreconciliation.cpp:51 in 0d7823a206 outdated
    46+        LOCK(m_mutex);
    47+        // We do not support reconciliation salt/version updates.
    48+        assert(m_states.find(peer_id) == m_states.end());
    49+
    50+        LogPrint(BCLog::TXRECON, "Pre-register peer=%d.\n", peer_id);
    51+        uint64_t local_recon_salt{GetRand(UINT64_MAX)};
    


    jonatack commented at 10:50 am on June 21, 2022:

    0d7823a20650ef3b5f2c2da6b33f6dd142636c38 suggest LogPrintLevel with BCLog::Level::Debug (see #25203 (review)), and no full stop i.e. s/%d./%d/, and const, while retouching

    0-        LogPrint(BCLog::TXRECON, "Pre-register peer=%d.\n", peer_id);
    1-        uint64_t local_recon_salt{GetRand(UINT64_MAX)};
    2+        LogPrintLevel(BCLog::TXRECON, BCLog::Level::Debug, "Pre-register peer=%d\n", peer_id);
    3+        const uint64_t local_recon_salt{GetRand(UINT64_MAX)};
    

    Edit: same for the other logging added in this PR.

  193. in src/node/txreconciliation.h:55 in 0d7823a206 outdated
    38+ * changes (sketch extensions instead of bisections, and an extra INV exchange round), both
    39+ * are motivated in BIP-330.
    40+ */
    41+class TxReconciliationTracker
    42+{
    43+    class Impl;
    


    jonatack commented at 11:39 am on June 21, 2022:
    0d7823a20650ef3b5f2c2da6b3 might be good to add a prefix to the Impl class name? e.g. ChainImpl, SketchImpl, NodeImpl, PeerManagerImpl, MainSignalsImpl, etc.

    naumenkogs commented at 10:05 am on June 22, 2022:
    I used the pattern from nanobench.h and txrequest.cpp as it is I don’t have a strong opinion here, but I thought Impl is ok because it’s very internal to this file?

    jonatack commented at 7:09 pm on June 23, 2022:
    nanobench.h is from an external library; I suggested it as it seems to be the practice throughout the codebase.
  194. in src/node/txreconciliation.cpp:56 in ff65eda016 outdated
    51+} // namespace
    52+
    53+/** Actual implementation for TxReconciliationTracker's data structure. */
    54+class TxReconciliationTracker::Impl
    55+{
    56+    mutable Mutex m_mutex;
    


    jonatack commented at 12:36 pm on June 21, 2022:
    0d7823a20650ef3b5f2 maybe give m_mutex a more specific name, such as seen with git grep "mutable Mutex"

    naumenkogs commented at 10:10 am on June 22, 2022:
    Not sure about this, because it’s local (which is also clear from m_). Looking to hear from other reviewers :)

    jonatack commented at 7:07 pm on June 23, 2022:

    For distinguishing grepability and the recent naming changes follow this.

    0src/net.h:686:    mutable Mutex m_addr_local_mutex;
    1src/net.h:1047:    mutable Mutex m_total_bytes_sent_mutex;
    2src/net.h:1074:    mutable Mutex m_added_nodes_mutex;
    3src/net_processing.cpp:330:    mutable Mutex m_addr_send_times_mutex;
    4src/net_processing.cpp:617:    mutable Mutex m_peer_mutex;
    5src/policy/fees.h:241:    mutable Mutex m_cs_fee_estimator;
    
  195. jonatack commented at 1:41 pm on June 21, 2022: contributor

    Reviewed commits 4718ab6d7de3288b01d22a610f280ed198e2e991 and a410f3e3c9e9cc2. Successfully tested building 4718ab6d7de3288b01d22a610f280e with -Wthread-safety-negative, couldn’t build a410f3e3c9e9cc2 due to:

    0node/txreconciliation.cpp:23:9: error: ‘uint256 {anonymous}::ComputeSalt(uint64_t, uint64_t)’ 
    1defined but not used [-Werror=unused-function]
    2   23 | uint256 ComputeSalt(uint64_t salt1, uint64_t salt2)
    3      |         ^~~~~~~~~~~
    

    It would be good to add code when it is used so the commits are hygienic, i.e. build without warnings or errors.

  196. in src/net_processing.cpp:3007 in 9dd3715e03 outdated
    2998@@ -2999,6 +2999,57 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    2999         return;
    3000     }
    3001 
    3002+    // Received from a peer demonstrating readiness to announce transactions via reconciliations.
    3003+    // This feature negotiation should happen: between VERSION and VERACK to avoid relay problems
    3004+    // from switching announcement protocols after the connection is up.
    3005+    if (msg_type == NetMsgType::SENDRECON) {
    3006+        if (!m_txreconciliation) {
    3007+            LogPrint(BCLog::NET, "sendrecon from peer=%d ignored according to our config\n", pfrom.GetId());
    


    jonatack commented at 2:34 pm on June 21, 2022:

    9dd3715e034d29a20 suggestions, IIUC

    0     // Received from a peer demonstrating readiness to announce transactions via reconciliations.
    1-    // This feature negotiation should happen: between VERSION and VERACK to avoid relay problems
    2+    // This feature negotiation must happen between VERSION and VERACK to avoid relay problems
    3     // from switching announcement protocols after the connection is up.
    4     if (msg_type == NetMsgType::SENDRECON) {
    5         if (!m_txreconciliation) {
    6-            LogPrint(BCLog::NET, "sendrecon from peer=%d ignored according to our config\n", pfrom.GetId());
    7+            LogPrint(BCLog::NET, "sendrecon from peer=%d ignored, as our node does not have transaction reconciliation enabled\n", pfrom.GetId());
    8             return;
    9         }
    
  197. in src/net_processing.cpp:3021 in 9dd3715e03 outdated
    3016+        }
    3017+
    3018+        if (!peer->GetTxRelay()) {
    3019+            // Disconnect peers that send a SENDRECON message even though we indicated we don't
    3020+            // support transaction relay.
    3021+            LogPrint(BCLog::NET, "sendrecon received from peer=%d to which we indicated no tx relay; disconnecting\n", pfrom.GetId());
    


    jonatack commented at 2:35 pm on June 21, 2022:

    9dd3715e034d29a2024 Should BIP330 be updated to state that SENDRECON must happen between VERSION and VERACK and only to peers that signal tx relay, or is that considered an optional implementation-level question?

    Edit: I now see this is done in https://github.com/naumenkogs/bips/blob/bip_0330_updates/bip-0330.mediawiki#sendrecon, though maybe s/should/must/ happen between version and verack.

  198. in src/net_processing.cpp:3522 in 9dd3715e03 outdated
    3030+        vRecv >> they_initiator >> they_responder >> peer_recon_version >> remote_salt;
    3031+
    3032+        if (m_txreconciliation->IsPeerRegistered(pfrom.GetId())) {
    3033+            // A peer is already registered, meaning we already received SENDRECON from them.
    3034+            LogPrint(BCLog::NET, "reconciliation protocol violation from peer=%d; disconnecting\n", pfrom.GetId());
    3035+            pfrom.fDisconnect = true;
    


    jonatack commented at 2:39 pm on June 21, 2022:

    BIP330 states, “The sendrecon message announces support for the reconciliation protocol. It is expected to be only sent once, and ignored by nodes that don’t support it.”

    Should that be updated to “should or must be sent only once, failing which the peer is disconnected,” and ignored by nodes that don’t support it?

    Edit: I now see this is done in https://github.com/naumenkogs/bips/blob/bip_0330_updates/bip-0330.mediawiki#sendrecon (it would be good to update BIP330).

  199. in src/protocol.h:262 in 0d7823a206 outdated
    256@@ -257,6 +257,14 @@ extern const char* CFCHECKPT;
    257  * @since protocol version 70016 as described by BIP 339.
    258  */
    259 extern const char* WTXIDRELAY;
    260+/**
    261+ * Contains 2 1-byte bools, a 4-byte version number and an 8-byte salt.
    


    jonatack commented at 2:56 pm on June 21, 2022:
    0d7823a20650ef3b5f2c2da6b33f6 would it be worthwhile, instead of two bools, to use bit flags in one uint8_t to save a byte per message?

    naumenkogs commented at 10:31 am on June 22, 2022:
    I’d say no because it’s done once per connection.

    jonatack commented at 8:53 am on June 24, 2022:
    Just a thought, could be done later, but service bits/flags (like in the VERSION net message) may provide future flexibility as well.
  200. in src/net_processing.cpp:3034 in 9dd3715e03 outdated
    3029+        uint64_t remote_salt;
    3030+        vRecv >> they_initiator >> they_responder >> peer_recon_version >> remote_salt;
    3031+
    3032+        if (m_txreconciliation->IsPeerRegistered(pfrom.GetId())) {
    3033+            // A peer is already registered, meaning we already received SENDRECON from them.
    3034+            LogPrint(BCLog::NET, "reconciliation protocol violation from peer=%d; disconnecting\n", pfrom.GetId());
    


    jonatack commented at 3:02 pm on June 21, 2022:

    9dd3715e034d29a2024696224f13607 allow distinguishing this event and log message from the currently identical one below on line 3046

    0            LogPrintLevel(BCLog::NET, BCLog::Level::Debug, "reconciliation protocol violation from peer=%d (sendrecon received from already registered peer); disconnecting\n", pfrom.GetId());
    
  201. in src/node/txreconciliation.cpp:53 in 9dd3715e03 outdated
    41+    /**
    42+     * These values are used to salt short IDs, which is necessary for transaction reconciliations.
    43+     * TODO: they are public to ignore -Wunused-private-field. They should be made private once they
    44+     * are used.
    45+     */
    46+    uint64_t m_k0, m_k1;
    


    jonatack commented at 3:06 pm on June 21, 2022:

    9dd3715e034d29a2024696224f1 this builds fine for me and they seem to be already used in the ctor

     0 class ReconciliationState
     1 {
     2+private:
     3+    uint64_t m_k0, m_k1;
     4 public:
     5     /**
     6      * Reconciliation protocol assumes using one role consistently: either a reconciliation
     7@@ -38,13 +40,6 @@ public:
     8      * */
     9     bool m_we_initiate;
    10 
    11-    /**
    12-     * These values are used to salt short IDs, which is necessary for transaction reconciliations.
    13-     * TODO: they are public to ignore -Wunused-private-field. They should be made private once they
    14-     * are used.
    15-     */
    16-    uint64_t m_k0, m_k1;
    17-
    18     ReconciliationState(bool we_initiate, uint64_t k0, uint64_t k1) : m_we_initiate(we_initiate), m_k0(k0), m_k1(k1) {}
    19 };
    

    naumenkogs commented at 10:36 am on June 22, 2022:
     0  CXX      node/libbitcoin_node_a-txreconciliation.o
     1node/txreconciliation.cpp:49:71: warning: field 'm_we_initiate' will be initialized after field 'm_k0' [-Wreorder-ctor]
     2    ReconciliationState(bool we_initiate, uint64_t k0, uint64_t k1) : m_we_initiate(we_initiate), m_k0(k0), m_k1(k1) {}
     3                                                                      ^
     4node/txreconciliation.cpp:40:14: warning: private field 'm_k0' is not used [-Wunused-private-field]
     5    uint64_t m_k0, m_k1;
     6             ^
     7node/txreconciliation.cpp:40:20: warning: private field 'm_k1' is not used [-Wunused-private-field]
     8    uint64_t m_k0, m_k1;
     9                   ^
    103 warnings generated.
    

    jonatack commented at 7:05 pm on June 23, 2022:
    What compiler/version and configure are you using?

  202. in src/net_processing.cpp:3027 in 9dd3715e03 outdated
    3022+            pfrom.fDisconnect = true;
    3023+            return;
    3024+        }
    3025+
    3026+        bool they_initiator;
    3027+        bool they_responder;
    


    jonatack commented at 3:11 pm on June 21, 2022:

    9dd3715e034d29a2024696224f1360754 naming nit, maybe is_sender (or is_initator) and is_responder

    0-        bool they_initiator;
    1-        bool they_responder;
    2+        bool is_sender, is_responder;
    

    naumenkogs commented at 10:38 am on June 22, 2022:
    I’d say is is confusing, because it could also apply to the local state. That’s why i distinguish me/they.

    jonatack commented at 7:04 pm on June 23, 2022:
    How about is_peer_{sender/receiver}

    naumenkogs commented at 7:46 am on June 27, 2022:
    Sure, i can apply this while making other changes
  203. jonatack commented at 3:39 pm on June 21, 2022: contributor
    Reviewed 9dd3715e034d2
  204. in src/net_processing.cpp:3499 in ff65eda016 outdated
    3018+        if (!m_txreconciliation) {
    3019+            LogPrint(BCLog::NET, "sendrecon from peer=%d ignored according to our config\n", pfrom.GetId());
    3020+            return;
    3021+        }
    3022+
    3023+        if (pfrom.fSuccessfullyConnected) {
    


    ariard commented at 4:14 pm on June 21, 2022:

    Hand-testing this requirement between 2 peers on regtest, I wonder if this check is correct.

    When we receive peer’s VERSION, we reply with VERACK L2707, then we send SENDRECON L2742 so isn’t the order always to be VERACK-SENDRECON from a receiver viewpoint and trigger a violation ?

    Confirmed by the logs I can observe (additional logs of messages reception) are mine:

    02022-06-21T16:00:29Z Received a VERACK from peer=0
    12022-06-21T16:00:29Z New outbound peer connected: version: 70016, blocks=0, peer=0 (manual)
    22022-06-21T16:00:29Z Received a SENDRECON from peer=0
    32022-06-21T16:00:29Z sendrecon received after verack from peer=0; disconnecting
    

    This check should be covered in p2p_sendrecon.py, dunno if I’m missing something.


    naumenkogs commented at 9:16 am on June 23, 2022:
    Can you send me the commit hash where you got these line numbers (L2707, L2742)? I indeed had this bug in some version, but i fixed it a week or two ago. I think I also added the test for this issue.

    ariard commented at 6:44 pm on June 23, 2022:

    21bbd5e5d

    Effectively it’s not the head branch. Though it was subtle enough to be missed during some previous round of reviews, myself included :/ I’ll do manual tests again if the PR is ready again for review.

  205. naumenkogs force-pushed on Jun 23, 2022
  206. naumenkogs force-pushed on Jun 24, 2022
  207. DrahtBot added the label Needs rebase on Jun 29, 2022
  208. naumenkogs force-pushed on Jun 29, 2022
  209. laanwj commented at 9:24 am on June 29, 2022: member

    Kind of annoying that our linter doesn’t handle Log* calls with arguments over multiple lines

    0ERROR: There were 1 failed tests in the lint-files.py lint test. Please resolve the above errors.
    1All calls to LogPrintf(), LogPrintfCategory(), LogPrint(), LogPrintLevel(), and WalletLogPrintf() should be terminated with "\n".
    2
    3src/net_processing.cpp:            LogPrintLevel(BCLog::NET, BCLog::Level::Debug,
    4src/node/txreconciliation.cpp:        LogPrintLevel(BCLog::TXRECONCILIATION, BCLog::Level::Debug,
    5src/node/txreconciliation.cpp:            LogPrintLevel(BCLog::TXRECONCILIATION, BCLog::Level::Debug,
    
  210. DrahtBot removed the label Needs rebase on Jun 29, 2022
  211. naumenkogs force-pushed on Jun 29, 2022
  212. in src/protocol.h:268 in 2c04a2d43a outdated
    262+ * The 2 booleans indicate that a node is willing to participate in transaction
    263+ * reconciliation, respectively as an initiator or as a receiver.
    264+ * The salt is used to compute short txids needed for efficient
    265+ * txreconciliation, as described by BIP 330.
    266+ */
    267+extern const char* SENDTXRCNCL;
    


    dergoegge commented at 11:56 am on June 30, 2022:
    The BIP should be updated to reflect the new name. (sendrecon -> sendtxrcncl)
  213. in src/node/txreconciliation.h:61 in 731caefda9 outdated
    67+    std::optional<bool> RegisterPeer(NodeId peer_id, bool peer_inbound, bool recon_initiator,
    68+                                     bool recon_responder, uint32_t peer_recon_version, uint64_t remote_salt);
    69+
    70     /**
    71      * Attempts to forget txreconciliation-related state of the peer (if we previously stored any).
    72-     * After this, we won't be able to reconcile transactions with the peer.
    


    dergoegge commented at 12:29 pm on June 30, 2022:
    nit: this looks like an accidental change in 731caefda93190082491eeefeec5b6840e321ae5
  214. in src/net_processing.h:27 in 2c04a2d43a outdated
    22@@ -22,6 +23,10 @@ static const bool DEFAULT_PEERBLOOMFILTERS = false;
    23 static const bool DEFAULT_PEERBLOCKFILTERS = false;
    24 /** Threshold for marking a node to be discouraged, e.g. disconnected and added to the discouragement filter. */
    25 static const int DISCOURAGEMENT_THRESHOLD{100};
    26+/** Whether transaction reconciliation protocol should be enabled by default. */
    27+static const bool DEFAULT_TXRECONCILIATION_ENABLE = false;
    


    dergoegge commented at 1:19 pm on June 30, 2022:
    0static const bool DEFAULT_TXRECONCILIATION_ENABLE{false};
    
  215. dergoegge commented at 1:49 pm on June 30, 2022: member

    I’ve reviewed the code and tests, and I think the PR is in a really good state. I’ll ack once the lint issues are resolved.

    Maybe squash d0ed9ad191f0c03cf901386140d365f8f4979aa8 and b7d92ec6c96ede31af995f1269c8527da8fe5ff5 into one? A node running on d0ed9ad191f0c03cf901386140d365f8f4979aa8 would eventually run out of memory as the reconciliation state for peers is never erased from m_states. This is only a theoretical concern as there won’t be a release at exactly that commit and m_states only holds a few bytes per peer.

  216. naumenkogs force-pushed on Jul 4, 2022
  217. naumenkogs force-pushed on Jul 4, 2022
  218. DrahtBot added the label Needs rebase on Jul 19, 2022
  219. naumenkogs force-pushed on Jul 26, 2022
  220. DrahtBot removed the label Needs rebase on Jul 26, 2022
  221. naumenkogs commented at 11:54 am on July 26, 2022: member
    The PR is ready for review again! Please take a look.
  222. bitcoin deleted a comment on Jul 30, 2022
  223. bitcoin deleted a comment on Jul 30, 2022
  224. bitcoin deleted a comment on Jul 30, 2022
  225. in src/node/txreconciliation.h:53 in 362fc47127 outdated
    37+ * This is a modification of the Erlay protocol (https://arxiv.org/abs/1905.10518) with two
    38+ * changes (sketch extensions instead of bisections, and an extra INV exchange round), both
    39+ * are motivated in BIP-330.
    40+ */
    41+class TxReconciliationTracker
    42+{
    


    jonatack commented at 11:55 am on August 1, 2022:

    7a5603c might be good to add an explicit private here

    0 class TxReconciliationTracker
    1 {
    2+private:
    3     class Impl;
    
  226. in src/node/txreconciliation.h:52 in 362fc47127 outdated
    47+    explicit TxReconciliationTracker(uint32_t recon_version);
    48+    ~TxReconciliationTracker();
    49+
    50+    /**
    51+     * Step 0. Generates initial part of the state (salt) required to reconcile txs with the peer.
    52+     * The salt used for short ID computation required for txreconciliation.
    


    jonatack commented at 12:04 pm on August 1, 2022:

    https://github.com/bitcoin/bitcoin/commit/7a5603c61b999ef66bdba7a7bfe1b4044dc502bc

    0     * The salt is used for short ID computation required for txreconciliation.
    
  227. in src/node/txreconciliation.h:20 in 362fc47127 outdated
    15+ * Transaction reconciliation is a way for nodes to efficiently announce transactions.
    16+ * This object keeps track of all txreconciliation-related communications with the peers.
    17+ * The high-level protocol is:
    18+ * 0.  Txreconciliation protocol handshake.
    19+ * 1.  Once we receive a new transaction, add it to the set instead of announcing immediately.
    20+ * 2.  At regular intervals, a txreconciliation initiator requests a sketch from the peer, where a
    


    jonatack commented at 12:05 pm on August 1, 2022:

    https://github.com/bitcoin/bitcoin/commit/7a5603c61b999ef66bdba7a7bfe1b4044dc502bc

    0 * 2.  At regular intervals, a txreconciliation initiator requests a sketch from a peer, where a
    
  228. in src/node/txreconciliation.cpp:64 in 362fc47127 outdated
    51+} // namespace
    52+
    53+/** Actual implementation for TxReconciliationTracker's data structure. */
    54+class TxReconciliationTracker::Impl
    55+{
    56+    mutable Mutex m_txreconciliation_mutex;
    


    jonatack commented at 12:10 pm on August 1, 2022:

    7a5603c might be good to add an explicit private here

    0 class TxReconciliationTracker::Impl
    1 {
    2+private:
    3     mutable Mutex m_txreconciliation_mutex;
    
  229. in src/node/txreconciliation.cpp:83 in 362fc47127 outdated
    69+public:
    70+    explicit Impl(uint32_t recon_version) : m_recon_version(recon_version) {}
    71+
    72+    uint64_t PreRegisterPeer(NodeId peer_id) EXCLUSIVE_LOCKS_REQUIRED(!m_txreconciliation_mutex)
    73+    {
    74+        LOCK(m_txreconciliation_mutex);
    


    jonatack commented at 12:19 pm on August 1, 2022:

    https://github.com/bitcoin/bitcoin/commit/7a5603c61b999ef66bdba7a7bfe1b4044dc502bc I think we still want the runtime lock assertion here to catch anything the clang thread safety analysis might miss with the annotation

    0     uint64_t PreRegisterPeer(NodeId peer_id) EXCLUSIVE_LOCKS_REQUIRED(!m_txreconciliation_mutex)
    1     {
    2+        AssertLockNotHeld(m_txreconciliation_mutex);
    3         LOCK(m_txreconciliation_mutex);
    
  230. in src/node/txreconciliation.cpp:83 in 362fc47127 outdated
    78+        LogPrintLevel(BCLog::TXRECONCILIATION, BCLog::Level::Debug, "Pre-register peer=%d\n", peer_id);
    79+        const uint64_t local_salt{GetRand(UINT64_MAX)};
    80+
    81+        // We do this exactly once per peer (which are unique by NodeId, see GetNewNodeId) so it's
    82+        // safe to assume we don't have this record yet.
    83+        assert(m_states.emplace(peer_id, local_salt).second);
    


    jonatack commented at 12:38 pm on August 1, 2022:

    https://github.com/bitcoin/bitcoin/commit/7a5603c61b999ef66bdba7a7bfe1b4044dc502bc do not use assert with side effects IIUC

    0-        assert(m_states.emplace(peer_id, local_salt).second);
    1+        const bool result{m_states.emplace(peer_id, local_salt).second};
    2+        assert(result);
    

    or

    0+#include <util/check.h>
    1 #include <util/system.h>
    2
    3-        assert(m_states.emplace(peer_id, local_salt).second);
    4+        Assert(m_states.emplace(peer_id, local_salt).second);
    
  231. in src/init.cpp:485 in 362fc47127 outdated
    481@@ -481,6 +482,7 @@ void SetupServerArgs(ArgsManager& argsman)
    482     argsman.AddArg("-seednode=<ip>", "Connect to a node to retrieve peer addresses, and disconnect. This option can be specified multiple times to connect to multiple nodes.", ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    483     argsman.AddArg("-networkactive", "Enable all P2P network activity (default: 1). Can be changed by the setnetworkactive RPC command", ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    484     argsman.AddArg("-timeout=<n>", strprintf("Specify socket connection timeout in milliseconds. If an initial attempt to connect is unsuccessful after this amount of time, drop it (minimum: 1, default: %d)", DEFAULT_CONNECT_TIMEOUT), ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    485+    argsman.AddArg("-txreconciliation", strprintf("Enable transaction reconciliations per BIP-330 (default: %d)", DEFAULT_TXRECONCILIATION_ENABLE), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::CONNECTION);
    


    jonatack commented at 1:06 pm on August 1, 2022:

    https://github.com/bitcoin/bitcoin/commit/7a5603c61b999ef66bdba7a7bfe1b4044dc502bc user-facing nit

    0    argsman.AddArg("-txreconciliation", strprintf("Enable transaction reconciliations per BIP 330 (default: %d)", DEFAULT_TXRECONCILIATION_ENABLE), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::CONNECTION);
    
  232. jonatack commented at 1:06 pm on August 1, 2022: contributor

    Verified debug build and unit tests for each commit after rebasing this branch to current master, ran bitcoind -txreconciliation -debug=txreconciliation with this branch and checked txreconciliation logging,

    02022-08-01T12:39:25Z [txreconciliation:debug] Pre-register peer=147
    12022-08-01T12:39:25Z [txreconciliation:debug] Forget txreconciliation state of peer=146
    22022-08-01T12:39:28Z [txreconciliation:debug] Forget txreconciliation state of peer=147
    32022-08-01T12:39:32Z [txreconciliation:debug] Pre-register peer=148
    42022-08-01T12:39:35Z [txreconciliation:debug] Forget txreconciliation state of peer=148
    
    0$ ./src/bitcoind -help-debug | grep -A1 "\-txreconciliation"
    1  -txreconciliation
    2       Enable transaction reconciliations per BIP-330 (default: 0)
    

    Reviewed first two commits 29123f1df2 and 7a5603c61b999ef66bdba7a7bfe1b4044dc502bc, ACK with a few comments that follow. (Continuing with review of the remaining commits.)

  233. in src/node/txreconciliation.cpp:18 in 362fc47127 outdated
    12+
    13+namespace {
    14+
    15+/** Static salt component used to compute short txids for sketch construction, see BIP-330. */
    16+static const std::string RECON_STATIC_SALT = "Tx Relay Salting";
    17+static const HashWriter RECON_SALT_HASHER = TaggedHash(RECON_STATIC_SALT);
    


    aureleoules commented at 3:49 pm on August 2, 2022:

    the static keyword is redundant in unnamed namespaces

    0const std::string RECON_STATIC_SALT = "Tx Relay Salting";
    1const HashWriter RECON_SALT_HASHER = TaggedHash(RECON_STATIC_SALT);
    
  234. in src/node/txreconciliation.cpp:163 in 362fc47127 outdated
    146+
    147+TxReconciliationTracker::TxReconciliationTracker(uint32_t recon_version) : m_impl{std::make_unique<TxReconciliationTracker::Impl>(recon_version)} {}
    148+
    149+TxReconciliationTracker::~TxReconciliationTracker() = default;
    150+
    151+uint64_t TxReconciliationTracker::PreRegisterPeer(NodeId peer_id)
    


    aureleoules commented at 3:54 pm on August 2, 2022:

    I believe this can be made const

    0uint64_t TxReconciliationTracker::PreRegisterPeer(NodeId peer_id) const
    

    jonatack commented at 6:22 pm on August 4, 2022:
    It is performing side effects.
  235. in src/node/txreconciliation.cpp:176 in 362fc47127 outdated
    159+{
    160+    return m_impl->RegisterPeer(peer_id, peer_inbound, recon_initiator, recon_responder,
    161+                                peer_recon_version, remote_salt);
    162+}
    163+
    164+void TxReconciliationTracker::ForgetPeer(NodeId peer_id)
    


    aureleoules commented at 3:54 pm on August 2, 2022:

    I believe this can be made const

    0void TxReconciliationTracker::ForgetPeer(NodeId peer_id) const
    

    jonatack commented at 6:22 pm on August 4, 2022:
    It is performing side effects.
  236. in src/net_processing.cpp:3419 in 362fc47127 outdated
    3062@@ -3040,6 +3063,18 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    3063             // they may wish to request compact blocks from us
    3064             m_connman.PushMessage(&pfrom, msgMaker.Make(NetMsgType::SENDCMPCT, /*high_bandwidth=*/false, /*version=*/CMPCTBLOCKS_VERSION));
    3065         }
    3066+
    3067+        if (m_txreconciliation) {
    3068+            if (!peer->m_wtxid_relay || !m_txreconciliation->IsPeerRegistered(pfrom.GetId())) {
    


    aureleoules commented at 3:56 pm on August 2, 2022:
    maybe merge the if statements for readability?

    naumenkogs commented at 10:46 am on August 18, 2022:
    i think the two-step condition is more readable :)
  237. DrahtBot added the label Needs rebase on Aug 3, 2022
  238. in src/node/txreconciliation.cpp:143 in 362fc47127 outdated
    127+        return true;
    128+    }
    129+
    130+    void ForgetPeer(NodeId peer_id) EXCLUSIVE_LOCKS_REQUIRED(!m_txreconciliation_mutex)
    131+    {
    132+        LOCK(m_txreconciliation_mutex);
    


    jonatack commented at 10:59 am on August 3, 2022:

    05ca0fe

    0     void ForgetPeer(NodeId peer_id) EXCLUSIVE_LOCKS_REQUIRED(!m_txreconciliation_mutex)
    1     {
    2+        AssertLockNotHeld(m_txreconciliation_mutex);
    3         LOCK(m_txreconciliation_mutex);
    
  239. in src/node/txreconciliation.cpp:152 in 362fc47127 outdated
    135+        }
    136+    }
    137+
    138+    bool IsPeerRegistered(NodeId peer_id) const EXCLUSIVE_LOCKS_REQUIRED(!m_txreconciliation_mutex)
    139+    {
    140+        LOCK(m_txreconciliation_mutex);
    


    jonatack commented at 11:16 am on August 3, 2022:

    https://github.com/bitcoin/bitcoin/commit/457047f2dc3a7d7e293b731b9febaf54edfcfe39

    0     bool IsPeerRegistered(NodeId peer_id) const EXCLUSIVE_LOCKS_REQUIRED(!m_txreconciliation_mutex)
    1     {
    2+        AssertLockNotHeld(m_txreconciliation_mutex);
    3         LOCK(m_txreconciliation_mutex);
    
  240. in src/node/txreconciliation.h:61 in 362fc47127 outdated
    56+     */
    57+    uint64_t PreRegisterPeer(NodeId peer_id);
    58+
    59+    /**
    60+     * Step 0. Once the peer agreed to reconcile txs with us, generate the state required to track
    61+     * ongoing reconciliations. Should be called only after pre-registering the peer and only once.
    


    jonatack commented at 11:19 am on August 3, 2022:

    b22525d s/should/must/?

    0     * ongoing reconciliations. Must be called only after pre-registering the peer and only once.
    
  241. in src/node/txreconciliation.cpp:59 in 362fc47127 outdated
    54+class TxReconciliationTracker::Impl
    55+{
    56+    mutable Mutex m_txreconciliation_mutex;
    57+
    58+    // Local protocol version
    59+    const uint32_t m_recon_version;
    


  242. in src/net_processing.cpp:3177 in 362fc47127 outdated
    3172+            LogPrintLevel(BCLog::NET, BCLog::Level::Debug, "txreconciliation protocol violation from peer=%d (sendtxrcncl received from already registered peer); disconnecting\n", pfrom.GetId());
    3173+            pfrom.fDisconnect = true;
    3174+            return;
    3175+        }
    3176+
    3177+        const auto registered = m_txreconciliation->RegisterPeer(pfrom.GetId(), pfrom.IsInboundConn(),
    


    jonatack commented at 11:45 am on August 3, 2022:

    b22525d stating the type seems clearer here and using auto doesn’t simplify or shorten the code

    0        const bool registered = m_txreconciliation->RegisterPeer(pfrom.GetId(), pfrom.IsInboundConn(),
    
  243. in src/net_processing.cpp:3146 in 362fc47127 outdated
    3137@@ -3103,6 +3138,58 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    3138         return;
    3139     }
    3140 
    3141+    // Received from a peer demonstrating readiness to announce transactions via reconciliations.
    3142+    // This feature negotiation must happen between VERSION and VERACK to avoid relay problems
    3143+    // from switching announcement protocols after the connection is up.
    3144+    if (msg_type == NetMsgType::SENDTXRCNCL) {
    3145+        if (!m_txreconciliation) {
    3146+            LogPrint(BCLog::NET, "sendtxrcncl from peer=%d ignored, as our node does not have txreconciliation enabled\n", pfrom.GetId());
    


    jonatack commented at 11:47 am on August 3, 2022:
    https://github.com/bitcoin/bitcoin/commit/b22525d2f785f2a8a9ec928dab25ea9fa0452213 the four remaining LogPrint(BCLog::NET, statements in this commit can be LogPrintLevel(BCLog::NET, BCLog::Level::Debug,
  244. in src/node/txreconciliation.h:68 in 362fc47127 outdated
    63+     * - true if the peer was registered
    64+     * - false if the peer violates the protocol
    65+     * - nullopt if nothing was done (e.g., we haven't pre-registered this peer)
    66+     */
    67+    std::optional<bool> RegisterPeer(NodeId peer_id, bool peer_inbound, bool recon_initiator,
    68+                                     bool recon_responder, uint32_t peer_recon_version, uint64_t remote_salt);
    


    jonatack commented at 12:30 pm on August 3, 2022:
    https://github.com/bitcoin/bitcoin/commit/b22525d2f785f2a8a9ec928dab25ea9fa0452213 naming nit, it might be clearer per usual convention to add an is_ prefix to the 3 boolean parameter names, i.e. s/recon_initiator/is_recon_initiator/. Note that the param names here are different from those in TxReconciliationTracker::Impl::RegisterPeer(), it’s unclear to me if this is intended.
  245. in src/node/txreconciliation.cpp:101 in 362fc47127 outdated
    86+
    87+    std::optional<bool> RegisterPeer(NodeId peer_id, bool peer_inbound, bool they_may_initiate,
    88+                                     bool they_may_respond, uint32_t peer_recon_version,
    89+                                     uint64_t remote_salt) EXCLUSIVE_LOCKS_REQUIRED(!m_txreconciliation_mutex)
    90+    {
    91+        LOCK(m_txreconciliation_mutex);
    


    jonatack commented at 12:36 pm on August 3, 2022:

    https://github.com/bitcoin/bitcoin/commit/b22525d2f785f2a8a9ec928dab25ea9fa0452213

    0     {
    1+        AssertLockNotHeld(m_txreconciliation_mutex);
    2         LOCK(m_txreconciliation_mutex);
    
  246. in src/node/txreconciliation.cpp:108 in 362fc47127 outdated
    93+
    94+        // A peer should be in the pre-registered state to proceed here.
    95+        if (recon_state == m_states.end()) return std::nullopt;
    96+        uint64_t* local_salt = std::get_if<uint64_t>(&recon_state->second);
    97+        // A peer is already registered. This should be checked by the caller.
    98+        Assume(local_salt);
    


    jonatack commented at 12:38 pm on August 3, 2022:

    https://github.com/bitcoin/bitcoin/commit/b22525d2f785f2a8a9ec928dab25ea9fa0452213 for Assume

    0 #include <node/txreconciliation.h>
    1 
    2+#include <util/check.h>
    3 #include <util/system.h>
    
  247. in src/node/txreconciliation.cpp:125 in 362fc47127 outdated
    120+
    121+        LogPrintLevel(BCLog::TXRECONCILIATION, BCLog::Level::Debug, "Register peer=%d with the following params: " /* Continued */
    122+                                                                    "we_initiate=%i, they_initiate=%i.\n",
    123+                      peer_id, we_initiate, they_initiate);
    124+
    125+        uint256 full_salt = ComputeSalt(*local_salt, remote_salt);
    


    jonatack commented at 12:41 pm on August 3, 2022:

    https://github.com/bitcoin/bitcoin/commit/b22525d2f785f2a8a9ec928dab25ea9fa0452213 type-safe, const

    0        const uint256 full_salt{ComputeSalt(*local_salt, remote_salt)};
    
  248. in src/node/txreconciliation.cpp:101 in 362fc47127 outdated
     96+        uint64_t* local_salt = std::get_if<uint64_t>(&recon_state->second);
     97+        // A peer is already registered. This should be checked by the caller.
     98+        Assume(local_salt);
     99+
    100+        // If the peer supports the version which is lower than ours, we downgrade to the version
    101+        // they support. For now, this only guarantees that nodes with future reconciliation
    


    jonatack commented at 12:46 pm on August 3, 2022:
    https://github.com/bitcoin/bitcoin/commit/b22525d2f785f2a8a9ec928dab25ea9fa0452213 nit s/they support/it supports/ “they” as used here is a bit confusing, as it means either multiple subjects (a group of peers) or one subject (a peer) while unnecessarily anthropomorphising a bitcoind software peer that can just more clearly be “it”
  249. jonatack commented at 1:00 pm on August 3, 2022: contributor
    WIP review currently up to and in https://github.com/bitcoin/bitcoin/commit/b22525d2f785f2a8a9ec928dab25ea9fa0452213, but I need to restart the computer so putting this up so as not to lose it.
  250. in src/net_processing.h:29 in 362fc47127 outdated
    22@@ -22,6 +23,10 @@ static const bool DEFAULT_PEERBLOOMFILTERS = false;
    23 static const bool DEFAULT_PEERBLOCKFILTERS = false;
    24 /** Threshold for marking a node to be discouraged, e.g. disconnected and added to the discouragement filter. */
    25 static const int DISCOURAGEMENT_THRESHOLD{100};
    26+/** Whether transaction reconciliation protocol should be enabled by default. */
    27+static const bool DEFAULT_TXRECONCILIATION_ENABLE{false};
    28+/** Supported transaction reconciliation protocol version */
    29+static constexpr uint32_t TXRECONCILIATION_VERSION{1};
    


    jonatack commented at 4:59 pm on August 4, 2022:

    7a5603c

    • I think it may make more sense to declare DEFAULT_TXRECONCILIATION_ENABLE and TXRECONCILIATION_VERSION in txreconciliation.h rather than in net_processing.cpp. I believe doing so would not require any other changes.

    • nit, DEFAULT_TXRECONCILIATION_ENABLE can be constexpr

  251. in src/node/txreconciliation.cpp:88 in 362fc47127 outdated
    83+        assert(m_states.emplace(peer_id, local_salt).second);
    84+        return local_salt;
    85+    }
    86+
    87+    std::optional<bool> RegisterPeer(NodeId peer_id, bool peer_inbound, bool they_may_initiate,
    88+                                     bool they_may_respond, uint32_t peer_recon_version,
    


    jonatack commented at 5:36 pm on August 4, 2022:

    https://github.com/bitcoin/bitcoin/commit/b22525d2f785f2a8a9ec928dab25ea9fa0452213 Suggest s/they_may_initiate/peer_may_initiate/ and s/they_may_respond/peer_may_respond/ for clearer param names (also peer_recon_version is used, not they_recon_version, so using peer_ prefixes throughout seems more consistent and clear to readers of the code than sometimes they_ and sometimes peer_)

    Note also that the param names here in TxReconciliationTracker::Impl::RegisterPeer() are currently different from the param names in TxReconciliationTracker::RegisterPeer(), unsure if that is intended.

  252. in src/node/txreconciliation.cpp:105 in 362fc47127 outdated
    100+        // If the peer supports the version which is lower than ours, we downgrade to the version
    101+        // they support. For now, this only guarantees that nodes with future reconciliation
    102+        // versions have the choice of reconciling with this current version. However, they also
    103+        // have the choice to refuse supporting reconciliations if the common version is not
    104+        // satisfactory (e.g. too low).
    105+        const uint32_t recon_version = std::min(peer_recon_version, m_recon_version);
    


    jonatack commented at 5:40 pm on August 4, 2022:

    https://github.com/bitcoin/bitcoin/commit/b22525d2f785f2a8a9ec928dab25ea9fa0452213 type-safe

    0        const uint32_t recon_version{std::min(peer_recon_version, m_recon_version)};
    
  253. in src/net_processing.cpp:3072 in 362fc47127 outdated
    3067+        if (m_txreconciliation) {
    3068+            if (!peer->m_wtxid_relay || !m_txreconciliation->IsPeerRegistered(pfrom.GetId())) {
    3069+                // We could have optimistically pre-registered/registered the peer.
    3070+                // Forget about the reconciliation state if either:
    3071+                // - peer haven't announced WTXIDRELAY
    3072+                // - OR peer haven't sent us SENDTXRCNCL
    


    jonatack commented at 5:59 pm on August 4, 2022:

    58cf445f s/peer haven’t/peer hasn’t/

    0-                // - peer haven't announced WTXIDRELAY
    1-                // - OR peer haven't sent us SENDTXRCNCL
    2+                // - peer hasn't announced WTXIDRELAY
    3+                // - OR peer hasn't sent us SENDTXRCNCL
    

    or can optionally be shorter

    0-                // Forget about the reconciliation state if either:
    1-                // - peer haven't announced WTXIDRELAY
    2-                // - OR peer haven't sent us SENDTXRCNCL
    3+                // Forget about the reconciliation state if peer hasn't announced WTXIDRELAY or sent us SENDTXRCNCL.
    
  254. in src/node/txreconciliation.cpp:111 in 362fc47127 outdated
    106+        // v1 is the lowest version, so suggesting something below must be a protocol violation.
    107+        if (recon_version < 1) return false;
    108+
    109+        // Must match SENDTXRCNCL logic.
    110+        bool they_initiate = they_may_initiate && peer_inbound;
    111+        bool we_initiate = !peer_inbound && they_may_respond;
    


    jonatack commented at 6:03 pm on August 4, 2022:
    https://github.com/bitcoin/bitcoin/commit/b22525d2f785f2a8a9ec928dab25ea9fa0452213 nit, these two bools can be const to clarify they won’t be changed
  255. in src/node/txreconciliation.cpp:45 in 362fc47127 outdated
    34+public:
    35+    /**
    36+     * Reconciliation protocol assumes using one role consistently: either a reconciliation
    37+     * initiator (requesting sketches), or responder (sending sketches). This defines our role.
    38+     * */
    39+    bool m_we_initiate;
    


    jonatack commented at 6:06 pm on August 4, 2022:
  256. jonatack commented at 6:23 pm on August 4, 2022: contributor

    Finished review of the non-test commits. Needs rebase, but it’s trivial.

    https://github.com/bitcoin/bitcoin/commit/7a5603c61b999ef66bdba7a7bfe1b4044dc502bc Would it make sense to move the TxReconciliationTracker::Impl class to its own file?

    58cf445f67 commit message: “p2p: clear txreconciliation state for no-wtxid peers” -> s/no/non/

  257. ariard commented at 11:13 pm on August 9, 2022: member
    @naumenkogs I believe this needs rebase. I’ll review back soon once current open comments are addressed.
  258. naumenkogs force-pushed on Aug 19, 2022
  259. naumenkogs force-pushed on Aug 19, 2022
  260. naumenkogs force-pushed on Aug 19, 2022
  261. DrahtBot removed the label Needs rebase on Aug 19, 2022
  262. naumenkogs force-pushed on Aug 22, 2022
  263. naumenkogs commented at 10:49 am on August 22, 2022: member
    @ariard @jonatack addressed all comments and rebased, please review :)
  264. DrahtBot added the label Needs rebase on Aug 30, 2022
  265. naumenkogs force-pushed on Aug 31, 2022
  266. DrahtBot removed the label Needs rebase on Aug 31, 2022
  267. DrahtBot added the label Needs rebase on Sep 1, 2022
  268. naumenkogs force-pushed on Sep 1, 2022
  269. DrahtBot removed the label Needs rebase on Sep 1, 2022
  270. jonatack commented at 12:02 pm on September 5, 2022: contributor
    Thanks for updating, re-reviewing.
  271. in src/net_processing.cpp:3419 in 3cf848124f outdated
    3410@@ -3388,6 +3411,17 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    3411             // they may wish to request compact blocks from us
    3412             m_connman.PushMessage(&pfrom, msgMaker.Make(NetMsgType::SENDCMPCT, /*high_bandwidth=*/false, /*version=*/CMPCTBLOCKS_VERSION));
    3413         }
    3414+
    3415+        if (m_txreconciliation) {
    3416+            if (!peer->m_wtxid_relay || !m_txreconciliation->IsPeerRegistered(pfrom.GetId())) {
    3417+                // We could have optimistically pre-registered/registered the peer.
    3418+                // Forget about the reconciliation state if peer hasn't announced WTXIDRELAY
    3419+                // or sent us SENDTXRCNCL.
    


    ariard commented at 4:52 pm on September 5, 2022:
    I think we RegisterPeera peer on their SENDTXNRCNCL message reception, comment could be updated to drop out the pre-registration reference and make it clear both conditions must be checked: announcement of WTXIDRELAY and announcement of SENDTXRCNCL?

    naumenkogs commented at 9:02 am on September 6, 2022:

    I see the comment is not ideal, but i want to understand you concern fully before proceeding with the change.

    To help me better understand your concern, could you provide a minimal diff in the comment text you’d suggest?


    ariard commented at 7:01 pm on September 8, 2022:

    Something like:

    “We could have optimistically pre-registered the peer. Forget about the reconciliation state if peer hasn’t announced WTXIDRELAY and sent us SENDTXRCN”

    Comment matching exactly the code from my understanding.

  272. ariard commented at 5:00 pm on September 5, 2022: member

    SGTM 3cf84812

    WIll proceed to manual testing soon.

  273. in src/node/txreconciliation.h:76 in 3cf848124f outdated
    69+     * - true if the peer was registered
    70+     * - false if the peer violates the protocol
    71+     * - nullopt if nothing was done (e.g., we haven't pre-registered this peer)
    72+     */
    73+    std::optional<bool> RegisterPeer(NodeId peer_id, bool is_peer_inbound, bool is_peer_recon_initiator,
    74+                                     bool is_peer_recon_responder, uint32_t peer_recon_version, uint64_t remote_salt);
    


    jonatack commented at 10:35 am on September 10, 2022:

    a0f1d51381

    • #23443 (review) doesn’t seem addressed yet.

    • nit, param naming mismatch between declaration and definition

     0diff --git a/src/node/txreconciliation.cpp b/src/node/txreconciliation.cpp
     1index 723cd88aa2..7f60c0c390 100644
     2--- a/src/node/txreconciliation.cpp
     3+++ b/src/node/txreconciliation.cpp
     4@@ -165,10 +165,10 @@ uint64_t TxReconciliationTracker::PreRegisterPeer(NodeId peer_id)
     5 }
     6
     7 std::optional<bool> TxReconciliationTracker::RegisterPeer(NodeId peer_id, bool is_peer_inbound,
     8-                                                          bool is_recon_initiator, bool is_recon_responder,
     9+                                                          bool is_peer_recon_initiator, bool is_peer_recon_responder,
    10                                                           uint32_t peer_recon_version, uint64_t remote_salt)
    11 {
    12-    return m_impl->RegisterPeer(peer_id, is_peer_inbound, is_recon_initiator, is_recon_responder,
    13+    return m_impl->RegisterPeer(peer_id, is_peer_inbound, is_peer_recon_initiator, is_peer_recon_responder,
    14                                 peer_recon_version, remote_salt);
    15 }
    
  274. in src/test/txreconciliation_tests.cpp:86 in 3cf848124f outdated
    81+
    82+    tracker.ForgetPeer(peer_id0);
    83+    BOOST_CHECK(!tracker.IsPeerRegistered(peer_id0));
    84+}
    85+
    86+BOOST_AUTO_TEST_SUITE_END()
    


    jonatack commented at 10:37 am on September 10, 2022:

    7d18db7f8b nit, missing newline at end of file

    0diff --git a/src/test/txreconciliation_tests.cpp b/src/test/txreconciliation_tests.cpp
    1-BOOST_AUTO_TEST_SUITE_END()
    2\ No newline at end of file
    3+BOOST_AUTO_TEST_SUITE_END()
    
  275. jonatack commented at 10:42 am on September 10, 2022: contributor
    Work-in-progress ACK: Diff-reviewed, debug-built and ran all tests for each commit. Did some initial testing on mainnet with ./src/bitcoind -txreconciliation -debug=txreconciliation. To-do: review the tests and verify behavior.
  276. in src/logging.h:38 in f88fa3db93 outdated
    58-        QT          = (1 << 19),
    59-        LEVELDB     = (1 << 20),
    60-        VALIDATION  = (1 << 21),
    61-        I2P         = (1 << 22),
    62-        IPC         = (1 << 23),
    63+        NONE             = 0,
    


    sipa commented at 8:09 pm on September 15, 2022:

    In commit “log: Add tx reconciliation log category”

    Is it worth changing the indentation of all the existing log categories if you’re not going to align TXRECONCILIATION with it?

    I’d suggest not touching them, and leaving TXRECONSILIATION longer, or shortening it to TXRECON?

  277. in src/net_processing.cpp:1775 in 18599f32b6 outdated
    1773 }
    1774 
    1775 PeerManagerImpl::PeerManagerImpl(CConnman& connman, AddrMan& addrman,
    1776                                  BanMan* banman, ChainstateManager& chainman,
    1777-                                 CTxMemPool& pool, bool ignore_incoming_txs)
    1778+                                 CTxMemPool& pool, std::unique_ptr<TxReconciliationTracker> txreconciliation,
    


    sipa commented at 8:18 pm on September 15, 2022:

    In commit “p2p: Announce reconciliation support”

    Is it necessary to let the constructor caller decide whether tx reconciliation will be used? Can’t PeerManagerImpl decide this on its own? That would avoid some complexity in the constructor and ::make arguments.

  278. in src/net_processing.cpp:3271 in 3cf848124f outdated
    3264@@ -3261,6 +3265,25 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    3265             if (fRelay) pfrom.m_relays_txs = true;
    3266         }
    3267 
    3268+        if (greatest_common_version >= WTXID_RELAY_VERSION && m_txreconciliation) {
    


    sipa commented at 9:18 pm on September 15, 2022:
    Is there no new protocol version number to gating the new txrecon messages?

    naumenkogs commented at 6:50 am on September 16, 2022:
    Currently no. Do you see any benefit in doing so?

    sipa commented at 2:24 pm on September 16, 2022:
    Yeah, it seems that some non-Bitcoin Core nodes treat the presence of unknown messages on the wire as a protocol violation (Bitcoin Core ignores them). There was a small debate around this during BIP155 introduction, and it seems they expected a protocol bump whenever new messages are introduced to avoid this (which is historically how protocol changes with new messages have pretty much always been done).

    maflcko commented at 2:43 pm on September 16, 2022:
    Is this still the case? At least btcd will not require a protocol bump, reading https://github.com/btcsuite/btcd/blob/38ee9a41c8f8aa24a079a28f5e8a86faecffdfe1/wire/protocol.go#L57-L59 . Are there any others?

    sipa commented at 2:55 pm on September 16, 2022:
    I believe libbitcoin makes a similar assumption: https://github.com/libbitcoin/libbitcoin-network/wiki/Protocol-Versioning

    naumenkogs commented at 8:49 am on September 19, 2022:

    I am low-key leaning towards not bumping the version following the BIP155 path. This PR got several acks already, which also means some sort of (at least soft) approval from those reviewers. Might be not a big deal, but I think I already went 3 times back-and-forth with this issue.

    I will keep this thread open for those willing to insist on the version bump. @sipa or anyone please let me know if not bumping prevents you from acking.

    This can also be changed in the future, anyone can open a PR to the repo (or the BIP).


    sipa commented at 2:06 pm on September 19, 2022:
    @naumenkogs No, that seems reasonable. In fact I think it’s better to use arbitrary-messages-before-verack as a negotiation mechanism than the necessarily serial protocol versioning, but I was concerned this could create issues for peers.

    ajtowns commented at 6:16 am on October 7, 2022:
    How about we actually document the messages-before-verack idea as a standard then? Here’s a draft https://github.com/ajtowns/bips/blob/202210-p2pfeatures/bip-p2pfeatures.mediawiki

    naumenkogs commented at 7:17 am on October 7, 2022:
    Looks good, I think it’s worth opening a PR to the bips repo?

    ajtowns commented at 8:16 am on October 7, 2022:
  279. sipa commented at 9:25 pm on September 15, 2022: member
    Concept ACK
  280. naumenkogs force-pushed on Sep 19, 2022
  281. naumenkogs force-pushed on Sep 19, 2022
  282. in src/node/txreconciliation.cpp:82 in e70f4c3f39 outdated
    39+public:
    40+    explicit Impl() {}
    41+
    42+    uint64_t PreRegisterPeer(NodeId peer_id) EXCLUSIVE_LOCKS_REQUIRED(!m_txreconciliation_mutex)
    43+    {
    44+        AssertLockNotHeld(m_txreconciliation_mutex);
    


    mzumsande commented at 9:10 pm on September 22, 2022:
    I don’t know too much about the locking code, but why use both EXCLUSIVE_LOCKS_REQUIRED(!m_txreconciliation_mutex) and AssertLockNotHeld(m_txreconciliation_mutex) - don’t both of these ensure basically the same thing?

    jonatack commented at 5:05 pm on September 26, 2022:
    My understanding (someone correct if inaccurate) is that these are complementary; the annotation does compile-time Clang thread safety analysis (configure.ac adds -Wthread-safety-analysis when available), and the assertion performs a belt-and-suspenders check at runtime. See also src/threadsafety.h, src/sync.{h,cpp} and the developer notes “Threads and synchronization” section and code examples.
  283. in src/test/txreconciliation_tests.cpp:48 in 8273066ae8 outdated
    43+    BOOST_REQUIRE(tracker.RegisterPeer(1, true, true, false, 2, salt) == ReconciliationRegisterResult::SUCCESS);
    44+    BOOST_CHECK(tracker.IsPeerRegistered(1));
    45+
    46+    // Do not register if there were no pre-registration for the peer.
    47+    BOOST_REQUIRE(tracker.RegisterPeer(100, true, true, false, 1, salt) == ReconciliationRegisterResult::NOT_FOUND);
    48+    BOOST_CHECK(!tracker.IsPeerRegistered(100));
    


    mzumsande commented at 9:55 pm on September 22, 2022:
    nit: Could also add a test case for successfully registering an outbound peer.
  284. in src/net_processing.cpp:3506 in e91690e67d outdated
    3500+            LogPrintLevel(BCLog::NET, BCLog::Level::Debug, "sendtxrcncl received after verack from peer=%d; disconnecting\n", pfrom.GetId());
    3501+            pfrom.fDisconnect = true;
    3502+            return;
    3503+        }
    3504+
    3505+        if (!peer->GetTxRelay()) {
    


    mzumsande commented at 3:59 pm on September 23, 2022:

    Maybe if (!peer->GetTxRelay() || m_ignore_incoming_txs)

    If we are in blocksonly mode, we still create a peer->GetTxRelay() object but indicated in our version message that we don’t support transaction relay. So if our peer sent us a sendtxrcncl, we would currently ignore tx reconciliation instead of disconnecting (because we don’t send a sendtxrcncl msg ourselves) - which isn’t bad but doesn’t follow the “Must not be sent if peer specified no support for transaction relay (fRelay=0) in “version”. Otherwise, the sender should be disconnected.” clause from the BIP.


    vasild commented at 11:01 am on October 7, 2022:

    Yeah, the code and the comments disagree:

    0if (!peer->GetTxRelay()) {
    1    // Disconnect peers that send a SENDTXRCNCL message even though we indicated we don't
    2    // support transaction relay.
    

    We set txrelay (thus GetTxRelay() will be true) on this condition:

    0!pfrom.IsBlockOnlyConn() && (fRelay || (peer->m_our_services & NODE_BLOOM)) // fRelay comes from the peer
    

    but we indicate transaction relay support on this condition:

    0!m_ignore_incoming_txs && !pnode.IsBlockOnlyConn() && !pnode.IsFeelerConn();
    

    They are not the same.

    Also noted by @jonatack here.

  285. mzumsande commented at 5:04 pm on September 23, 2022: contributor

    ACK e91690e67dad180c7fb9bed0409a9c4567d3e5df

    Haven’t had a deeper look in a while, but I think this looks really good, and the tests for the added logic are exhaustive. I only have some minor, non-blocking comments below.

  286. in test/functional/p2p_sendtxrcncl.py:127 in e91690e67d outdated
    122+        sendtxrcncl_wrong_role = create_sendtxrcncl_msg(initiator=False)
    123+        peer = self.nodes[0].add_p2p_connection(PeerNoVerack(), send_version=True, wait_for_verack=False)
    124+        peer.send_message(sendtxrcncl_wrong_role)
    125+        peer.wait_for_disconnect()
    126+
    127+        self.log.info('SENDTXRCNCL with version=0 triggers a disconnect')
    


    ariard commented at 5:14 pm on September 26, 2022:
    Could test the lawful case where version>=1 to ensure nodes deployed do not disconnect accidentally eventual reconciliation overhaul peers in the future.
  287. in src/node/txreconciliation.h:23 in e91690e67d outdated
    18+
    19+enum ReconciliationRegisterResult {
    20+    NOT_FOUND = 0,
    21+    SUCCESS = 1,
    22+    PROTOCOL_VIOLATION = 2,
    23+};
    


    ariard commented at 5:14 pm on September 26, 2022:
    Could be documented.
  288. ariard approved
  289. ariard commented at 5:42 pm on September 26, 2022: member

    Tested ACK e91690e

    Code changes since last review:

    • introducing ReconciliationRegisterResult and consequent comment clarification
    • txreconciliation setting initialized in PeerManagerImpl

    Mutated the functional test (p2p_sendtxrcncl.py) manually to exercise coverage:

    • SENDTXRCNCL flags settings L129 in txreconciliation.cpp, tested L65
    • SENDTXRCNCL before VERACK L3498 in net_processing.cpp, tested L73
    • SENDTXRCNCL without WTXIDRELAY L3418 in net_processing.cpp, tested L142
    • SENDTXRCNCL sent pre-WTXID L3270 in net_processing.cpp, tested L81
    • SENDTXRCNCL not sent on fRelay=false L3272 in net_processing.cpp, tested L93
    • duplicated SENDTXRCNCL L3518 in net_processing.cpp, tested L105
    • incorrect SENDTXRCNCL flag settings initiator=responder=0 L129 in net_processing.cpp, tested L113
    • incorrect SENDTXRCNCL flag settings initiator=0;responder=1 L129 in net_processing.cpp, tested L121
    • SENDTXRCNCL version=0 L117 in txreconciliation.cpp, tested L127
    • SENDTXRCNCL not sent if blocksonly L3276 in net_processing.cpp, tested L183
    • SENDTXRCNCL not sent if block-relay-only L3260 in net_processing.cpp, tested L158
    • SENDTXRCNCL should not be received if block-relay-only L3505 in net_processing.cpp, tested L164
    • SENDTXRCNCL not sent if -txreconciliation flag is not set L3493, tested L177

    Comments left are minor and can be deferred to a follow-up, I think.

  290. vasild commented at 11:29 am on September 29, 2022: contributor

    This is based on the unofficial BIP330 at naumenkogs/bips. It would be good to officialize it to bitcoin/bips before merge. Otherwise we would have the Bitcoin Core disagree with the (official) spec. Could get messy if other implementations implement this based on bitcoin/bips.

    I can open a PR against bitcoin/bips with the 3 extra commits that are in naumenkogs/bips, any objections? Or @naumenkogs, do you want to do that or maybe squash them in 1 commit?

  291. sipa commented at 6:28 pm on September 29, 2022: member
    ACK e91690e67dad180c7fb9bed0409a9c4567d3e5df
  292. dergoegge commented at 10:18 am on September 30, 2022: member

    Code review ACK e91690e67dad180c7fb9bed0409a9c4567d3e5df

    Adding a fuzz target for TxReconciliationTracker could be worth it (not necessary in this PR), especially once we add more functionality besides registering/forgetting peers.

    Thank you @naumenkogs for maintaining this PR for such a long time! I will commit more of my time on review for the next couple PRs.

  293. fanquake commented at 2:59 pm on October 2, 2022: member
    I think a good time to merge this could be after we’ve tagged a 24.0rc2.
  294. in src/node/txreconciliation.h:22 in e91690e67d outdated
    17+static constexpr uint32_t TXRECONCILIATION_VERSION{1};
    18+
    19+enum ReconciliationRegisterResult {
    20+    NOT_FOUND = 0,
    21+    SUCCESS = 1,
    22+    PROTOCOL_VIOLATION = 2,
    


    jonatack commented at 10:02 am on October 5, 2022:

    In commit “p2p: Finish negotiating reconciliation support”, thank you for switching to an enum to represent these states. If you retouch, suggest using an enum class (as also recommended in the developer notes) and optionally there is no need to set the enumerator values after the first one.

    0-enum ReconciliationRegisterResult {
    1+enum class ReconciliationRegisterResult {
    2     NOT_FOUND = 0,
    3-    SUCCESS = 1,
    4-    PROTOCOL_VIOLATION = 2,
    5+    SUCCESS,
    6+    PROTOCOL_VIOLATION,
    7 };
    
     0--- a/src/node/txreconciliation.cpp
     1+++ b/src/node/txreconciliation.cpp
     2@@ -102,7 +102,7 @@ public:
     3         auto recon_state = m_states.find(peer_id);
     4 
     5         // A peer should be in the pre-registered state to proceed here.
     6-        if (recon_state == m_states.end()) return NOT_FOUND;
     7+        if (recon_state == m_states.end()) return ReconciliationRegisterResult::NOT_FOUND;
     8         uint64_t* local_salt = std::get_if<uint64_t>(&recon_state->second);
     9         // A peer is already registered. This should be checked by the caller.
    10         Assume(local_salt);
    11@@ -114,7 +114,7 @@ public:
    12         // satisfactory (e.g. too low).
    13         const uint32_t recon_version{std::min(peer_recon_version, m_recon_version)};
    14         // v1 is the lowest version, so suggesting something below must be a protocol violation.
    15-        if (recon_version < 1) return PROTOCOL_VIOLATION;
    16+        if (recon_version < 1) return ReconciliationRegisterResult::PROTOCOL_VIOLATION;
    17 
    18         // Must match SENDTXRCNCL logic.
    19         const bool they_initiate = is_peer_recon_initiator && is_peer_inbound;
    20@@ -126,7 +126,7 @@ public:
    21         assert(!(they_initiate && we_initiate));
    22 
    23         // The peer set both flags to false, we treat it as a protocol violation.
    24-        if (!(they_initiate || we_initiate)) return PROTOCOL_VIOLATION;
    25+        if (!(they_initiate || we_initiate)) return ReconciliationRegisterResult::PROTOCOL_VIOLATION;
    26 
    27         LogPrintLevel(BCLog::TXRECONCILIATION, BCLog::Level::Debug, "Register peer=%d with the following params: " /* Continued */
    28                                                                     "we_initiate=%i, they_initiate=%i.\n",
    29@@ -134,7 +134,7 @@ public:
    30 
    31         const uint256 full_salt{ComputeSalt(*local_salt, remote_salt)};
    32         recon_state->second = TxReconciliationState(we_initiate, full_salt.GetUint64(0), full_salt.GetUint64(1));
    33-        return SUCCESS;
    34+        return ReconciliationRegisterResult::SUCCESS;
    35     }
    
  295. in test/functional/p2p_sendtxrcncl.py:78 in e91690e67d outdated
    73+        self.log.info('SENDTXRCNCL should be sent before VERACK')
    74+        peer = self.nodes[0].add_p2p_connection(PeerTrackMsgOrder(), send_version=True, wait_for_verack=True)
    75+        peer.wait_for_verack()
    76+        verack_index = [i for i, msg in enumerate(peer.messages) if msg.msgtype == b'verack'][0]
    77+        sendtxrcncl_index = [i for i, msg in enumerate(peer.messages) if msg.msgtype == b'sendtxrcncl'][0]
    78+        assert(sendtxrcncl_index < verack_index)
    


    jonatack commented at 10:06 am on October 5, 2022:

    nit, in commit “test: Add functional tests for sendtxrcncl from inbound,” this line doesn’t raise in the CI linter yet because it uses an older version of flake8, but it does raise when running ./test/lint/lint-python.py locally with up-to-date dependencies.

    0        assert sendtxrcncl_index < verack_index
    
  296. in src/net_processing.cpp:3283 in e91690e67d outdated
    3277+                const uint64_t recon_salt = m_txreconciliation->PreRegisterPeer(pfrom.GetId());
    3278+                // We suggest our txreconciliation role (initiator/responder) based on
    3279+                // the connection direction.
    3280+                m_connman.PushMessage(&pfrom, msg_maker.Make(NetMsgType::SENDTXRCNCL,
    3281+                                                             !pfrom.IsInboundConn(),
    3282+                                                             pfrom.IsInboundConn(),
    


    jonatack commented at 10:38 am on October 5, 2022:

    In commit “p2p: Announce reconciliation support”, minor doc suggestion if you retouch

    0-                      !pfrom.IsInboundConn(),
    1-                      pfrom.IsInboundConn(),
    2+                      !pfrom.IsInboundConn(), // can initiate txreconciliation
    3+                      pfrom.IsInboundConn(),  // can receive txreconciliation
    
  297. in src/net_processing.cpp:3509 in e91690e67d outdated
    3503+        }
    3504+
    3505+        if (!peer->GetTxRelay()) {
    3506+            // Disconnect peers that send a SENDTXRCNCL message even though we indicated we don't
    3507+            // support transaction relay.
    3508+            LogPrintLevel(BCLog::NET, BCLog::Level::Debug, "sendtxrcncl received from peer=%d to which we indicated no tx relay; disconnecting\n", pfrom.GetId());
    


    jonatack commented at 11:04 am on October 5, 2022:

    We only initialize the Peer::TxRelay#m_relay_txs data structure if this isn’t an outbound block-relay-only connection, and if -fRelay=true (the peer wishes to receive transaction announcements) or we’re offering NODE_BLOOM to this peer (meaning that the peer may turn on transaction relay later). So peer->GetTxRelay() could be false independently of what we indicate (see df660ddb1cce1ee330346fe1728d868f41ad0256). Suggestion:

    0-            // Disconnect peers that send a SENDTXRCNCL message even though we indicated we don't
    1-            // support transaction relay.
    2-            LogPrintLevel(BCLog::NET, BCLog::Level::Debug, "sendtxrcncl received from peer=%d to which we indicated no tx relay; disconnecting\n", pfrom.GetId());
    3+            // Disconnect peers that send a SENDTXRCNCL message even though we don't relay transactions to this peer.
    4+            LogPrintLevel(BCLog::NET, BCLog::Level::Debug, "sendtxrcncl received from peer=%d with which we don't relay transactions; disconnecting\n", pfrom.GetId());
    

    (Could possibly drop the two comments that echo what the log prints, here and for verack just above.)


    naumenkogs commented at 11:30 am on October 19, 2022:
    moving this discussion here
  298. jonatack commented at 11:27 am on October 5, 2022: contributor

    Been running thanks to @vasild a sanity check on the upper bound of inv bandwidth improvement on my node topology using additional logging he wrote to look at redundant INV traffic received (among a number of stats that we added to getnettotals). We’re discussing the code and results with @naumenkog, but on my topology over a few days, there seems to be a good potential improvement in the redundant inv traffic received: ~89-91% seems redundant after stabilizing, or a third of total bytes received (this may vary greatly between nodes).

    Some non-blocking comments follow while WIP re-reviewing.

  299. in src/node/txreconciliation.cpp:129 in e91690e67d outdated
    124+        // tie-breaking. For now, this is mutually exclusive because both are based on the
    125+        // inbound flag.
    126+        assert(!(they_initiate && we_initiate));
    127+
    128+        // The peer set both flags to false, we treat it as a protocol violation.
    129+        if (!(they_initiate || we_initiate)) return PROTOCOL_VIOLATION;
    


    vasild commented at 11:19 am on October 6, 2022:

    The comment “The peer set both flags to false” is wrong because even if the peer sent initiator=false responder=true and he is the initiator of the P2P connection (inbound from our point of view), then this condition will be true. Disconnecting in this case is not mentioned in BIP 330.

    In practice this code enforces that the initiator of the TCP connection is also the reconciliation initiator and is not a responder.

    I find the assert() dangerous because it involves some relatively non-obvious logic in which 3 variables are involved and 2 of them are provided by the (possibly malicious) peer. Now is ok, but I see this on the brink of having a remotely triggered assert after some refactor or a subtle code change elsewhere in the code to e.g. allow the peer to influence is_peer_inbound. Maybe just disconnect in this case, or consider dropping the two booleans altogether: https://github.com/bitcoin/bips/pull/1376.


    naumenkogs commented at 12:41 pm on October 6, 2022:

    Yes, the comment seems wrong, in the sense it’s not covering this case of disconnection you mentioned. This disconnection is also not covered by the BIP.

    A minimal change would be to update the comment (perhaps acceptable to do it in the follow-up?). Disconnecting instead of asserting here is probably also a good idea.

    As for the dropping booleans, I suggest continuing the discussion in the bips PR you linked.


    vasild commented at 2:55 pm on October 6, 2022:

    Yes, I think this is non-blocker. Maybe something like this better describes it:

    0// The peer set both flags to false, or just initiator=false and it is an inbound peer, we treat it as a protocol violation.
    
  300. vasild commented at 12:07 pm on October 6, 2022: contributor
    Approach ACK e91690e67dad180c7fb9bed0409a9c4567d3e5df
  301. in src/init.cpp:488 in e91690e67d outdated
    485@@ -485,6 +486,7 @@ void SetupServerArgs(ArgsManager& argsman)
    486     argsman.AddArg("-seednode=<ip>", "Connect to a node to retrieve peer addresses, and disconnect. This option can be specified multiple times to connect to multiple nodes.", ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    487     argsman.AddArg("-networkactive", "Enable all P2P network activity (default: 1). Can be changed by the setnetworkactive RPC command", ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    488     argsman.AddArg("-timeout=<n>", strprintf("Specify socket connection timeout in milliseconds. If an initial attempt to connect is unsuccessful after this amount of time, drop it (minimum: 1, default: %d)", DEFAULT_CONNECT_TIMEOUT), ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    489+    argsman.AddArg("-txreconciliation", strprintf("Enable transaction reconciliations per BIP 330 (default: %d)", DEFAULT_TXRECONCILIATION_ENABLE), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::CONNECTION);
    


    vasild commented at 9:00 am on October 7, 2022:

    nit: “per BIP 330” looks too opaque for this help text, it is too long and technical to read the BIP itself in order to understand what this option is about.

    0    argsman.AddArg("-txreconciliation", strprintf("Enable transaction reconciliations for more efficient transaction propagation (default: %d)", DEFAULT_TXRECONCILIATION_ENABLE), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::CONNECTION);
    

    There are other BIP xyz mentions in the help texts, feel free to ignore.

  302. in src/node/txreconciliation.cpp:92 in e91690e67d outdated
    87+        LogPrintLevel(BCLog::TXRECONCILIATION, BCLog::Level::Debug, "Pre-register peer=%d\n", peer_id);
    88+        const uint64_t local_salt{GetRand(UINT64_MAX)};
    89+
    90+        // We do this exactly once per peer (which are unique by NodeId, see GetNewNodeId) so it's
    91+        // safe to assume we don't have this record yet.
    92+        Assert(m_states.emplace(peer_id, local_salt).second);
    


    vasild commented at 9:43 am on October 7, 2022:

    You can remove this assert() from above:

    0assert(m_states.find(peer_id) == m_states.end());
    

    They both check for the same thing. That way we will do one lookup instead of two, maybe insignificant wrt performance but will be less code.

    PS why use assert() in one place and Assert() in another? Did you intend to use Assume()?

  303. in src/node/txreconciliation.cpp:64 in e91690e67d outdated
    59+
    60+/** Actual implementation for TxReconciliationTracker's data structure. */
    61+class TxReconciliationTracker::Impl
    62+{
    63+private:
    64+    mutable Mutex m_txreconciliation_mutex;
    


    vasild commented at 10:32 am on October 7, 2022:

    nit: “txreconciliation” is already in the name of the class TxReconciliationTracker. This can be just m_mutex.

    For example, another member is called m_states, not m_txreconciliation_states.


    naumenkogs commented at 2:13 pm on October 20, 2022:
    This was the initial version, but then @jonatack asked for the more verbose name, and I didn’t care much… Maybe he has to say something.

    jonatack commented at 5:06 pm on October 20, 2022:
    The rationale is that m_txreconciliation_mutex returns more useful results than m_mutex when git grepping the codebase, and the more specific mutex naming seems to be a convention in the codebase as much as the generic naming.
  304. in src/node/txreconciliation.cpp:108 in e91690e67d outdated
    103+
    104+        // A peer should be in the pre-registered state to proceed here.
    105+        if (recon_state == m_states.end()) return NOT_FOUND;
    106+        uint64_t* local_salt = std::get_if<uint64_t>(&recon_state->second);
    107+        // A peer is already registered. This should be checked by the caller.
    108+        Assume(local_salt);
    


    vasild commented at 11:08 am on October 7, 2022:
    This should be assert() since we dereference local_salt below. It will crash anyway if it is nullptr. Better crash with a meaningful message produced by the assert().
  305. in src/test/txreconciliation_tests.cpp:15 in e91690e67d outdated
    10+
    11+BOOST_FIXTURE_TEST_SUITE(txreconciliation_tests, BasicTestingSetup)
    12+
    13+BOOST_AUTO_TEST_CASE(RegisterPeerTest)
    14+{
    15+    TxReconciliationTracker tracker(1);
    


    vasild commented at 11:20 am on October 7, 2022:
    Use TXRECONCILIATION_VERSION instead of 1?
  306. in src/test/txreconciliation_tests.cpp:22 in e91690e67d outdated
    17+
    18+    // Prepare a peer for reconciliation.
    19+    tracker.PreRegisterPeer(0);
    20+
    21+    // Both roles are false, don't register.
    22+    BOOST_CHECK(tracker.RegisterPeer(/*peer_id=*/0, /*is_peer_inbound=*/true,
    


    vasild commented at 11:31 am on October 7, 2022:
    Here and elsewhere, BOOST_CHECK_EQUAL(x, y) is preferred over BOOST_CHECK(x == y) because the former gives a better message should the check fail.
  307. vasild approved
  308. vasild commented at 11:47 am on October 7, 2022: contributor

    ACK e91690e67dad180c7fb9bed0409a9c4567d3e5df

    Some non showstoppers below.

  309. vasild commented at 8:45 am on October 10, 2022: contributor

    In the OP:

    • the link to BIP 330 can be updated and this can be removed: “Note this is not what’s in the bitcoin/bips repo, but an updated version”.
    • s/sendrecon/sendtxrcncl
  310. DrahtBot added the label Needs rebase on Oct 13, 2022
  311. log: Add tx reconciliation log category 24e36fac0a
  312. p2p: Announce reconciliation support
    If we're connecting to the peer which might support
    transaction reconciliation, we announce we want to reconcile
    with them.
    
    We store the reconciliation salt so that when the peer
    responds with their salt, we are able to compute the
    full reconciliation salt.
    
    This behavior is enabled with a CLI flag.
    3fcf78ee6a
  313. p2p: Forget peer's reconciliation state on disconnect 4470acf076
  314. Add helper to see if a peer is registered for reconciliations 36cf6bf216
  315. p2p: Finish negotiating reconciliation support
    Once we received a reconciliation announcement support
    message from a peer and it doesn't violate our protocol,
    we store the negotiated parameters which will be used
    for future reconciliations.
    88d326c8e3
  316. p2p: clear txreconciliation state for non-wtxid peers
    We optimistically pre-register a peer for txreconciliations
    upon sending txreconciliation support announcement.
    But if, at VERACK, we realize that the peer never sent
    WTXIDRELAY message, we should unregister the peer
    from txreconciliations, because txreconciliations rely on wtxids.
    f63f1d3f4b
  317. test: Add unit tests for reconciliation negotiation b99ee9d22d
  318. test: Add functional tests for sendtxrcncl from inbound cfcef60779
  319. test: Add functional tests for sendtxrcncl message from outbound e56d1d2afd
  320. naumenkogs force-pushed on Oct 17, 2022
  321. glozow commented at 10:41 am on October 17, 2022: member
    Looks like rebased for #25667. Pinging @sipa @dergoegge @vasild @ariard @mzumsande for re-ACK
  322. DrahtBot removed the label Needs rebase on Oct 17, 2022
  323. vasild approved
  324. vasild commented at 11:43 am on October 17, 2022: contributor
    ACK e56d1d2afdd477be6dd462d838617d385bac5d7b
  325. vasild commented at 11:48 am on October 17, 2022: contributor
    The newly added test p2p_sendtxrcncl.py failed just on “Win64 native” :sob:
  326. dergoegge commented at 1:45 pm on October 17, 2022: member
    re-ACK e56d1d2afdd477be6dd462d838617d385bac5d7b
  327. sipa commented at 4:55 pm on October 17, 2022: member
    re-ACK e56d1d2afdd477be6dd462d838617d385bac5d7b. No differences with a rebase of previously reviewed e91690e67dad180c7fb9bed0409a9c4567d3e5df.
  328. glozow commented at 5:01 pm on October 17, 2022: member

    The newly added test p2p_sendtxrcncl.py failed just on “Win64 native” sob

    Job: https://cirrus-ci.com/task/6088863331385344 Reran and it turned green. Maybe coincidental timeout?

  329. maflcko commented at 5:04 pm on October 17, 2022: member

    The reason was likely

    node0 2022-10-17T10:37:19.451252Z [C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\src\net_processing.cpp:3460] [ProcessMessage] [net] wtxidrelay received after verack from peer=8; disconnecting

    https://cirrus-ci.com/task/6088863331385344?logs=functional_tests#L605

    However, I can’t see how this could possibly happen in the framework. (peer8 is a python node)

  330. mzumsande commented at 5:06 pm on October 17, 2022: contributor
    re-ACK e56d1d2afdd477be6dd462d838617d385bac5d7b
  331. glozow merged this on Oct 17, 2022
  332. glozow closed this on Oct 17, 2022

  333. naumenkogs commented at 7:00 pm on October 17, 2022: member
    Thank you all for making this happen! I will create a PR for fixups, and finish the second PR shortly.
  334. ariard commented at 11:16 pm on October 17, 2022: member
    Post-Merge ACK e91690e
  335. in src/net_processing.cpp:3277 in e56d1d2afd
    3272+            // Per BIP-330, we announce txreconciliation support if:
    3273+            // - protocol version per the VERSION message supports WTXID_RELAY;
    3274+            // - we intended to exchange transactions over this connection while establishing it
    3275+            //   and the peer indicated support for transaction relay in the VERSION message;
    3276+            // - we are not in -blocksonly mode.
    3277+            if (pfrom.m_relays_txs && !m_ignore_incoming_txs) {
    


    maflcko commented at 6:21 am on October 18, 2022:
    Is this going to send SENDTXRCNCL to feeler connections?
  336. in src/init.cpp:488 in e56d1d2afd
    484@@ -484,6 +485,7 @@ void SetupServerArgs(ArgsManager& argsman)
    485     argsman.AddArg("-seednode=<ip>", "Connect to a node to retrieve peer addresses, and disconnect. This option can be specified multiple times to connect to multiple nodes.", ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    486     argsman.AddArg("-networkactive", "Enable all P2P network activity (default: 1). Can be changed by the setnetworkactive RPC command", ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    487     argsman.AddArg("-timeout=<n>", strprintf("Specify socket connection timeout in milliseconds. If an initial attempt to connect is unsuccessful after this amount of time, drop it (minimum: 1, default: %d)", DEFAULT_CONNECT_TIMEOUT), ArgsManager::ALLOW_ANY, OptionsCategory::CONNECTION);
    488+    argsman.AddArg("-txreconciliation", strprintf("Enable transaction reconciliations per BIP 330 (default: %d)", DEFAULT_TXRECONCILIATION_ENABLE), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::CONNECTION);
    


    maflcko commented at 12:09 pm on October 18, 2022:
    Probably unrelated, but it might be good to split the connection section into net and net_processing options. (This one would go into net_processing)
  337. sidhujag referenced this in commit 233608ba8a on Oct 23, 2022
  338. bitcoin locked this on Oct 25, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-03 10:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me