private broadcast: limit outstanding txs to count of 1000 #35406

pull instagibbs wants to merge 2 commits into bitcoin:master from instagibbs:2026-05-private_cap changing 12 files +194 −18
  1. instagibbs commented at 3:03 PM on May 28, 2026: member

    Add a belt-and-suspenders feature, limit the amount of memory possible when unlucky or simply misconfigured. The worst case limit is roughly 400kB * 1000 = 400MB, regardless of usage pattern.

    Before this change, sheer volume of broadcasts, mismatches in standardness rules, or simply fee mismatches may result in unbounded growth of memory usage. As the feature may be expande in the future, explicit bounds helps reasoning going forward.

    Tests are a little repetitive of each other, but I like having them at both layers.

  2. DrahtBot commented at 3:03 PM on May 28, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35406.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Approach ACK stickies-v
    Stale ACK optout21

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #35315 (refactor: Use NodeClock::time_point in more places by maflcko)
    • #35252 (net: send decoy transactions via private broadcast by andrewtoth)
    • #35016 (net: deduplicate private broadcast state and snapshot types by takeshikurosawaa)
    • #34707 (net: keep finished private broadcast txs in memory by andrewtoth)
    • #34271 (net_processing: make m_tx_for_private_broadcast optional by vasild)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  3. instagibbs force-pushed on May 28, 2026
  4. fanquake commented at 3:14 PM on May 28, 2026: member

    Is this for backport?

  5. instagibbs commented at 3:20 PM on May 28, 2026: member

    @fanquake worthy of backport for safety and user expectations going forward imo

  6. fanquake added the label Needs Backport (31.x) on May 28, 2026
  7. fanquake requested review from vasild on May 28, 2026
  8. fanquake requested review from andrewtoth on May 28, 2026
  9. instagibbs force-pushed on May 28, 2026
  10. DrahtBot added the label CI failed on May 28, 2026
  11. instagibbs force-pushed on May 28, 2026
  12. in src/private_broadcast.cpp:24 in 65ad93bc12
      21 | +    m_transactions.try_emplace(tx, std::prev(m_insertion_order.end()));
      22 | +
      23 | +    if (m_transactions.size() > MAX_TRANSACTIONS) {
      24 | +        // Evict the oldest entry (FIFO). The front of m_insertion_order is the
      25 | +        // earliest-added tx; the just-inserted tx sits at the back.
      26 | +        const CTransactionRef oldest{m_insertion_order.front()};
    


    andrewtoth commented at 7:28 PM on May 28, 2026:

    I feel like this would be a lot simpler by just using the time_added field, instead of adding a parallel data structure? It would be an O(n) scan, but it's capped at MAX_TRANSACTIONs.

            const CTransactionRef oldest{std::ranges::min_element(
                m_transactions, {},
                [](const auto& el) { return el.second.time_added; })};
    

    instagibbs commented at 7:36 PM on May 28, 2026:

    Fair, if we're landing on 1000 cap or less I'll simplify.


    andrewtoth commented at 7:39 PM on May 28, 2026:

    Even 10k should be fine. It's not performance sensitive. This is guarding an edge case. I don't think anyone would hit this limit in practice.


    instagibbs commented at 8:05 PM on May 28, 2026:

    done (right, obviously overkill for something that only happens once per rpc submission! not a recurring task)

  13. instagibbs force-pushed on May 28, 2026
  14. instagibbs force-pushed on May 28, 2026
  15. in src/test/private_broadcast_tests.cpp:164 in de1059f51b
     156 | @@ -157,4 +157,59 @@ BOOST_AUTO_TEST_CASE(stale_unpicked_tx)
     157 |      BOOST_CHECK_EQUAL(stale_state[0], tx);
     158 |  }
     159 |  
     160 | +BOOST_AUTO_TEST_CASE(fifo_eviction_at_cap)
     161 | +{
     162 | +    PrivateBroadcast pb;
     163 | +    constexpr size_t kCap{PrivateBroadcast::MAX_TRANSACTIONS};
     164 | +    constexpr size_t kOver{5};
    


    andrewtoth commented at 9:30 PM on May 28, 2026:

    style nit: don't use hungarian notation.


    instagibbs commented at 1:29 PM on May 29, 2026:

    done

  16. in src/rpc/mempool.cpp:60 in de1059f51b outdated
      59 | @@ -60,6 +60,9 @@ static RPCMethod sendrawtransaction()
      60 |          "dedicated, short-lived connections to Tor or I2P peers or IPv4/IPv6 peers\n"
    


    andrewtoth commented at 9:31 PM on May 28, 2026:

    In commit: de1059f51bfbebc5f59423ee8e23608093381c92

    The commit message:

    As the feature may be expande in the future, explicity bounds helps reasoning going forward.

    Should be:

    As the feature may be expanded in the future, explicit bounds helps reasoning going forward.


    optout21 commented at 7:39 AM on May 29, 2026:

    The same applies to the PR description as well.


    instagibbs commented at 1:29 PM on May 29, 2026:

    done

  17. DrahtBot removed the label CI failed on May 28, 2026
  18. in test/functional/p2p_private_broadcast_cap.py:22 in de1059f51b outdated
      17 | +
      18 | +class PrivateBroadcastCapTest(BitcoinTestFramework):
      19 | +    def set_test_params(self):
      20 | +        self.num_nodes = 1
      21 | +        # -privatebroadcast is incompatible with the framework's default
      22 | +        # -connect=0; allow autoconnect (no actual peers will succeed though).
    


    andrewtoth commented at 9:33 PM on May 28, 2026:

    We can use the -maxconnections=0 workaround for this if needed (not sure it's needed).


    instagibbs commented at 1:29 PM on May 29, 2026:

    leaving as is

  19. in src/private_broadcast.cpp:19 in de1059f51b outdated
      13 | @@ -12,8 +14,25 @@ bool PrivateBroadcast::Add(const CTransactionRef& tx)
      14 |      EXCLUSIVE_LOCKS_REQUIRED(!m_mutex)
      15 |  {
      16 |      LOCK(m_mutex);
      17 | -    const bool inserted{m_transactions.try_emplace(tx).second};
      18 | -    return inserted;
      19 | +    if (!m_transactions.try_emplace(tx).second) return false;
      20 | +
      21 | +    if (m_transactions.size() > MAX_TRANSACTIONS) {
    


    optout21 commented at 7:45 AM on May 29, 2026:

    de1059f private broadcast: limit outstanding txs to count of 1000:

    The implicit assumption here is that size exceeds max by exactly one. This is true, for as long as adding only happens only through Add, and never simultaneously. Changing the if to while would be more robust -- it's not really needed, but it's worth considering.


    instagibbs commented at 1:29 PM on May 29, 2026:

    We have the trailing Assume() that would catch this in debug builds, I think it's easiest to think about this way? Leaving as is.

  20. optout21 commented at 8:36 AM on May 29, 2026: contributor

    ACK 401459243159e34fed93dc3ae1ec19c1c19075fc

    Reviewed code, executed tests locally, LGTM.

    This PR adds a safety-net max limit for the private broadcast queue. While it can be argued that over-use is not very likely and not very catastrophic (usage through protected RPC interface, modest expected regular usage, reduced memory usage), it is good practice to have safety limits in place. It makes the risk explicit (however small), and introduces a "just in case" limit. The limit of 1000 transaction is mostly arbitrary number, and not configurable. This is perfectly acceptable, having in mind the low-probability, low-harm nature of the queue being full.

  21. instagibbs force-pushed on May 29, 2026
  22. in src/private_broadcast.cpp:26 in 9aeb4f8810
      23 | +        // expanded in scope.
      24 | +        const auto oldest{std::ranges::min_element(
      25 | +                m_transactions, {},
      26 | +                [](const auto& el) { return el.second.time_added; })};
      27 | +        const CTransactionRef& tx_to_evict{oldest->first};
      28 | +        LogDebug(BCLog::PRIVBROADCAST,
    


    stickies-v commented at 2:12 PM on May 29, 2026:

    I think this should be LogInfo at least?


    instagibbs commented at 2:35 PM on June 1, 2026:

    Now that the caller is getting an error, going to leave it as debug

  23. stickies-v commented at 2:19 PM on May 29, 2026: contributor

    Concept ACK for bounding resource usage.

    Approach-wise, did you consider letting sendrawtransaction error instead of dropping the oldest tx? When it fails, user can use abortprivatebroadcast to drop what they want, rather than us doing it silently.

  24. andrewtoth commented at 2:25 PM on May 29, 2026: contributor

    Approach-wise, did you consider letting sendrawtransaction error instead of dropping the oldest tx?

    Might be easiest way, just do a length check in Add and return false if max or higher.

  25. instagibbs commented at 2:35 PM on May 29, 2026: member

    Approach-wise, did you consider letting sendrawtransaction error instead of dropping the oldest tx? When it fails, user can use abortprivatebroadcast to drop what they want, rather than us doing it silently.

    In the misconfigured scenario, I think either strategy works just fine, we just don't want to crash and nothing is propagating for some reason.

    In the high volume scenario, where we were simply wrong about how users intend to use it (now or going forward), they will have to know about and able to handle this explicitly (I have low confidence that people are doing sensible things with our weird error reporting fwiw).

    One pro of explicit handling is that the caller will never be surprised that something ended up being dropped.

    I'm slightly predisposed to handling it implicitly and allowing naive rebroadcasting to "fix" things, but I can see it both ways.

  26. andrewtoth commented at 3:07 PM on May 29, 2026: contributor

    I think hitting this means something is misconfigured, so explicitly erroring will alert the user to the issue. I would be in favor of @stickies-v's approach.

  27. fanquake added the label Private Broadcast on May 29, 2026
  28. stickies-v commented at 3:20 PM on May 29, 2026: contributor

    they will have to know about and able to handle this explicitly

    Isn't that true regardless? When we silently drop, they'll need to (well, should) track / manage submitted txs and implement logic on how/when to retry the tx. Regardless, from an interface design pov, I think explicit is better than implicit.

    Not familiar with wallet, so just my 2c, won't push back on it further.

  29. instagibbs commented at 3:22 PM on May 29, 2026: member

    so explicitly erroring will alert the user to the issue.

    Just noting again to be clear: Outside of manual use I expect the error to simply be ignore/logged by many integrations, along with "already in mempool" kinda messages.

    I suppose with good docs, the error-ignoring user can simply check the broadcast set, evict whatever the like, and it will self-heal next attempt at broadcasting.

    I think hitting this means something is misconfigured

    We shouldn't ignore usage patterns that plausibly exist. Question is what is best when our assumptions are violated.

    Isn't that true regardless? When we silently drop, they'll need to (well, should) track / manage submitted txs and implement logic on how/when to retry the tx.

    The caller knows a tx isn't being confirmed by looking at blocks and calls sendrawtransaction again. No need to even look at error messages in normal operation with or without private broadcast.

    Regardless, from an interface design pov, I think explicit is better than implicit.

    Agreed. I think with good docs it can be made idiot proof. Let me get that implemented.

  30. vasild commented at 4:58 PM on May 29, 2026: contributor

    A few thoughts:

    • Just 1000? The OP says 400kB * 1000 = 400MB, but where did that 400kB come from? A typical transaction is a few hundred bytes. Plus some storage overhead, I think we can reasonably assume one transaction takes 500-1000 bytes of RAM in the private broadcast queue. So, in 400MB one can store about 400'000 transactions, conservatively.

    • I think it would be better to return an immediate error from the sendrawtransaction RPC because then the user (or the user program driving this):

    1. knows which transaction couldn't be sent - the one it got error about
    2. is more likely to have the transaction at hand so it can retry after a short wait for the queue to empty otherwise "some of the transactions you scheduled some time ago was dropped but you do not know which one". That is harder to handle.
    • I guess Bitcoin Core runs on a variety of hardware. Maybe some machines have 100x more RAM than others. It is not reasonable to assume that one number will fit all. So consider making this manually configurable via config option (eh :/) or set it internally based on the amount of RAM available.
  31. instagibbs commented at 5:26 PM on May 29, 2026: member

    Just 1000? The OP says 400kB * 1000 = 400MB, but where did that 400kB come from?

    it's the currently maximum sized transaction we will accept into our mempool (or private broadcast set). I think the simplest alternative is to have the serialized sized accounting instead of pure tx accounting, so the average case would be much higher and unlikely to be hit. Just increases the required testing a bit, and users rectifying it would have to abandon a variable amount of stuck broadcasts to make room for new ones.

    I think I'd rather bump this number a few X and keep the accounting / abandon flow as simple as possible.

    I guess Bitcoin Core runs on a variety of hardware. Maybe some machines have 100x more RAM than others. It is not reasonable to assume that one number will fit all. So consider making this manually configurable via config option (eh :/) or set it internally based on the amount of RAM available.

    eh same to me, would rather we keep things as simple as possible unless there is real demand for something other than a anti-footgun

  32. instagibbs force-pushed on Jun 1, 2026
  33. instagibbs commented at 1:47 PM on June 1, 2026: member

    Pushed an update where the caller is informed that the queue is full and FIFO is removed, as discussed.

  34. sedited requested review from andrewtoth on Jun 4, 2026
  35. sedited requested review from optout21 on Jun 4, 2026
  36. in src/private_broadcast.cpp:22 in b466ec5681 outdated
      20 | +    // Re-adding an already-tracked transaction is a no-op regardless of the cap.
      21 | +    if (m_transactions.contains(tx)) return AddResult::AlreadyPresent;
      22 | +
      23 | +    if (m_transactions.size() >= MAX_TRANSACTIONS) return AddResult::QueueFull;
      24 | +
      25 | +    m_transactions.try_emplace(tx);
    


    optout21 commented at 9:12 AM on June 4, 2026:

    b466ec5 private broadcast: limit outstanding txs to count of 1000:

    Nit: The return value of try_emplace is not checked here, though I don't see why it could fail. Maybe it could be assert-ed.


    instagibbs commented at 4:30 PM on June 4, 2026:

    It's building a pair with the key being default value. I don't think an assert gives us much given the check just above. Will leave as is unless others agree.

  37. optout21 commented at 9:21 AM on June 4, 2026: contributor

    ACK e96c343137f638b8eb096d262edc743695bc414d

    Rebase only.

    Prev: ACK 46b858ce7a43d39a28b85492c327602e8099a22d Re-review, new changes: The private queue full case is handled now as an explicit error, without implicitly deleting old entries. The error is propagated all the way through transaction broadcast RPC. This makes the case more explicit for callers, avoids the Core node to make policy decisions, but makes it more difficult for clients to handle it properly (i.e., delete old ones explicitly.). Overall I ACK this change.

    (Prev: #35406#pullrequestreview-4387609655)

  38. DrahtBot requested review from stickies-v on Jun 4, 2026
  39. fanquake commented at 4:20 PM on June 4, 2026: member

    Do you wanna rebase this now that #35410 is in?

  40. instagibbs force-pushed on Jun 4, 2026
  41. in doc/release-notes-35406.md:8 in e96c343137
       0 | @@ -0,0 +1,8 @@
       1 | +P2P and network changes
       2 | +-----------------------
       3 | +
       4 | +- The private-broadcast queue (transactions submitted via `sendrawtransaction`
       5 | +when `-privatebroadcast` is enabled and not yet echoed back from the network)
       6 | +is now capped at 1000 entries. When full, new submissions are rejected. It is
       7 | +up to the caller to inspect the queue via `getprivatebroadcastinfo` and free
       8 | +up space when stuck via `abandonprivatebroadcast`. (#35406)
    


    stickies-v commented at 1:22 PM on June 5, 2026:
    up space when stuck via `abortprivatebroadcast`. (#35406)
    

    instagibbs commented at 3:15 PM on June 8, 2026:

    done

  42. in src/rpc/mempool.cpp:65 in e96c343137
      59 | @@ -60,6 +60,9 @@ static RPCMethod sendrawtransaction()
      60 |          "dedicated, short-lived connections to Tor or I2P peers or IPv4/IPv6 peers\n"
      61 |          "via the Tor network. This conceals the transaction's origin. The transaction\n"
      62 |          "will only enter the local mempool when it is received back from the network.\n"
      63 | +        "The private broadcast queue is bounded: when it is full, this RPC fails and\n"
      64 | +        "the transaction is not scheduled, until an existing one completes or is\n"
      65 | +        "aborted. Use getprivatebroadcastinfo to inspect the queue.\n"
    


    stickies-v commented at 1:22 PM on June 5, 2026:
            "aborted. Use getprivatebroadcastinfo to inspect the queue and abortprivatebroadcast to abort.\n"
    

    instagibbs commented at 3:15 PM on June 8, 2026:

    done

  43. in src/node/transaction.cpp:137 in e96c343137
     132 | @@ -133,7 +133,9 @@ TransactionError BroadcastTransaction(NodeContext& node,
     133 |          node.peerman->InitiateTxBroadcastToAll(txid, wtxid);
     134 |          break;
     135 |      case TxBroadcast::NO_MEMPOOL_PRIVATE_BROADCAST:
     136 | -        node.peerman->InitiateTxBroadcastPrivate(tx);
     137 | +        if (!node.peerman->InitiateTxBroadcastPrivate(tx)) {
     138 | +            return TransactionError::PRIVATE_BROADCAST_FULL;
    


    stickies-v commented at 4:19 PM on June 5, 2026:

    It seems a bit awkward to compress the PrivateBroadcast::AddResult result into a bool to then later decompress it into a TransactionError again. Have you considered returning a TransactionError from InitiateTxBroadcastPrivate instead? Seems more upgradable too.

    I also think [[nodiscard]] would be good here.

    <details> <summary>git diff on e96c343137</summary>

    diff --git a/src/net_processing.cpp b/src/net_processing.cpp
    index 8b6bdba75c..bdc4292166 100644
    --- a/src/net_processing.cpp
    +++ b/src/net_processing.cpp
    @@ -540,7 +540,7 @@ public:
         std::vector<CTransactionRef> AbortPrivateBroadcast(const uint256& id) override EXCLUSIVE_LOCKS_REQUIRED(!m_peer_mutex);
         void SendPings() override EXCLUSIVE_LOCKS_REQUIRED(!m_peer_mutex);
         void InitiateTxBroadcastToAll(const Txid& txid, const Wtxid& wtxid) override EXCLUSIVE_LOCKS_REQUIRED(!m_peer_mutex);
    -    bool InitiateTxBroadcastPrivate(const CTransactionRef& tx) override EXCLUSIVE_LOCKS_REQUIRED(!m_peer_mutex);
    +    node::TransactionError InitiateTxBroadcastPrivate(const CTransactionRef& tx) override EXCLUSIVE_LOCKS_REQUIRED(!m_peer_mutex);
         void SetBestBlock(int height, std::chrono::seconds time) override
         {
             m_best_height = height;
    @@ -2264,20 +2264,20 @@ void PeerManagerImpl::InitiateTxBroadcastToAll(const Txid& txid, const Wtxid& wt
         }
     }
     
    -bool PeerManagerImpl::InitiateTxBroadcastPrivate(const CTransactionRef& tx)
    +node::TransactionError PeerManagerImpl::InitiateTxBroadcastPrivate(const CTransactionRef& tx)
     {
         const auto txstr{strprintf("txid=%s, wtxid=%s", tx->GetHash().ToString(), tx->GetWitnessHash().ToString())};
         switch (m_tx_for_private_broadcast.Add(tx)) {
         case PrivateBroadcast::AddResult::Added:
             LogDebug(BCLog::PRIVBROADCAST, "Requesting %d new connections due to %s", NUM_PRIVATE_BROADCAST_PER_TX, txstr);
             m_connman.m_private_broadcast.NumToOpenAdd(NUM_PRIVATE_BROADCAST_PER_TX);
    -        return true;
    +        return node::TransactionError::OK;
         case PrivateBroadcast::AddResult::AlreadyPresent:
             LogDebug(BCLog::PRIVBROADCAST, "Ignoring unnecessary request to schedule an already scheduled transaction: %s", txstr);
    -        return true;
    +        return node::TransactionError::OK;
         case PrivateBroadcast::AddResult::QueueFull:
             LogDebug(BCLog::PRIVBROADCAST, "Rejecting private broadcast, queue full (cap=%u): %s", PrivateBroadcast::MAX_TRANSACTIONS, txstr);
    -        return false;
    +        return node::TransactionError::PRIVATE_BROADCAST_FULL;
         } // no default case, so the compiler can warn about missing cases
         assert(false);
     }
    diff --git a/src/net_processing.h b/src/net_processing.h
    index 651e669ca2..b251e868c1 100644
    --- a/src/net_processing.h
    +++ b/src/net_processing.h
    @@ -9,6 +9,7 @@
     #include <consensus/amount.h>
     #include <net.h>
     #include <node/txorphanage.h>
    +#include <node/types.h>
     #include <private_broadcast.h>
     #include <protocol.h>
     #include <uint256.h>
    @@ -146,10 +147,10 @@ public:
         /**
          * Initiate a private transaction broadcast. This is done
          * asynchronously via short-lived connections to peers on privacy networks.
    -     * [@retval](/bitcoin-bitcoin/contributor/retval/) true The transaction is scheduled for private broadcast (or was already scheduled).
    -     * [@retval](/bitcoin-bitcoin/contributor/retval/) false Rejected because the private broadcast queue is full.
    +     * [@retval](/bitcoin-bitcoin/contributor/retval/) node::TransactionError::OK The transaction is scheduled for private broadcast (or was already scheduled).
    +     * [@retval](/bitcoin-bitcoin/contributor/retval/) node::TransactionError::PRIVATE_BROADCAST_FULL Rejected because the private broadcast queue is full.
          */
    -    virtual bool InitiateTxBroadcastPrivate(const CTransactionRef& tx) = 0;
    +    [[nodiscard]] virtual node::TransactionError InitiateTxBroadcastPrivate(const CTransactionRef& tx) = 0;
     
         /** Send ping message to all peers */
         virtual void SendPings() = 0;
    diff --git a/src/node/transaction.cpp b/src/node/transaction.cpp
    index 0d462103d4..e7877c6985 100644
    --- a/src/node/transaction.cpp
    +++ b/src/node/transaction.cpp
    @@ -133,10 +133,7 @@ TransactionError BroadcastTransaction(NodeContext& node,
             node.peerman->InitiateTxBroadcastToAll(txid, wtxid);
             break;
         case TxBroadcast::NO_MEMPOOL_PRIVATE_BROADCAST:
    -        if (!node.peerman->InitiateTxBroadcastPrivate(tx)) {
    -            return TransactionError::PRIVATE_BROADCAST_FULL;
    -        }
    -        break;
    +        return node.peerman->InitiateTxBroadcastPrivate(tx);
         }
     
         return TransactionError::OK;
    
    

    </details>


    instagibbs commented at 3:15 PM on June 8, 2026:

    taken thanks

  44. in src/private_broadcast.h:61 in e96c343137
      53 | @@ -50,13 +54,20 @@ class PrivateBroadcast
      54 |          std::vector<PeerSendInfo> peers;
      55 |      };
      56 |  
      57 | +    /// Outcome of Add().
      58 | +    enum class AddResult {
      59 | +        Added,          //!< The transaction was newly added.
      60 | +        AlreadyPresent, //!< The transaction was already present; no change.
      61 | +        QueueFull,      //!< Rejected: the queue is already at MAX_TRANSACTIONS.
    


    stickies-v commented at 4:28 PM on June 5, 2026:

    nit: this documentation style is now discouraged by developer-notes, instead it is recommended to:

    <details> <summary>git diff on e96c343137</summary>

    diff --git a/src/private_broadcast.h b/src/private_broadcast.h
    index 1968d496fe..27af749cf0 100644
    --- a/src/private_broadcast.h
    +++ b/src/private_broadcast.h
    @@ -56,9 +56,12 @@ public:
     
         /// Outcome of Add().
         enum class AddResult {
    -        Added,          //!< The transaction was newly added.
    -        AlreadyPresent, //!< The transaction was already present; no change.
    -        QueueFull,      //!< Rejected: the queue is already at MAX_TRANSACTIONS.
    +        //! The transaction was newly added.
    +        Added,
    +        //! The transaction was already present; no change.
    +        AlreadyPresent,
    +        //! Rejected: the queue is already at MAX_TRANSACTIONS.
    +        QueueFull,
         };
     
         /**
    
    

    </details>


    instagibbs commented at 3:15 PM on June 8, 2026:

    done

  45. in src/private_broadcast.cpp:7 in e96c343137
       2 | @@ -3,17 +3,24 @@
       3 |  // file COPYING or https://opensource.org/license/mit/.
       4 |  
       5 |  #include <private_broadcast.h>
       6 | +
       7 | +#include <logging.h>
    


    stickies-v commented at 4:29 PM on June 5, 2026:

    nit: I don't think we need this?

    <details> <summary> iwyu output</summary>

    link

    2026-06-04T16:25:41.6566108Z /home/runner/work/_temp/src/private_broadcast.cpp should add these lines:
    2026-06-04T16:25:41.6566591Z #include <compare>    // for operator<, strong_ordering
    2026-06-04T16:25:41.6568246Z #include <memory>     // for shared_ptr
    2026-06-04T16:25:41.6568572Z #include <utility>    // for get, move
    2026-06-04T16:25:41.6568793Z 
    2026-06-04T16:25:41.6569029Z /home/runner/work/_temp/src/private_broadcast.cpp should remove these lines:
    2026-06-04T16:25:41.6569486Z - #include <logging.h>  // lines 7-7
    2026-06-04T16:25:41.6569789Z - #include <util/check.h>  // lines 8-8
    2026-06-04T16:25:41.6569994Z 
    2026-06-04T16:25:41.6570223Z The full include-list for /home/runner/work/_temp/src/private_broadcast.cpp:
    2026-06-04T16:25:41.6570741Z #include <private_broadcast.h>
    2026-06-04T16:25:41.6571101Z #include <algorithm>  // for __max_element_fn, max, max_element
    2026-06-04T16:25:41.6571514Z #include <compare>    // for operator<, strong_ordering
    2026-06-04T16:25:41.6571872Z #include <memory>     // for shared_ptr
    2026-06-04T16:25:41.6572174Z #include <utility>    // for get, move
    

    </details>


    instagibbs commented at 3:15 PM on June 8, 2026:

    removed

  46. in src/private_broadcast.h:70 in e96c343137
      68 | -     * @retval false The transaction was already present.
      69 | +     * @return Whether the transaction was newly added, was already present, or
      70 | +     * was rejected because the queue is full (see AddResult).
      71 |       */
      72 | -    bool Add(const CTransactionRef& tx)
      73 | +    AddResult Add(const CTransactionRef& tx)
    


    stickies-v commented at 4:41 PM on June 5, 2026:

    I think this would benefit from [[nodiscard]] since we're returning error codes.


    instagibbs commented at 3:15 PM on June 8, 2026:

    done

  47. in test/functional/p2p_private_broadcast_cap.py:86 in e96c343137 outdated
      81 | +            assert child["wtxid"] in present_wtxids, \
      82 | +                f"tx index {i} (wtxid={child['wtxid']}) should still be present"
      83 | +        for child in children[MAX_TRANSACTIONS:]:
      84 | +            assert child["wtxid"] not in present_wtxids, \
      85 | +                f"rejected tx (wtxid={child['wtxid']}) must not be in the queue"
      86 | +
    


    stickies-v commented at 6:00 PM on June 5, 2026:

    I think this can be simplified by just checking the response is unchanged?

    <details> <summary>git diff on e96c343137</summary>

    diff --git a/test/functional/p2p_private_broadcast_cap.py b/test/functional/p2p_private_broadcast_cap.py
    index 57b9cf5390..909116aba1 100755
    --- a/test/functional/p2p_private_broadcast_cap.py
    +++ b/test/functional/p2p_private_broadcast_cap.py
    @@ -74,15 +74,7 @@ class PrivateBroadcastCapTest(BitcoinTestFramework):
                 assert_raises_rpc_error(-25, "Private broadcast queue is full",
                                         node.sendrawtransaction, child["hex"])
     
    -        pbinfo = node.getprivatebroadcastinfo()
    -        assert_equal(len(pbinfo["transactions"]), MAX_TRANSACTIONS)
    -        present_wtxids = {t["wtxid"] for t in pbinfo["transactions"]}
    -        for i, child in enumerate(children[:MAX_TRANSACTIONS]):
    -            assert child["wtxid"] in present_wtxids, \
    -                f"tx index {i} (wtxid={child['wtxid']}) should still be present"
    -        for child in children[MAX_TRANSACTIONS:]:
    -            assert child["wtxid"] not in present_wtxids, \
    -                f"rejected tx (wtxid={child['wtxid']}) must not be in the queue"
    +        assert_equal(pbinfo["transactions"], node.getprivatebroadcastinfo()["transactions"])
     
             # Re-submitting an already-queued transaction is a no-op, not an error,
             # even when the queue is full.
    
    

    </details>


    instagibbs commented at 3:15 PM on June 8, 2026:

    done

  48. in src/private_broadcast.cpp:13 in e96c343137
       9 |  
      10 |  #include <algorithm>
      11 |  
      12 |  
      13 | -bool PrivateBroadcast::Add(const CTransactionRef& tx)
      14 | +PrivateBroadcast::AddResult PrivateBroadcast::Add(const CTransactionRef& tx)
    


    stickies-v commented at 6:01 PM on June 5, 2026:

    nit: would benefit from [[nodiscard]]


    instagibbs commented at 3:14 PM on June 8, 2026:

    done

  49. in src/init.cpp:1 in e96c343137


    stickies-v commented at 6:09 PM on June 5, 2026:

    I don't think documenting this in init has a lot of value, there is nothing the user can do here, it shouldn't generally occur, and the only way this can manifest is through the sendrawtransaction RPC where you've added proper documentation?


    instagibbs commented at 3:14 PM on June 8, 2026:

    As the feature expands, various places will be effected yet this limit will still be enforced. Removed since I weakly hold this position.

  50. stickies-v commented at 6:12 PM on June 5, 2026: contributor

    Approach ACK, did pretty thorough code review on 143125343fcfe9ebe701d8a714e3e43209738a75 so I think I'm mostly out of comments after this

  51. private broadcast: limit outstanding txs to count of 1000
    Add a belt-and-suspenders feature, limit the amount of memory
    possible when unlucky or simply misconfigured. The worst case limit is
    roughly 400kB * 1000 = 400MB, regardless of usage pattern.
    
    Before this change, sheer volume of broadcasts, mismatches in
    standardness rules, or simply fee mismatches may result in unbounded
    growth of memory usage. As the feature may be expanded in
    the future, explicit bounds helps reasoning going forward.
    efd470ffe9
  52. private broadcast: add release note for limited cap 004db8dae5
  53. instagibbs force-pushed on Jun 8, 2026
  54. instagibbs commented at 3:17 PM on June 8, 2026: member

    One thing I noticed is that we're returning a generic rpc error (-25), probably worth adding a new code for this to allow the caller to programmatically react to it? Or is this generic error sufficient.

  55. in src/private_broadcast.cpp:19 in 004db8dae5
      17 | -    const bool inserted{m_transactions.try_emplace(tx).second};
      18 | -    return inserted;
      19 | +    // Re-adding an already-tracked transaction is a no-op regardless of the cap.
      20 | +    if (m_transactions.contains(tx)) return AddResult::AlreadyPresent;
      21 | +
      22 | +    if (m_transactions.size() >= MAX_TRANSACTIONS) return AddResult::QueueFull;
    


    vasild commented at 9:02 AM on June 9, 2026:

    Not too difficult to switch the limit from "number of transactions" to "number of bytes" and also base the limit on the actual amount of RAM on the system.

        const size_t max_mem{GetTotalRam().value_or(16_GiB) / 100}; // Use max 1% of the RAM.
        size_t used_mem{0};
        for (const auto& [t, _] : m_transactions) {
            used_mem += t->ComputeTotalSize();
            if (used_mem > max_mem) {
                return AddResult::QueueFull;
            }
        }
    

    instagibbs commented at 1:03 PM on June 9, 2026:

    I do not want to do dynamic cap. Switching to aggregate serialized size cap alone would bring the effective capacity up ~100x to ~500x based on most usage patterns. The downside is the aborting UX becomes a bit more complicated if it's ever hit (and I need to write more comprehensive tests to test boundary stuff, which is fine)

    edit: I'd want to do some more analysis and testing to see how things work when you have >>1k oustanding broadcasts probably first. I'll think along these lines for a bit.


    instagibbs commented at 3:19 PM on June 9, 2026:

    made a longer comment in the main thread #35406 (comment)

    I'm declining to take on such a change at this time.

  56. vasild commented at 9:33 AM on June 9, 2026: contributor

    Just 1000? The OP says 400kB * 1000 = 400MB, but where did that 400kB come from?

    it's the currently maximum sized transaction we will accept into our mempool (or private broadcast set).

    I see. That is imposing a generic limit on everybody based on an extreme edge case. That would unnecessary cripple normal usage. I mean - trying to protect a user on Raspberry Pi who sends 1000 400KB transactions within seconds and has a problem because the memory usage goes up with 400MB. I think that is less likely compared to a bitcoind running on a normal machine and sending many normal-sized transactions and hitting the 1000 limit unnecessary (that would take about 1MB of RAM).

    I think the simplest alternative is to have the serialized sized accounting ...

    Yes, that looks better to me. See below for an example on how to do that.

    One thing I noticed is that we're returning a generic rpc error (-25), probably worth adding a new code for this to allow the caller to programmatically react to it? Or is this generic error sufficient.

    I think a separate code could be useful for the callers of the RPC. Maybe use RPC_OUT_OF_MEMORY instead of introducing a new one.

  57. instagibbs commented at 3:18 PM on June 9, 2026: member

    Along the lines of seeing if we should raise the effective cap by doing byte accounting rather than tx count, I worked on a set of benchmarks to see how this all scales. Some of the bigger benchmarks take a super long time to setup:

    https://github.com/instagibbs/bitcoin/commit/33c32490f3e02061b64a930680c267e1c731d054

    A few salient points:

    1. statuses are never cleaned up, even when the peer is disconnected, so the O(N*S) scanning mentioned in the benchmark can be exacerbated by this if say, the tx is non-standard to most nodes or misconfiguration. The good news is that cpu usage seems to be linear growth as S grows. We should be pruning state as we can (need to work on this separately, noting the hole only)
    2. CPU usage grows super-linearly in N(number of txs), probably due to cache misses as the map grows.
    3. The various O(N*S) costs are during the net thread lock being taken, and take 10s of musec at 10k items, and would be in the ~10's of ms range at 100k items. That's not crazy, but fundamentally still not bounded until we cap 'S' somehow.
    4. The actual cost would be the re-attempt loop, which even with warmed caches could take up to 1s to validate with fairly boring txs (1-in-1-out in my benchmark), and proportionally longer as the backlog grows. My bench reports 50ms for 1k items. This is all taking cs_main, for the entire period. This does not grow with 'S'.

    All this to say, I think it's reasonable to do the 1k count limit for now for backport, work on bounding the overall memory/cpu usage is a more principled fashion, then relaxing the limit later.

    Another mitigation (not being proposed for this PR) could be to not hold cs_main the entire loop and test in batches.

  58. in src/test/private_broadcast_tests.cpp:46 in efd470ffe9
      42 | @@ -43,15 +43,15 @@ BOOST_AUTO_TEST_CASE(basic)
      43 |      // Make a transaction and add it.
      44 |      const auto tx1{MakeDummyTx(/*id=*/1, /*num_witness=*/0)};
      45 |  
      46 | -    BOOST_CHECK(pb.Add(tx1));
      47 | -    BOOST_CHECK(!pb.Add(tx1));
      48 | +    BOOST_CHECK(pb.Add(tx1) == PrivateBroadcast::AddResult::Added);
    


    andrewtoth commented at 9:45 PM on June 9, 2026:

    These can all be BOOST_CHECK_EQUAL.

  59. in test/functional/p2p_private_broadcast_cap.py:83 in efd470ffe9
      78 | +
      79 | +        # Re-submitting an already-queued transaction is a no-op, not an error,
      80 | +        # even when the queue is full.
      81 | +        self.log.info("Re-submitting an already-queued tx should not error")
      82 | +        node.sendrawtransaction(children[0]["hex"])
      83 | +        assert_equal(len(node.getprivatebroadcastinfo()["transactions"]), MAX_TRANSACTIONS)
    


    andrewtoth commented at 9:58 PM on June 9, 2026:

    Can do the same here as above for consistency

            assert_equal(pbinfo["transactions"], node.getprivatebroadcastinfo()["transactions"])
    
  60. andrewtoth commented at 10:09 PM on June 9, 2026: contributor

    We did design the private broadcast queue to assume there will not be a very large data set. See #29415 (comment). So, it makes sense to bound it. I think 1000 entries is on the small side though. Using worst case 400kb transactions seems less than ideal.

  61. instagibbs commented at 12:51 PM on June 10, 2026: member

    @vasild the OOM code is unused, I honestly don't know what the guidance is for usage of it...


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-06-21 09:50 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me