Reimplement private broadcast as node state-less jobs + package relay support #36608

Empty commit marking where the parts of this pull request begin. Part 1 moves code that bitcoin-privbcast shares with the node into the common library and exports what it needs from net and netbase. No behaviour change.

bitcoin-privbcast (added later in this series) links the BIP324 cipher
without the node library. The cipher depends only on crypto, keys,
randomness and chain parameters, which the common library already
provides.

The transports and the messages they carry are needed by a tool that
must not link the node library (bitcoin-privbcast, added later in this
series). Move them, with NodeId and MAX_PROTOCOL_MESSAGE_LENGTH, to
net_transport.{h,cpp} in the common library. net.h includes the new
header and drops the includes that only the moved code used; files that
name NodeId include the new header themselves.

net.h includes the new header first, where bip324.h was: Qt's moc reads
net.h through the GUI's headers and cannot parse libstdc++'s <concepts>
unless an earlier header has defined _GLIBCXX_VISIBILITY.

Move-only: no code changes.

Empty commit marking the start of part 2: the specification and the standalone tool. Nothing in the node changes.

This is the design the following commits implement: what a private
broadcast job guarantees, the values peers and observers see, and the
interfaces of the tool and the node. Later commits cite its items by
number.

Start bitcoin_privbcast, the library that will run private broadcast
jobs for bitcoin-privbcast and the node. params.h holds the values of
the specification. DrawPlan draws a job's plan from the job's own
randomness: the schedule, the order of the DNS seeds and the seed of
the order in which each DNS seed hands out its candidates.
MIN_DNS_SEEDS is the floor R3 sets for the mainnet seed list, and a
unit test holds the list to it, so a release that drops seeds below
what fills a job fails the test rather than leaving opportunities
empty.

Implements C1 and R3; the values of Parameters.

Assign fixes every opportunity's candidate once discovery ends, from
the plan and the discovery result alone. An endpoint that several
seeds returned is credited to the seed that comes first in the plan's
seed order, through the lowest query set that returned it. Each seed
hands out its candidates in an order drawn at random by a generator of
its own, seeded from the plan and the seed's place in that order. Slot
j's first attempt comes from query set j mod 4 whenever a seed it may
draw from has one, so a single exit supplies at most two first
attempts. Neither which seed answers first nor the order of a seed's
answers changes the assignment.

Implements C2, R2, R4, R5a, R5b and R5c.

A job reaches recipients and resolves DNS seed names through the proxy,
on streams its event loop drives without blocking. Socks5Client holds
no socket: it offers the bytes to write, consumes the proxy's replies,
and leaves bytes past the final reply to the P2P transport. It offers
only username and password authentication, with the credentials it is
given or fresh ones, and sends a numeric address as IPV4 or IPV6
without resolving anything.

Implements B2 and the Proxy interface section.

An attempt is one connection to a recipient once the proxy connected
it: a BIP324 session as initiator, the handshake with the job's
VERSION, one INV, the transaction served at most once, and the PING.
It holds no socket and reads no clock: the job's loop passes the bytes
read and the time; each call records what happened (a completed write,
the connection, bytes received) and then enforces the attempt's
deadlines, so that nothing is acted on at or after one.

The attempt draws its VERSION and PING nonces from the random context
it is given, and takes its BIP324 key, ellswift entropy and garbage
from a key source. Without one, as in production, V2Transport draws
them from the strong RNG. The tests and the fuzz target inject a
source that draws from their seeded context, so that an attempt's
bytes are a function of that context and of what the peer sent.

Implements E1-E8, A2, B4, D3 and the attempt part of D2.

A job reaches the network only through streams to the SOCKS5 proxy,
which its event loop drives without blocking. ProxyStream connects to
the proxy by address or unix socket path, resolving nothing, and runs
the exchange of its Socks5Client; each stage, the connect and each
step of the exchange, times out on the steady clock. Once the proxy
has connected the stream, its socket carries the destination's bytes.

Discovery launches every RESOLVE query at job start, each on a stream
of its own, in four query sets: each set has credentials of its own
and one query of every seed, so Tor builds four circuits rather than
one per query, and each seed is still asked through four exits.
Discovery ends when every query has ended or when the discovery window
does, whatever is still pending. It keeps the answers that are public
routable IPv4 or IPv6 addresses, with the chain's default port, and
counts the others; its result gives each seed's answers by the query
set they came through, whatever order they arrived in. An exception
thrown while it acts on one query ends only that query, as a failed
stream would.

The tests run discovery against a scripted proxy behind fake sockets
that connect as non-blocking sockets do. The fake socket and the
proxy's side of the SOCKS5 exchange go into the test utilities, as the
job's tests use them too.

Implements B1, B2, C3, D1, R1, R2 and C7 for the queries.

A job is one event loop on its caller's thread. Step() runs the
deadlines and the dials that are due, waits on every open stream until
the next deadline, and does the I/O; Run() steps to the end. The plan
is drawn when the job is made, before any socket exists, and discovery
starts with the first step. When discovery ends the assignment is
fixed, and each slot dials its opportunities at their scheduled times,
one attempt at a time, until one reaches the announcement point or
none is left. The slots share only the loop, so a stalled stream holds
up no other slot, and the cancel flag ends everything at the next step.

An exception thrown while the loop acts on one attempt ends only that
attempt, as a failed stream would. One thrown anywhere else in a step
fails the job: it stops as a cancelled job does, and its report gives
the error.

The report mirrors Interface/Report, every time an offset from job
start on the plan clock. Snapshot() gives the progress the node is to
show, from any thread. ExitStatus() gives the tool's exit status for a
report.

Implements A1, A3, A4, C1-C8, D1, D2, H1, H3 and the report interface.

The SOCKS5 server gains Tor's RESOLVE command, reply delays for a slow
or stalled proxy, a mode that offers only unauthenticated access and a
mode that behaves as an ordinary non-Tor proxy. A client that hangs up
before its greeting, as a cancelled job's streams can, and a connection
that Windows reports as aborted count as closed, not as errors. The P2P
network thread judges a failed protocol by its own reconnect flag, so
that several listeners can share one loop. The framework learns where
bitcoin-privbcast is and how to skip a test without it.

Prepares the functional test that the next commit adds with the binary.

The standalone tool: it reads one transaction on stdin, runs one job
through the Tor proxy named by -tor and prints the report on stdout,
with progress lines on stderr. It takes no configuration file and only
a loopback proxy, refuses the regtest overrides on other chains and
checks its arguments and input before any network activity. The
discover command runs discovery alone.

Build and packaging: the BUILD_PRIVBCAST option (on with BUILD_TESTS),
the privbcast subcommand of the bitcoin wrapper, the man page, the
Windows resource file, the manpage generator and Windows CI lists, and
the test configuration that tells the functional framework the binary
exists.

The functional tests tool_privbcast.py, tool_privbcast_proxy.py and
tool_privbcast_peers.py, with their harness in
test_framework/privbcast.py, are the specification's black-box
acceptance suite for the tool: scripted recipients and a scripted proxy
check every wire-visible parameter, the schedule, the assignment, the
exit statuses and cancellation.

Implements B3, H2, U1 and the Interface section for bitcoin-privbcast.

Add the user guide to private broadcast with its bitcoin-privbcast
part: the Tor proxy the tool needs, how to send a transaction and read
the exit status and the report, discover, and the limits of what a job
protects.

Empty commit marking the start of part 3: the node runs each private broadcast as the tool's job, and the old relay and connections are removed.

The node runs each private broadcast as a privbcast::Job, the tool's
job, on a thread of its own. PrivbcastQueue queues the submitted
transactions, starts and cancels the jobs from one scheduler thread,
and keeps in memory the records that getprivatebroadcastinfo and
abortprivatebroadcast are to show. It reaches the node only through
callbacks for networking, the onion proxy and the mempool, and takes
the job runner as a parameter, so that the tests drive it with stub
jobs and mocked clocks. A job that failed, or whose runner threw, ends
aborted with the error in its record.

The scheduler's start rule is a pure function, StartPolicy::Decide(),
which each pass of the scheduler applies: a fuzz target checks the
rule on its own, and another the queue through its interface.

bitcoin_node now links bitcoin_privbcast.

Implements N3-N7, N9, N10, the node's D2 and D3, U3. Not yet wired:
the next commit switches -privatebroadcast to it.

With -privatebroadcast, sendrawtransaction now queues a job on the
node's PrivbcastQueue instead of handing the transaction to the
connection manager, getprivatebroadcastinfo lists the jobs with their
progress and reports, and abortprivatebroadcast aborts them. Startup
requires onion to be able to become reachable, refuses the regtest
seed overrides on other chains, -signetseednode, -signetchallenge, and
off regtest a non-zero -mocktime, and refuses to start rather than trim
connections when the jobs' file descriptors are not available. -connect
and -proxyrandomize no longer matter.

The old private broadcast connections and relay stay in the tree, no
longer fed, until the commits that remove them.

Implements N1, N2, N8, N10, N11, U1, U2 and Interface/Node; replaces
the old implementation's tests with the specification's.

Since -privatebroadcast runs jobs, nothing hands PeerManager a
transaction for private broadcast any more. Remove its queue
(src/private_broadcast.{h,cpp}) with the methods, the retry task and
the private_broadcast option that served it, and its handling of
private broadcast connections: the fixed VERSION, the INV after
VERACK, the GETDATA and PONG handling, the incoming message filter,
the lifetime limit and the exemption from discouragement. The unit and
fuzz tests of that code go with it, including the net_tests case for
the addrman exception and the denialofservice_tests cases for the
discouragement exemption.

Delete-only: besides whole lines, conditions and arguments are cut out
of lines, and the lines of a removed branch's alternative move out one
indentation level. git show -w --word-diff-regex='[A-Za-z0-9_]+|[^[:space:]]'
shows deletions only, apart from the size of the node array in
denialofservice_tests, which goes back from 5 to 3.

With the relay gone, nothing asks CConnman for private broadcast
connections. Remove the connection type, the thread that opened such
connections with its counters and choice of network, and the places
that treated them specially: the incoming nonce check, the fresh I2P
session, the guard against direct connections, the outbound message
filter, the fixed protocol version, the exception in disconnection by
subnet and their term in the orphanage's static_assert. The type's
names go from getpeerinfo's help, bitcoin-cli -netinfo and the GUI. The
proxy override of ConnectNode() and OpenNetworkConnection(), which only
that thread set, goes too, with its copy in CNode and in the
reconnection record and the fuzz input for it.

Delete-only: besides whole lines, conditions, arguments and list
separators are cut out of lines, what followed a cut argument joins
its line, and one line moves out an indentation level with the if
around it. git show -w --word-diff-regex='[A-Za-z0-9_]+|[^[:space:]]'
shows deletions only.

Without private broadcast connections, a node advertises
PROTOCOL_VERSION on every connection, and CNode::AdvertisedVersion()
only returns it. Use the constant again where the method was called.

Add the node part of the private broadcast guide: which Tor proxy the
jobs use, how sendrawtransaction queues them and when they start, what
is not queued again, how to follow and abort jobs, what disabling
networking and shutdown do, which node settings do not apply, what
the default log shows, the file descriptors the jobs need, and the
link between a job's connections and the node's peers at shutdown.

Empty commit marking the start of part 4: one parent and its child. The validation commits here serve only this part.

A child can pay for a parent too cheap to enter mempools alone when the
recipient takes the two together. A job may now carry the parent: each
attempt announces only the child, serves the parent to a peer that asks
for it, and holds the PING for that request. A job without a parent
runs and reports as before.

Implements F1-F4 and the package amendments of E6-E8.

bitcoin-privbcast send also reads two transactions, an unconfirmed
parent and its child in either order, checks that they are such a pair
and runs the job in package mode. The acceptance suite gains its
package sections in tool_privbcast_package.py.

Implements the Extension's Interface for bitcoin-privbcast.

Two prioritisetransaction deltas near INT64_MIN made the sum overflow.

ProcessNewPackage's test mode checks each transaction without package
feerates or replacements, as testmempoolaccept does. With
test_as_submission it runs submission's evaluation instead and stops
short of the mempool, so a caller can learn whether a child pays for its
parent before relaying them. A transaction that passed before another
failed is reported by its own result, as submission reports it. Results
do not list replaced transactions, and a parent that passes alone gives
no answer for its child, which it does not stage.

From #35286, without its RPC.

A test does not stage a transaction that passes alone, so the child
that spends it was found missing its inputs. Test the new transactions
together instead, each paying for itself. Their replacements get
package RBF's checks of fees, feerate diagram and cluster limits, but
not its rules that the package have no mempool ancestors and pay more
per vbyte than the parent: submission would take the two in turn, each
replacing what it may. A replacement of an ancestor of the package is
refused instead, as it would take away the package's inputs. For a
parent and child this accepts at least what submission accepts.

With -privatebroadcast, submitpackage now queues a private broadcast
job instead of adding the transactions to the mempool. It takes one
transaction, or a parent and its child, and queues one job, the
child's, which carries the parent. A transaction whose txid is in the
mempool counts as accepted; the rest is test-accepted, one transaction
alone as by sendrawtransaction, two as a package, after the package
checks on the whole submission, two as submitpackage evaluates them.
maxfeerate follows each path's own convention. Unit tests drive the RPC through its table.

The queue takes the parent with the transaction: a submission with a
parent is covered only by a job with the same parent, one without by a
job with any parent. getprivatebroadcastinfo lists the parent's txid
and documents the report's package fields; abortprivatebroadcast
matches the child's ids only. Both submission RPCs ask the queue
whether an onion proxy is configured, which lets unit tests drive
them without the process-wide proxy registry. The acceptance suite gains its package
sections, run as p2p_private_broadcast.py --package.

Implements the Extension's N5 and N11 amendments and its Interface for
submitpackage.

Add the specification's section "Extension: one parent, one child",
which the preceding commits implement: a job that carries a child and
its unconfirmed parent announces only the child and serves the parent
on request (F1-F4), and the tool and the node take such a pair.

Add the guide's part on a parent and its child: when it helps, which
recipients accept such a package, what the tool and the node check and
what peers can tell; and how submitpackage queues a job under
-privatebroadcast.