validation, net: Process blocks asynchronously and reduce cs_main contention #36244

Group the three successful block-check caches in a CBlock-specific,
value-copyable type. Use atomic loads and stores so immutable blocks can
be checked and copied concurrently with independent validation states.
Keep the payload copy and move operations compiler-generated.

The bits memoize successful checks; they do not publish payload changes
or guarantee exactly-once checking. Callers must keep block contents and
effective consensus parameters unchanged while sharing a block. Copies
may observe the independently valid success bits at different times.

Keep the initial ProcessNewBlock lock placement unchanged.

Serialize the initial CheckBlock call with a private manager mutex and
release it before waiting for cs_main. Keep initial rejection handling
and synchronous validation notifications under cs_main, with master
acceptance, storage, import and activation locking unchanged.

This narrows the initial checking lock; it does not unlock acceptance or
block-file writes or promise FIFO acceptance for concurrent callers.

Separate ProcessNewBlock execution from subsequent peer-state updates,
while continuing to call both synchronously at all three submission sites.

Keep processing success distinct from new_block. Only new_block controls
the existing timestamp and request housekeeping, including write errors.
Move optimistic compact reconstruction's additional transaction-valid
request cleanup into the continuation, using a fresh index lookup under
cs_main. Other submission paths do not gain that cleanup.

Move header acceptance, storage policy, block/contextual checks, and
invalid-block handling into PreWriteCheckBlock(). Return admission success
separately from whether block data should be stored, distinguishing
rejections, successful no-ops, and blocks eligible for persistence.

Keep AcceptBlock() responsible for synchronous persistence with its
existing locking and new_block semantics. Preserve ProcessNewBlock()'s
initial check, notification ordering, and chain activation.

Move block notifications, disk writes, index updates and flushing into
StoreBlock(), called synchronously by AcceptBlock() after admission.
Preserve cs_main locking, the existing disk-position import path, and
new_block assignment before storage.

This exposes the persistence stage for reuse by a future validation worker.

Pass a caller-owned BlockValidationState through ProcessNewBlock's
CheckBlock and AcceptBlock stages, keeping chain-activation states local.
Update callers to supply fresh states while preserving synchronous
BlockChecked delivery, processing success, and new_block semantics.

Cover initial rejection, valid and duplicate submissions, and a block
that passes admission but fails validation when connected. This prepares
callers to distinguish initial validation results from future completion,
following the API preparation in PR #18963.

Move BlockProcessingResult from PeerManagerImpl into the validation API
and return it from ProcessNewBlock instead of returning bool and writing
new_block through an output parameter. Update all callers while keeping
validation, notifications, and peer completion synchronous.

Preserve both outcomes independently, including new_block remaining true
on a write failure. Neither field implies full consensus validity.
Extend the initial-state regression test to check both result fields and
exercise a real block-file write failure. Capture and assert its expected
error notification to keep successful test output clear.

This prepares the API for a future carrying both completion outcomes.

Wrap BlockProcessingResult in a future, fulfilling its promise at each
existing return. Keep validation synchronous and fulfill successful
completion after active and historical chainstate activation.

Update callers to consume the ready future before their existing
completion actions, preserving initial-state reporting, both result
fields, and notification ordering. Explicitly discard unused test
results for libc++ compatibility. Check future validity and immediate
readiness in the existing outcome regression scenarios.

Retain completion futures for full blocks, completed compact blocks,
and optimistic reconstruction. Poll without waiting, pausing new-message
dequeuing and the normal send loop for the submitting peer while its
completion is pending. Continue servicing previously queued getdata and
orphan work before checking for pending block processing.

Keep records in the peer manager by peer ID and poll them even when no
peers remain, so download cleanup survives source disconnection. Preserve
newer source attribution, check optimistic transaction validity at
completion, and apply punishment before dequeuing another message.
Handle failed futures without letting exceptions escape the message loop.

Add controlled-future coverage for peer pause/resumption, queued getdata
work, ordinary messages from other peers, disconnection, completion
ordering, and errors. Validation still executes synchronously and returns
ready futures.

Add a single-worker FIFO with owned blocks, processing callbacks, and
completion futures. Accept submissions while running and complete futures
with either a result or the callback's exception. Report inactive or
stopping queues without retaining the submitted work.

Make ChainstateManager own the queue. Support explicit startup, interruption,
and draining shutdown, and stop before node and test dependencies are
removed. Keep the kernel context alive throughout manager destruction.
Move BlockProcessingResult into the queue header without changing its API.

Add tests covering lifecycle, ownership, exceptions, FIFO processing,
concurrent producers and shutdown, and callback submission. Production
startup leaves the worker inactive and ProcessNewBlock remains synchronous;
validation admission, worker processing, and completion wakeups follow
separately.

Exclude received blocks awaiting processing completion from automatic
download selection and compact-block reconstruction. Avoid attributing
download stalls to peers when local processing holds back or advances the
download window.

Run initial validation and admission on the caller, then submit blocks needing
storage for persistence and chain activation. Retain inline execution before
the queue is started for kernel, fuzz, and other standalone callers.

Return initial errors directly and deferred cached-invalid outcomes through the
future so peer attribution remains tied to each submission. Notify listeners
when completions become available and adapt blocking callers to the split
result contract.

Start the block-processing queue during node and ordinary test setup. Keep fuzz
and kernel callers on the inline path, and stop the worker before its callback,
networking, mempool, and index dependencies are destroyed.

Add coverage for queued parent invalidation, duplicate rejection attribution,
pending downloads and stalling, disconnected sources, and worker-gate and
subscriber lifetimes.

Deliver BlockChecked through the validation task runner, retaining the
block and validation state until delivery. Keep peer completions pending
until a queued marker has run so validation feedback precedes source
cleanup and message processing resumption.

Wait for queued results before unregistering mining catchers. Adapt the
network fuzz targets to a scoped scheduler and serial runner, with
explicit callback and completion draining between messages, so callbacks
can acquire cs_main without blocking the emitter under that lock.

Cover queued ownership, callback ordering, peer teardown, and the fuzz
runner's deferred delivery.

Move mapBlockSource from cs_main to a PeerManager bookkeeping mutex,
following the source-attribution change in PR #18963. Source insertion and
ordinary completion cleanup can run while validation holds cs_main.

Acquire cs_main before the new mutex where chain state is also needed,
matching the modern PeerManager lock order. Preserve source ownership,
compact-block punishment exceptions, and callback ordering. Add a
regression that completes source cleanup while another thread holds
cs_main.

Move per-peer in-flight blocks and download/stalling timers out of
CNodeState, following PR #16324. Protect them, mapBlocksInFlight, and the
downloading-peer count with the block bookkeeping mutex.

Serialize request registration and cleanup with peer removal, retaining
the Peer until its request entries are cleared. Keep cs_main for chain
dependent reads and preserve request selection, limits, and timeouts.

When new_block is true, skip the redundant optimistic validity recheck
and its chain-lock acquisition after request cleanup. Retain the recheck
for completions with new_block=false.

Add a regression that registers and completes requests for both full
blocks and optimistic reconstructions while another thread holds
cs_main. Freeze its node clock so the completion timestamp assertion
cannot race a wall-clock second boundary.

Skip taking the chain lock when a peer has no orphan transactions to
reconsider. Defer the remaining send work when cs_main is unavailable so
the shared message handler can service other peers.

Keep fee-filter calculation inside the successful chain-lock scope so
block validation cannot acquire the mempool lock before the calculation.
Require cs_main in the helper's lock annotation.

Add a regression covering PING/PONG output, punishment, and deferred
header requests while another thread holds cs_main.

Use the validated header already stored with the download request to
determine SegWit activation. This lets BLOCKTXN bookkeeping and
reconstruction proceed under m_block_mutex while another block is being
validated. Release the lock before the existing admission checks.

Add a regression covering unexpected responses and reconstruction
failures while another thread holds cs_main, including full-block
fallback.

Clear other peers' requests after successful initial admission of an
optimistically reconstructed compact block, following #16324. Pending
processing still suppresses automatic downloads and pauses the source
until its completion and validation callbacks have been handled.

Remove the optimistic flag from pending completions and the remaining
completion-time validity recheck under cs_main. Cover admission with a
parked worker, queued duplicates, source disconnection, initial rejection,
and interrupted submission.

Validation: 41 unit tests passed in regular and ThreadSanitizer builds;
six compact-block, invalid-block and unrequested-block functional tests
passed.