Header-only support for waitNext()

Sjors commented at 4:38 pm on October 31, 2025: member

I’d like to expand the waitNext() Mining IPC function to optionally (via BlockWaitOptions) return a (empty) new template as soon as we have a header with enough proof-of-work, pending block download and/or validation.

https://github.com/bitcoin/bitcoin/blob/3cd4263bf66408760ec3b4ce6ccedcc87e0d53de/src/interfaces/mining.h#L63-L74

I suspect there’s little benefit if compact block reconstruction succeeds, and IIUC we hold cs_main during this process anyway so there’s no way to generate a template.

But if we need a round trip to our peer because compact block construction didn’t finish, it might be useful to create an empty block template and push it.

This can hopefully be implemented without touching validation code, as long as the kernel can provide a notification. Currently it only notifies us of a better tip, not a better header.

Closely related to this is the Stratum v2 concept of sending an empty (or optimistically filled) “future” template downstream for mining devices to load, followed by a prevhash messages when a new header is found. Whether that’s actually more performant than the above probably depends on the details of ASIC firmware.

Implementation wise, we’d need the same kernel notification for it. At the interface level it might look like a getFuture() method on BlockTemplate which could return a FutureTemplate interface with a wait() method on it. That would return a prevhash once we validated a header. The client then sends that downstream and immediately calls waitNext() on the original template in order to get a fresh full block template as soon as we processed the full block.

In both cases, we’d need a way to notify the TP if it the block turned out to be invalid. That way it can revert to the previous template.

Fi3 commented at 4:47 pm on October 31, 2025: none

it make sense for the TP to optimistically create templates as soon as there is a possible new block since this will reduce the time that a miner waste on stale jobs. Is kinda safe since the TP can stop following the optimistic path when 2 or more consecutive invalid blocks are received. If we leave it like it is now the miner will keep mine on the old job until the block template do not provide a new job.

So worst case we waste a bunch of hundreds milliseconds mining invalid jobs, that we are still going to waste mining on old jobs.

ismaelsadeeq commented at 11:41 am on November 7, 2025: member

I’d like to expand the waitNext() Mining IPC function to optionally (via BlockWaitOptions) return an (empty) new template as soon as we have a header with sufficient proof-of-work, pending block download and/or validation.

Why? you should expand on the motivations behind this and what exactly you aim to achieve with this feature.

I suspect there’s little benefit if compact block reconstruction succeeds. And, if I understand correctly, we hold cs_main during this process anyway, so there’s no way to generate a template.

Even if there is way to generate a template you haven’t validated the block, activate a new tip and evicted conflicts from the mempool yet, so generating a block template with the block assembler is likely to produce the same result as the one you’re currently working on.

To construct an empty block that builds on the header you validated, you have two options:

Update the block assembler code to accept a custom chain tip. This would require some refactoring instead I think it would be better to
Isolate the header creation section of the block assembler into a function that takes the current block index and consensus parameters, then loads the next block header into the proposed block template. This subroutine could then be used by the mining interface/block template manager to generate an empty block template.

Re: #33389 (comment)

Yes, it is compatible. However, the approach I am aiming for in #33758 relies on non-blocking validation interface notifications, so we should simply add a new notification that is executed once a new valid header is known. This notification will wake up the block template manager, which will then use the subroutine isolated in step 2 above to construct a header with an empty template and push it to subscribed clients.

As I mentioned in #33758, I would like to prune WaitAndCreateNewBlock and create a new GetNext() function on a block template manager that is decoupled from kernel notifications internals, the cs_main lock, and the kernel notification’s condition variable. The block template manager should have its own internal locks, maintain a copy of the chain tip, and wake up its own condition variables to execute what’s needed on the scheduler thread after receiving validation interface notifications.

So subscriptions to next block template, next block header, can easily be added and tested in isolation.

Sjors commented at 5:12 pm on November 7, 2025: member

I’d like to expand the waitNext() Mining IPC function to optionally (via BlockWaitOptions) return an (empty) new template as soon as we have a header with sufficient proof-of-work, pending block download and/or validation.

Why? you should expand on the motivations behind this and what exactly you aim to achieve with this feature.

To allow miners to start hashing on an empty block template as soon as we see a new header, rather than waiting for the block to be fully downloaded and validated. This provides miners with additional revenue because they are less likely to mine a stale block [0]. The risk however is that the new block was invalid, in which case they’re (briefly) mining on top of an invalid block.

[0] if they happen to find a solution during this time, afaik we won’t even broadcast it because it’s not better than our current tip. But even if we did broadcast it, other nodes won’t prefer it because of the first-seen rule.

ismaelsadeeq commented at 6:10 pm on November 7, 2025: member

Thanks for the context.

This provides miners with additional revenue because they are less likely to mine a stale block [0]

Before doing this, I think we should benchmark in practice the usual latency between knowing about a valid new block header, downloading, validating, and switching the chain tip. As you mentioned, for a node with a good connection and reasonable policy rules, compact block reconstruction should prevent any measurable latency. Also, we are likely going to have SENDTEMPLATE, which will help as well so might not be worth it if the time delta is not going to have impact with relation to the time it will take to generate the empty block, send to tp, and sending the new work to the miners in the pool. If the latency is not much, we just increase the chances of getting empty blocks and make miners lose on fees.

We could also do this smartly, and only send the empty block when the number of transaction not in mempool and extra pool is high after we get the compact block.

Malicious miners could collude to reduce the chances of smaller miners who use this mechanism from mining blocks and increasing their hashrate when they do this by default.

Whenever the chain tip changes on the valid chain, the malicious miners could collaborate and start working on an invalid block, first broadcasting a header with valid PoW but an invalid block (by consensus). As a result, smaller miners who rely on this mechanism might unknowingly keep working on a block that is valid in isolation but built on an invalid chain, which would later be rejected by the network.

They can do this repeatedly, and since they have more hashrate, their chances of succeeding increase. The chance for smaller miners is further limited to the time between the tip changes and the larger miners mining an invalid block. But I think this is not realistic and the time they spend finding the invalid block they should rather just be honest and use it to mine.

Sjors commented at 6:17 pm on November 7, 2025: member

we should benchmark in practice the usual latency between knowing about a valid new block header, downloading, validating, and switching the chain tip

Agreed, ideally I’d like to see data from actual miners.

compact block reconstruction should prevent any measurable latency

Yes, but with the recent trend of people using incompatible policies, we should prepare for a scenario where compact block relay (often) doesn’t work.

I don’t know if SENDTEMPLATE can fully compensate for broken compact relay or is more of a bandaid.

theoretical attack that this could open up

I agree that mining on just headers is not without risk. It should be something the miner can turn on or off. Unless of course we decide it’s really too unsafe, then we shouldn’t support it.

willcl-ark added the label Feature on Nov 18, 2025

willcl-ark added the label Mining on Nov 18, 2025

Header-only support for waitNext() #33756