ZMQ rawblock doesn't publish new block in case of chainsplit. #20376

Expected behavior I expect ZMQ raw block subscription to publish every raw block even in the case of a chainplit. Today two blocks (656477-8, with tip hash 00000000000000000005f8f74e57aa4584aacfed509b8a6feb20bc22e7d60a34) were orphaned and I decided to check my bitcoin node logs to see how it behaved during this event.

Actual behavior The node published a ZMQ notification for block 656476 and the first 656477 block it received at 18:47:39Z. Around 9 minutes later, the node receives blocks 656477 and 656478 that orphan the previous 656477 block. From the log, we see that the zmq notification for the second 656477 block was not published:

2020-11-11T18:27:47Z UpdateTip: new best=000000000000000000103c23bc9a56b6c08340f973fab41c7dfd6535700ad8f4 height=656476 version=0x37ffe000 log2_work=92.435868 tx=585895690 date='2020-11-11T18:27:18Z' prog
ress=1.000000 cache=24.6MiB(187792txo) warning='75 of last 100 blocks have unexpected version'
2020-11-11T18:27:47Z zmq: Publish rawblock 000000000000000000103c23bc9a56b6c08340f973fab41c7dfd6535700ad8f4
2020-11-11T18:47:39Z UpdateTip: new best=0000000000000000000457df9961a123212340daf566b899aaa54197d0648d1b height=656477 version=0x2000e000 log2_work=92.435884 tx=585898368 date='2020-11-11T18:49:01Z' prog
ress=1.000000 cache=26.5MiB(202707txo) warning='75 of last 100 blocks have unexpected version'
2020-11-11T18:47:39Z zmq: Publish rawblock 0000000000000000000457df9961a123212340daf566b899aaa54197d0648d1b
2020-11-11T18:55:44Z Pre-allocating up to position 0x5000000 in blk02316.dat
2020-11-11T18:56:10Z UpdateTip: new best=000000000000000000103c23bc9a56b6c08340f973fab41c7dfd6535700ad8f4 height=656476 version=0x37ffe000 log2_work=92.435868 tx=585895690 date='2020-11-11T18:27:18Z' prog
ress=0.999990 cache=26.0MiB(198487txo) warning='75 of last 100 blocks have unexpected version'
2020-11-11T18:56:10Z UpdateTip: new best=0000000000000000000821678efe7f53c388fd2c530ef46e6dd2d2f369c346cd height=656477 version=0x20000000 log2_work=92.435884 tx=585898371 date='2020-11-11T18:53:37Z' prog
ress=0.999999 cache=27.0MiB(206709txo) warning='74 of last 100 blocks have unexpected version'
***************************ZMQ NOTIFICATION MISSING HERE************************************************
2020-11-11T18:56:10Z Pre-allocating up to position 0xa00000 in rev02316.dat
2020-11-11T18:56:14Z UpdateTip: new best=0000000000000000000aaf4126ad8beb7f643462d5111cf89068c2f25f5742b1 height=656478 version=0x37ffe000 log2_work=92.435899 tx=585901367 date='2020-11-11T18:54:58Z' prog
ress=1.000000 cache=27.6MiB(211581txo) warning='74 of last 100 blocks have unexpected version'
2020-11-11T18:56:14Z BlockUntilSyncedToCurrentChain: txindex is catching up on block notifications
2020-11-11T18:56:14Z zmq: Publish rawblock 0000000000000000000aaf4126ad8beb7f643462d5111cf89068c2f25f5742b1

To reproduce Check your own logs if you had block zmq and debug=zmq enabled or reproduce a chainsplit on regtest.

System information Bitcoin core v0.20.1 running on Ubuntu 18.04

If I understand you correctly, the block at height 656477 with the hash 0000000000000000000821678efe7f53c388fd2c530ef46e6dd2d2f369c346cd did not get notified via the rawblock ZMQ publisher, correct?

The ZMQ documentation lists a few remarks about the ZMQ interface. One of them is:

Note that for *block topics, when the block chain tip changes, a reorganisation may occur and just the tip will be notified. It is up to the subscriber to retrieve the chain from the last known block to the new tip. Also note that no notification will occur if the tip was in the active chain--as would be the case after calling invalidateblock RPC. In contrast, the sequence topic publishes all block connections and disconnections.

Based on the documentation I'd say that ...c346cd was not published is expected behavior.

@0xB10C Well spotted, thanks!

If this behavior is intended to continue, feel free to close this issue.

@pedr0-fr Be aware that changes in zmq interfaces has been merged in master branch. See #19572 If you use the new sequence topic, all block connections and disconnections will be published. Maybe you want to give it a try.

0xB10C pointed out above this is intended behavior and also related PR #19572 has been merged. It looks like this issue can be closed.