Intermittent failure in p2p_1p1c_network.py", line 58, in raise_network_minfee assert_greater_than(node.getmempoolinfo()['mempoolminfee'], FEERATE_1SAT_VB) ; AssertionError: 0.00001000 <= 0.00001000 #30922

https://github.com/bitcoin/bitcoin/actions/runs/10923649833/job/30320807014?pr=30921#step:7:46110

node0 2024-09-18T14:10:27.733922Z [httpworker.2] [src/rpc/request.cpp:232] [parse] [rpc] ThreadRPCServer method=getmempoolinfo user=__cookie__ 
 test  2024-09-18T14:10:27.734000Z TestFramework (ERROR): Assertion failed 
                                   Traceback (most recent call last):
                                     File "/Users/runner/work/bitcoin/bitcoin/test/functional/test_framework/test_framework.py", line 132, in main
                                       self.run_test()
                                     File "/Users/runner/work/bitcoin/bitcoin/ci/scratch/build-aarch64-apple-darwin23.6.0/test/functional/p2p_1p1c_network.py", line 137, in run_test
                                       self.raise_network_minfee()
                                     File "/Users/runner/work/bitcoin/bitcoin/ci/scratch/build-aarch64-apple-darwin23.6.0/test/functional/p2p_1p1c_network.py", line 58, in raise_network_minfee
                                       assert_greater_than(node.getmempoolinfo()['mempoolminfee'], FEERATE_1SAT_VB)
                                     File "/Users/runner/work/bitcoin/bitcoin/test/functional/test_framework/util.py", line 82, in assert_greater_than
                                       raise AssertionError("%s <= %s" % (str(thing1), str(thing2)))
                                   AssertionError: 0.00001000 <= 0.00001000

I saw this several times today as well.

E.g. https://github.com/bitcoin/bitcoin/actions/runs/10962755696/job/30442729104?pr=29346

I think that the problem is in how fill_mempool works with multiple nodes: If node0 submits transactions so fast that they are being evicted from the mempool before they could be sent out to other peers, then the other peers will never see them, admit (and later remove) them from their mempool and therefore won't raise their minfee.

Not sure if this can be fixed within p2p_1p1c_network.py or if some sort of intermediate sync_mempool calls would need to be added to fill_mempool after the first couple of transactions.

Encountered this in PR #30946 as well.

test  2024-09-22T19:16:03.915000Z TestFramework (ERROR): Assertion failed 
                                   Traceback (most recent call last):
                                     File "/Users/runner/work/bitcoin/bitcoin/test/functional/test_framework/test_framework.py", line 132, in main
                                       self.run_test()
                                     File "/Users/runner/work/bitcoin/bitcoin/ci/scratch/build-aarch64-apple-darwin23.6.0/test/functional/p2p_1p1c_network.py", line 137, in run_test
                                       self.raise_network_minfee()
                                     File "/Users/runner/work/bitcoin/bitcoin/ci/scratch/build-aarch64-apple-darwin23.6.0/test/functional/p2p_1p1c_network.py", line 58, in raise_network_minfee
                                       assert_greater_than(node.getmempoolinfo()['mempoolminfee'], FEERATE_1SAT_VB)
                                     File "/Users/runner/work/bitcoin/bitcoin/test/functional/test_framework/util.py", line 82, in assert_greater_than
                                       raise AssertionError("%s <= %s" % (str(thing1), str(thing2)))
                                   AssertionError: 0.00001000 <= 0.00001000

Thanks @mzumsande for the detailed analysis and suggested fix. I've submitted it as #30948 to address the intermittent failure sooner rather than later. I hope it is fine.

Of course, if I'd been planning on opening a fix myself I would have mentioned that. I wasn't sure how to fix this best, see also my comment in the PR.