Block postprocessing is (relatively) slow. #8333

pstratem commented at 4:36 AM on July 12, 2016: contributor

On a system with high uptime and absurdly large dbcache and maxsigcachesize settings the time to remove conflicts is now approximately 50% of the time to process a new block.

This matters for miners time to switch to the new block.

Example:

2016-07-12 04:27:18.506662   - Load block from disk: 0.00ms [0.02s]
2016-07-12 04:27:18.506710     - Sanity checks: 0.00ms [0.01s]
2016-07-12 04:27:18.506759     - Fork checks: 0.05ms [0.03s]
2016-07-12 04:27:18.534023       - Connect 2977 transactions: 27.23ms (0.009ms/tx, 0.007ms/txin) [13.46s]
2016-07-12 04:27:18.534892     - Verify 4142 txins: 28.13ms (0.007ms/txin) [35.45s]
2016-07-12 04:27:18.546111     - Index writing: 11.21ms [4.49s]
2016-07-12 04:27:18.546149     - Callbacks: 0.05ms [0.03s]
2016-07-12 04:27:18.546589   - Connect total: 39.93ms [40.08s]
2016-07-12 04:27:18.551713   - Flush: 5.11ms [3.35s]
2016-07-12 04:27:18.551766   - Writing chainstate: 0.07ms [0.19s]
2016-07-12 04:27:18.583436 UpdateTip: new best=000000000000000002576865a40fcb68bc012366d3c08f7ae11a1df793d6eb0c height=420364 version=0x30000000 log2_work=84.974337 tx=141859971 date='2016-07-12 04:27:52' progress=1.000000 cache=770.0MiB(245590tx) warning='6 of last 100 blocks have unexpected version'
2016-07-12 04:27:18.583641   - Connect postprocess: 31.87ms [13.50s]
2016-07-12 04:27:18.583658 - Connect block: 76.99ms [57.14s]

laanwj added the label Validation on Jul 12, 2016

sipa commented at 6:36 AM on July 12, 2016: member

I believe that #8313 and/or #7946 may help here.

pstratem commented at 10:06 PM on July 12, 2016: contributor

This is measured on a bitcoind without a wallet so #7949 wont help, I will give #8313 a try.

rodentrabies commented at 10:00 PM on July 17, 2016: contributor

@pstratem So, have you tried it with #8313? Would be great to know if it makes any difference.

pstratem commented at 1:16 AM on July 18, 2016: contributor

Appears to have improved things significantly.

2016-07-18 01:13:47.424571   - Load block from disk: 0.00ms [0.03s]
2016-07-18 01:13:47.424604     - Sanity checks: 0.00ms [0.01s]
2016-07-18 01:13:47.424646     - Fork checks: 0.04ms [0.02s]
2016-07-18 01:13:47.466739       - Connect 1330 transactions: 42.07ms (0.032ms/tx, 0.012ms/txin) [12.27s]
2016-07-18 01:13:47.496563     - Verify 3483 txins: 71.90ms (0.021ms/txin) [29.65s]
2016-07-18 01:13:47.497006 Pre-allocating up to position 0x1000000 in rev00573.dat
2016-07-18 01:13:47.502868     - Index writing: 6.32ms [3.48s] 
2016-07-18 01:13:47.502894     - Callbacks: 0.03ms [0.02s]
2016-07-18 01:13:47.502993   - Connect total: 78.43ms [33.27s]
2016-07-18 01:13:47.513180   - Flush: 10.17ms [2.89s]
2016-07-18 01:13:47.513230   - Writing chainstate: 0.07ms [0.13s]
2016-07-18 01:13:47.530087 UpdateTip: new best=00000000000000000430823cb0b995d1408e0eeb73acc00510043257fecc2129 height=421209 version=0x20000000 log2_work=85.003443 tx=143060917 date='2016-07-18 01:12:12' progress=0.999999 cache=933.7MiB(235385tx) warning='5 of last 100 blocks have unexpected version'
2016-07-18 01:13:47.530203   - Connect postprocess: 16.97ms [11.58s]
2016-07-18 01:13:47.530219 - Connect block: 105.64ms [47.89s]
2016-07-18 01:13:47.531539 received: inv (109 bytes) peer=78

(Ignore the total numbers they're skewed by the mempool not being full at start)

laanwj added the label Resource usage on Jan 26, 2017

Sjors commented at 3:58 PM on March 16, 2018: member

@eklitzke might find this interesting

eklitzke commented at 4:38 PM on March 16, 2018: contributor

Set your dbcache smaller.

jnewbery commented at 7:00 PM on March 28, 2018: member

@pstratem - have you retried this since #8515 was merged?

jnewbery closed this on Mar 28, 2018

jnewbery reopened this on Mar 28, 2018

pstratem closed this on Jun 8, 2019

MarcoFalke locked this on Dec 16, 2021