WIP qt: mempool stats chart

RandyMcMillan commented at 8:14 am on May 6, 2021: contributor

add graphical mempool display to Node window

RandyMcMillan commented at 8:16 am on May 6, 2021: contributor

There is a lot if interest in #108 This is a rebase of original PR - it has grown stale.

Sjors commented at 9:20 am on May 6, 2021: member

Thanks for picking this up. Looks like it broke a few CI instances, probably because of the compiler warnings (that are treated as errors).

RandyMcMillan force-pushed on May 6, 2021

RandyMcMillan force-pushed on May 7, 2021

in src/qt/mempoolstats.cpp:24 in 1baeb53a4e outdated

19+static const int GRAPH_PADDING_RIGHT = 30+LABEL_RIGHT_SIZE;
20+static const int GRAPH_PADDING_TOP = 10;
21+static const int GRAPH_PADDING_TOP_LABEL = 10;
22+static const int GRAPH_PADDING_BOTTOM = 50;
23+
24+void ClickableTextItem::mousePressEvent(QGraphicsSceneMouseEvent *event)

jarolrod commented at 5:57 pm on May 8, 2021:

Clicking on the text item next to the RECT still reliably causes a segmentation fault

00   org.qt-project.QtCore         	0x0000000103eb5bf9 0x103c9f000 + 2190329
11   bitcoin-qt                    	0x00000001029ad160 ClickableRectItem::objectClicked(QGraphicsItem*) + 64 (moc_mempoolstats.cpp:247)
22   org.qt-project.QtCore         	0x0000000103eb6042 0x103c9f000 + 2191426
33   bitcoin-qt                    	0x00000001029acea0 ClickableTextItem::objectClicked(QGraphicsItem*) + 64 (moc_mempoolstats.cpp:134)

jarolrod commented at 6:02 pm on May 8, 2021: member

Concept ACK

I don’t know if @jonasschnelli has completely given up on his original PR yet.

This is still very much a work in progress. So, it should be labeled as such and made into a draft.

When you push changes, it would be nice if you could write a small post explaining what has changed between updates. This makes it easier for people to follow along.

RandyMcMillan renamed this:
~~qt: mempool stats chart~~
WIP qt: mempool stats chart
on May 8, 2021

RandyMcMillan marked this as a draft on May 8, 2021

hebasto added the label Feature on May 9, 2021

rebroad commented at 8:57 am on May 12, 2021: contributor

Has anyone worked out how to make the Y axis show memory usage rather than tx count? (ideally in a way that it shows a flat line once we reach the maxmempool).

Sjors commented at 7:00 pm on May 13, 2021: member

I would prefer MB over count too if it’s not too difficult. I might take a look a later to see how…

Can you split this PR a bit? E.g. one commit for the src/interfaces changes, one for mempoolstats.cpp (which probably should mostly be moved out of the GUI code), one for the UI change, etc.

Also for a followup: little dots to indicate your wallet unconfirmed transactions :-)

RandyMcMillan force-pushed on May 14, 2021

ghost commented at 2:30 am on May 15, 2021: none

Concept ACK

Has anyone worked out how to make the Y axis show memory usage rather than tx count? (ideally in a way that it shows a flat line once we reach the maxmempool).

There should be a way to select Tx count or vSize

Not sure if others will agree with it or maybe we can do this in follow up PRs, I created a rough design of mempool tab:

Left: History for mempool over a time period. User can select the time frame.

Right: Current fee rate distribution in mempool. I have used image from https://btc.bitaps.com (An option to switch between Tx count and vSize can be added)

User can enter a transaction id in the text edit widget to see where it is in the mempool (similar to https://mempool.observer):

Can be highlighted in the ‘fee distribution’ graph with dotted lines. I have used dotted lines and “glasses” sticker:

qt: mempool stats chart

Co-authored-by: Jonas Schnelli <dev@jonasschnelli.ch>
Co-authored-by: @RandyMcMillan <randy.lee.mcmillan@gmail.com>

b6d0a95612

RandyMcMillan force-pushed on May 16, 2021

0xB10C commented at 9:42 am on May 21, 2021: contributor

I agree that insights into a nodes mempool are really interesting and useful. I too understand that users want to use their own full node and not rely on external services. However, I’m not sure if the Bitcoin Core GUI is the right place for it.

From my experience, different types of users want different stats, charts, chart axis and so forth… I think having good interfaces to access this data and then visualizing it, for example, on a web page, scales way better and requires much less maintenance on the Bitcoin Core side. I’m leaning towards a Concept NACK for now.

Disclaimer: I run such a site visualizing the mempool of my node, but I don’t think this influences my reasoning here. I have no incentive to NACK here. I don’t make money off site visitors. The hosting is payed by myself and donations (sometimes listed as sponsors) and I’m not compensated for my time maintaining it.

hebasto commented at 2:51 pm on May 21, 2021: member

@0xB10C

I agree that insights into a nodes mempool are really interesting and useful. I too understand that users want to use their own full node and not rely on external services. However, I’m not sure if the Bitcoin Core GUI is the right place for it.

From my experience, different types of users want different stats, charts, chart axis and so forth… I think having good interfaces to access this data and then visualizing it, for example, on a web page, scales way better and requires much less maintenance on the Bitcoin Core side. I’m leaning towards a Concept NACK for now.

Disclaimer: I run such a site visualizing the mempool of my node, but I don’t think this influences my reasoning here. I have no incentive to NACK here. I don’t make money off site visitors. The hosting is payed by myself and donations (sometimes listed as sponsors) and I’m not compensated for my time maintaining it.

Could we limit this feature to some set of data (texts, tables, charts) that just could give to a pro-user useful info to manually set a feerate for his transaction?

Rspigler commented at 7:37 pm on May 21, 2021: contributor

I am a strong Concept ACK for @prayank23 ’s design. Improving Core’s GUI and widening the user base is important.

Sjors commented at 11:45 am on May 22, 2021: member

Personally I’m happy to review and test this, and it’s something I would use. Mempool weather is something I check before and during every transaction. Although websites can indeed iterate faster, it’s nice to have it in the node. The Hoenicke design that this is based on hasn’t changed in many years, so I’m also not terribly worried about maintenance.

And if we correctly split the data collection (node & rpc: seperate PR) from the visualisation (gui), then there may be other use cases too.

RandyMcMillan closed this on Jun 7, 2021

Rspigler commented at 4:31 am on June 9, 2021: contributor

:disappointed: Reason for close? Mark up for grabs?

ghost commented at 3:36 am on June 17, 2021: none

@0xB10C

However, I’m not sure if the Bitcoin Core GUI is the right place for it.

Mempool related charts and options are a basic thing which should have been there in Bitcoin Core GUI few years back. Sad that we won’t see it in v22.0 but still not too late to start working on it for future release.

From my experience, different types of users want different stats, charts, chart axis and so forth.

Agree. But that is true for lot of things in Bitcoin Core GUI. We should add atleast basic things that everyone finds useful.

I think having good interfaces to access this data and then visualizing it, for example, on a web page, scales way better and requires much less maintenance on the Bitcoin Core side.

Yes, website works better for charts. However, both can be used. One or two basic charts and options in Core GUI and other detailed mempool stats on different websites.

Is https://mempool.observer open source? Can users run it locally and use their own node? That would be helpful

Rspigler commented at 3:53 am on June 21, 2021: contributor

users want to use their own full node and not rely on external services

This.

rebroad commented at 4:55 pm on June 26, 2021: contributor

I would prefer MB over count too if it’s not too difficult. I might take a look a later to see how…

Can you split this PR a bit? E.g. one commit for the src/interfaces changes, one for mempoolstats.cpp (which probably should mostly be moved out of the GUI code), one for the UI change, etc.

Also for a followup: little dots to indicate your wallet unconfirmed transactions :-)

I’ve managed to do this… but I cheated a little bit as there is memory used in addition to the TXs that fluctuates over time, so if you want a nice flat line when your node starts pruning TXs from the mempool, then you won’t get that unless you cheat :) (which I have). I will upload my patches soon.

kristapsk commented at 8:47 pm on July 12, 2021: contributor

Concept ACK

However, I’m not sure if the Bitcoin Core GUI is the right place for it.

Mempool related charts and options are a basic thing which should have been there in Bitcoin Core GUI few years back. Sad that we won’t see it in v22.0 but still not too late to start working on it for future release.

I agree with @prayank23 here, if you use Bitcoin Core as a wallet (and I don’t see a reason running GUI if you don’t), this is probably the most important metric to decide what transaction fee to use for transaction.

promag commented at 10:34 am on July 13, 2021: contributor

Concept ACK. @0xB10C I think some basic mempool viz is reasonable. Will review.

0xB10C commented at 10:56 am on July 13, 2021: contributor

Thanks everybody for the feedback and discussion. I’ve reconsidered my position on this. I agree that some basic mempool weather in the GUI makes sense.