WIP: mempool stats chart

jonasschnelli commented at 2:38 pm on October 23, 2020: contributor

This PR adds a mempool statistics to the “debug” window. The diagram is heavy inspired by Jochen Hoenicke’s webversion (Thanks @jhoenicke for that).

macOS:

Ubuntu:

The approach is it to collect the fees also when the mempool chart or the node window is not open (collect it from the start of the application). Clicking on the fee-range color square highlights the fee group over time.

To keep the scope minimal:

flexible timeframe is out of scope (can be added later)

See also https://github.com/bitcoin/bitcoin/pull/8550 (an initial attempt, different concept).

TODO:

Make it non HiDPI compatible (linux/win/etc.).
Analyze memory consumption (should not be too bad)
Optimize performance
Try to make fee ranges dynamic

jonasschnelli added the label Feature on Oct 23, 2020

jonasschnelli added this to the milestone 0.22.0 on Oct 23, 2020

jonasschnelli force-pushed on Oct 23, 2020

jonatack commented at 2:55 pm on October 23, 2020: contributor

Concept ACK – nice!

jonasschnelli force-pushed on Oct 24, 2020

WIP: mempool stats chart bc785672c5

jonasschnelli force-pushed on Oct 24, 2020

RandyMcMillan commented at 10:03 pm on October 25, 2020: contributor

Should the background color of the Network Traffic display be changed to white as well? - to create a consistent UX within the different tabs? If so - should they share a common theme - so they can be toggled together?

RandyMcMillan commented at 11:55 pm on October 25, 2020: contributor

Concept ACK :)

It seems that the parent window should trigger a repaint/resize of the view simular to the peer table data model…

https://github.com/bitcoin-core/gui/blob/d67883d01e507dd22d1281f4a4860e79d6a46a47/src/qt/peertablemodel.cpp#L219

Screen Shot 2020-10-25 at 6 33 17 PM

A similar implementation may be useful to actually hide the tab until the node is up and running and some initial values have been populated
Should flushing the mempool prior to shut down - as an option - be added to the tab as a check box? default no? This would allow the window to check cached data to speed up the initial paint.

https://github.com/bitcoin-core/gui/blob/d67883d01e507dd22d1281f4a4860e79d6a46a47/src/qt/bitcoingui.cpp#L840

We can attempt to use similar settings to make a more consistent UX between the tabs by using the same dimensions for the data presentation view and the margin underneath…

PR #90

promag commented at 0:55 am on October 26, 2020: contributor

Concept ACK.

Sjors commented at 11:34 am on October 26, 2020: member

Very cool, concept ACK.

jonasschnelli commented at 1:08 pm on October 26, 2020: contributor

@RandyMcMillan Thanks for the feedback. I’d like to avoid visual alignment of both of the graphs in this PR. Off course this should happen but my takeaway from bitcoin/bitcoin#8550 is that scope creep must be avoided if we want to get this in master.

I recommend to do the visual alignment later on.

in src/qt/mempoolstats.cpp:176 in bc785672c5

171+                continue;
172+            }
173+            ClickableRectItem *fee_rect = new ClickableRectItem();
174+            fee_rect->setRect(4, c_y, c_w, c_h);
175+
176+            QColor brush_color = colors[(i < static_cast<int>(colors.size()) ? i : static_cast<int>(colors.size())-1)];

Sjors commented at 11:06 am on October 30, 2020:

0comparison of integers of different signs: 'int' and 'std::__1::vector<QColor, std::__1::allocator<QColor> >::size_type' (aka 'unsigned long') [-Werror,-Wsign-compare]

(a bunch of other warnings too)

Sjors commented at 11:17 am on October 30, 2020: member

The peers tab is dropped from the menu, try expanding std::vector<TabTypes> tabs() in rpcconsole.h

I managed to get a segfault by clicking on the chart (macOS), though only once. Also when clicking on an item in the legend.

You could split this into two PR’s, one where the node collects all the histogram data and exposes it via RPC (with a functional test). And a GUI PR on top.

hebasto commented at 8:43 pm on October 30, 2020: member

Concept ACK.

jarolrod commented at 7:39 pm on January 21, 2021: member

Concept ACK

One Note: On macOS 11.1, Qt 5.15.2 - I can reliably cause a segmentation fault by clicking on any of the text on the left side. I can click on the corresponding box, but clicking on the text will cause a segmentation fault. Here is a pastebin link to the error report: https://pastebin.com/E7my5X8V

ghost commented at 11:46 pm on March 4, 2021: none

IMO the bar graph used by https://btc.bitaps.com is better than https://jochen-hoenicke.de/queue/#BTC%20(default%20mempool),24h

Reason: Its easy to look at it and use the information while broadcasting a bitcoin transaction

hebasto commented at 2:36 pm on March 19, 2021: member

IMO the bar graph used by https://btc.bitaps.com is better than https://jochen-hoenicke.de/queue/#BTC%20(default%20mempool),24h

Reason: Its easy to look at it and use the information while broadcasting a bitcoin transaction

I like it. It uses the widget area more effective.

rebroad commented at 10:40 am on March 29, 2021: contributor

It would be ideal if the functionality of the previous graph and the new could be combined. e.g. allow different durations to be shown (from 3 hours, to 2 weeks), have one line for tx count, but show the fee rates within the memory usage graph. The scale (tx count and MB usage) could both be shown on the right, with the fee-rates shown on the left. Change the colours slightly so that the minfeerate and tx count are still clearly visible if plotted over the memusage/feerate. Oh, and ideally the window could be a separate window, rather than as a tab on the debug window.

At least, this would be perfection IMHO :)

rebroad commented at 12:14 pm on March 29, 2021: contributor

The functionality of clicking on a fee range and it displaying the number of transactions seems a bit off. In this example it’s saying there are over 1,000,000 transactions in the 30-40 satoshi range, yet the vertical axis only goes up to 20,000 tx.

RandyMcMillan commented at 8:18 am on May 6, 2021: contributor

#320 is a rebase of this PR.

hebasto commented at 1:46 pm on May 29, 2021: member

The “22.0” milestone seems unrealistic here. Removing it.

hebasto removed this from the milestone 22.0 on May 29, 2021

rebroad commented at 12:49 pm on July 3, 2021: contributor

Thinking it might be nice if the vertical axis scaled like it did on the old graph so that more detail can be seen…

Rspigler commented at 4:24 pm on July 13, 2021: contributor

@jonasschnelli Do you still have plans to work on this? Thanks!

laanwj commented at 4:09 pm on August 22, 2021: member

Concept ACK, this looks really neat. I think it’s good for people running the GUI to have some introspection into what is happening on their node, and visualizations like this are useful for that.

DrahtBot added the label Needs rebase on Dec 13, 2021

DrahtBot commented at 11:14 pm on December 13, 2021: contributor

🐙 This pull request conflicts with the target branch and needs rebase.

Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a “draft”.

DrahtBot commented at 12:12 pm on March 21, 2022: contributor

Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
Is it no longer relevant? ➡️ Please close.
Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.

hebasto commented at 3:14 pm on April 13, 2022: member

Closing due to inactivity. Feel free to reopen.

hebasto closed this on Apr 13, 2022

Rspigler commented at 6:32 pm on April 26, 2022: contributor

Mark up for grabs?

hebasto added the label Up for grabs on Apr 27, 2022

hebasto commented at 7:09 pm on April 27, 2022: member

Mark up for grabs?

Done.

bitcoin-core locked this on Apr 27, 2023

WIP: mempool stats chart #108