mempool superblock redesign

jgarzik commented at 5:33 pm on February 21, 2014: contributor

One of the proposals floated about, originally from Gavin or @sipa IIRC, was modelling the mempool after a “superblock.” This idea seems to elegantly solve a few problems.

Model the memory pool as one big block, whose size is constrained to X 1MB blocks. This super-block is sorted according to default bitcoind block-building rules: fee per kB, with a little area for under-fee (zero fee) high priority transactions. Any newly arriving transaction is either (a) added to the memory pool, possibly triggering removal of other transactions, or (b) dropped and not relayed.

This would seem to deal with spam dynamically, as transactions would drop off the low end of the superblock. This also effectively caps the size of the memory pool, reducing DoS attack surface. And, it plays into transaction lifetime work (#3722).

jgarzik added the label Brainstorming on Feb 21, 2014

luke-jr commented at 6:05 pm on February 21, 2014: member

+1 except for “…according to default bitcoind block-building rules…”

awfm9 commented at 10:16 am on February 27, 2014: none

Why is this not being discussed / not moving forward? I personally have been lurking on the Bitcoin github repository, just hoping for this proposal to find acceptance.

Not only would this proposal fix a lot of DoS & scalability issues. You could also decide to replace an old transaction in the superblock if a new transaction has the same inputs, but higher priority. At first glance, this will mitigate the problem of transactions getting stuck in the mempool.

However, it would also be a really good way to have dynamic fees, as planned, work even better.

laanwj commented at 11:09 am on February 27, 2014: member

@awishformore wishing and hoping doesn’t get us anywhere in life, if you want things to move forward, get involved in coding, reviewing and/or testing.

awfm9 commented at 4:09 pm on February 27, 2014: none

I wish I already had enough experience with the Bitcoin source to actively contribute. However, even if no one is working on this, at least the discussion should happen? What if someone decided to put all the work into coding this, and then some core developers just decide it’s not the way to go?

jgarzik commented at 5:58 pm on February 27, 2014: contributor

That happened a few times in the Linux kernel. Folks worked for months on a kernel subsystem, then upstream ultimately decided to go in a different direction. Hopefully you try to communicate and be guided by consensus, for or against an idea. Sometimes field testing is the only way to know an answer for certain.

laanwj commented at 8:36 am on February 28, 2014: member

You’re completely right. Not communicating anything is while working on it usually causes disappointment for all parties involved. I’m just tired of all the ‘waiting for the devs’ passivity.

As for the proposal I think this is a good approach overall. One danger is that the relaying/mempool rules are no longer predictable, and will very per node as well as over time. But on the other hand I think that’s unavoidable. Even currently there are differences between nodes due to the version.

It means that clients have to adapt and cope with non-relaying and non-confirming transactions better.

laanwj added the label Mempool on Jan 8, 2015

MarcoFalke commented at 7:49 pm on September 27, 2016: member

This also effectively caps the size of the memory pool, reducing DoS attack surface. And, it plays into transaction lifetime work (#3722).

This is now achieved by mempool limiting. Closing.

MarcoFalke closed this on Sep 27, 2016

DrahtBot locked this on Sep 8, 2021

mempool superblock redesign #3723