[WIP] sync mempool after new block #7955

No description provided.

Alas, I like this a lot but I think we can't use the existing mempool command for it: It's not filtered by set inventory known. (it's also computationally expensive even when fee filter is used).

What I'd like is a "TOPMEMPSYNC", which is functionally equal to mempool but considers only the top 2*BLK_SIZE txn in 'fee-rate' order (actual order can be implementation defined but should approximate mining preferences), and also filters its output through setinventoryknown. Then you can pick a peer and keep pulling from it every block and get one more block worth of transactions one block ahead of needing them.

@gmaxwell agreed, i opened this pull since i'm actually using very similar code for a mining setup where the pool's getblocktemplate node has maxmempool=2 for performance reasons

Go make a pass at actually implementing what I described? w/ the indexes in place it should take less than a half hour to give it a shot. (also, I'd like it for the block relay testing I'm doing... having to wait hours after it starts to get good stats is lame).

@gmaxwell will do, i'm not sure you want to filter through setinventoryknown though, since you might have filtered things out for having too low a fee which are now valid

To bad, so sad. -- it doesn't need to be perfect, ... otherwise the overhead of sending 2 blocks worth where it would end up sending the same txn over and over again makes it less attractive. With the filtering it would add basically no protocol overhead to run this with a single peer for the first 24 hours that you're up or so.

@gmaxwell it's just sending the transaction ids, 2 * MAX_BLOCK_SIZE / 250 * 23 = 256000

which yeah isn't optimal... but i guess it could be short ids actually? it is once per block

Overhead equal to 1/4 a maximum size block, every block.. vs no overhead compared to regular relay (only sending IDs you haven't seen before)... it's not even comparable. The short ID stuff works much better in the context of blocks because you don't need them for fetching and can salt with the block hash.

@gmaxwell i guess i need a BIP if im going to add a protocol message?

Not to write the code and begin testing with it! It would be such a simple spec, there would be no reason to not go write a BIP for it right away too...

Though I also think it would get subsumed by a greater mempool sync BIP later. But that is going to take a while, this can easily be done now and would be useful now.

Moving to issue