The first pull implements a “rolling bloom filter” class that keeps track of at least the last N elements added (but has a false positive rate when queried whether an element has been added).
The second uses that class to optimize the memory usage of CNode::setAddrKnown, to a fixed 37K of memory per CNode (from about 650K before any changes, or ~300K after @sipa ’s #6064 ).
For when you need to keep track of the last N items
you've seen, and can tolerate some false-positives.
Use a probabilistic bloom filter to keep track of which addresses
we think we have given our peers, instead of a list.
This uses much less memory, at the cost of sometimes failing to
relay an address to a peer-- worst case if the bloom filter happens
to be as full as it gets, 1-in-1,000.
Measured memory usage of a full mruset setAddrKnown: 650Kbytes
Constant memory usage of CRollingBloomFilter addrKnown: 37Kbytes.
This will also help heap fragmentation, because the 37K of storage
is allocated when a CNode is created (when a connection to a peer
is established) and then there is no per-item-remembered memory
allocation.
I plan on testing by restarting a full node with an empty peers.dat,
running a while with -debug=addrman and -debug=net, and making sure
that the 'addr' message traffic out is reasonable.
(suggestions for better tests welcome)
Labels
Refactoring