Save 7% total memory usage by using pointers as keys in mapNextTx #7991

pull kazcw wants to merge 3 commits into bitcoin:master from kazcw:memusage changing 6 files +88 −18
  1. kazcw commented at 8:31 pm on May 2, 2016: contributor

    I profiled bitcoind’s memory usage and noticed that the tx hashes as keys in mapNextTx are taking a large amount of space, but they’re redundant since the corresponding value has a pointer to the CTransaction that has the hash in it. On a system with 64-bit pointers, replacing these hashes with pointers or references would, once the mempool warms up, save about 7% of total memory usage.

    I explored different ways of doing this.

    I ruled out std::reference_wrapper (possibly the most straightforward way of using references as keys in a map), because it’s a big footgun and also the syntax for using it would be weird; you’d have to static_cast<COutpoint&>(it->first) to get the “value” of a key from an iterator.

    The simplest approach I found was to use a map<const COutPoint*, CInpoint, DereferencingComparator<const COutPoint*>> – but this also required strange syntax: map.find(&outpoint) is misleading, since the outpoints will be compared as values.

    The implementation here is a refinement of the DereferencingComparator approach: I created refmap as a simple proxy to a map that has pointers as keys and uses the DereferencingComparator; refmap passes through all methods except those that normally take a comparison value, which it takes by const reference and gives the address to the underlying map. This allows a map to use pointers as keys, and:

    • compare keys by pointed-to value
    • insert keys or return them from the map as pointers (which makes clear to the map user the constraint on the lifetime of the pointed-to key object)
    • perform value lookups by reference as a normal map would (making it clear that keys are compared by value)
  2. avoid explicitly referring to type of mapNextTx 485e5b7b4d
  3. eliminate extra map lookup 2b0f07cf91
  4. use a pointer as key in mapNextTx
    Saves about 7% of total memory usage (halves the size of each entry in
    mapNextTx, which comprised about 15% of memory usage in overnight test run).
    Since the mempool is DynamicUsage-regulated, this will translate to a larger
    mempool in the same amount of space.
    a0b866353c
  5. kazcw force-pushed on May 2, 2016
  6. kazcw closed this on May 3, 2016

  7. dcousens commented at 2:06 am on May 3, 2016: contributor
    Any reason for close?
  8. kazcw commented at 4:43 pm on May 3, 2016: contributor
    I’m exploring an alternative solution; I’ll PR one or the other soon.
  9. DrahtBot locked this on Sep 8, 2021


kazcw dcousens


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-17 18:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me