Large m(un)lock overhead with lots of keys #3949

issue laanwj openend this issue on March 24, 2014
  1. laanwj commented at 1:30 pm on March 24, 2014: member

    The current secure allocator uses mlock a lot, pretty much on a per-page granularity.

    The secure allocator makes use of a LockedPageManager to keep track of locked an non-locked pages. It locks the pages returned by the system allocator and unlocks them when no ‘secure’ allocations are on that page anymore.

    It would be preferable to use a pool-based system, which pre-allocates areas of mlocked memory and deals these out as necessary.

    The uses of SecureAllocator are:

    1. SecureString (string, used for wallet passphrases in CWallet interface).
    2. CKeyingMaterial (vector<char>, used for CCrypter interface and to store the master key in CCryptoKeyStore object).
    3. CPrivKey (used to store wallet keys)

    The allocation pattern for these three is different

    • SecureString
      • is only used for small periods (argument passing).
      • never stored for longer spans of time.
      • allocated infrequently and not in performance-critical places
      • dynamically-sized array of char
    • CKeyingMaterial
      • is used for argument passing, and one of these is stored for a longer time for locked wallets
      • allocated infrequently and not in performance-critical places
      • dynamically-sized array of char
    • CPrivkey, on the other hand
      • is allocated and deallocated frequently when loading a wallet or when generating large amounts of keys (for the keypool)
      • usually they are only allocated and stored, though deallocation happens when these objects are copied around, for example as return values
      • fixed-size objects

    As the CPrivKey allocation pattern is the only one that causes performance issues and it requires only fixed-size objects no arrays, a possible solution is to use a boost::pool of mlocked memory chunks for just these.

  2. laanwj added the label Priority Low on Mar 24, 2014
  3. pstratem commented at 9:42 pm on March 24, 2014: contributor

    I’ve done some benchmarking on this.

    The results for bare metal vs visualization are significant.

    VMWare: mlock takes ~540usec E3-1245: mlock takes ~13usec

    There is certainly a significant performance gain to be had here for wallets with a large number of keys.

    However the performance gains are probably not as important as I initially thought, given that most users who want to have significant numbers of keys can afford a dedicated server for bitcoind.

  4. sipa commented at 5:27 pm on January 23, 2017: member
    Fixed by #8753, I think.
  5. sipa closed this on Jan 23, 2017

  6. DrahtBot locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-17 18:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me