Compensate for memory peak at flush time #10126

pull sipa wants to merge 1 commits into bitcoin:master from sipa:peak_compensation changing 1 files +1 −1
  1. sipa commented at 7:08 pm on March 30, 2017: member

    It seems that our conversion from CCoinsCacheDB into a CDBBatch, and the following processing of the batch by LevelDB causes a spike in memory usage. In various experiments (all on x86_64, though), I’ve determined that this spike is between x1.5 and x1.9.

    Take this into account ahead of time before deciding whether to flush or not.

  2. Compensate for memory peak at flush time 7228ce853d
  3. laanwj added this to the milestone 0.14.1 on Mar 30, 2017
  4. laanwj added the label Resource usage on Mar 30, 2017
  5. gmaxwell commented at 7:09 pm on March 30, 2017: contributor
    Please tag as 0.14.1.
  6. laanwj added the label UTXO Db and Indexes on Mar 30, 2017
  7. laanwj added this to the "For backport" column in a project

  8. theuni commented at 11:56 pm on March 30, 2017: member
    @sipa are any of your experiments relatively simple to reproduce? I’d like to step through the high usage spots to get a better feel for what’s happening.
  9. sipa commented at 0:02 am on March 31, 2017: member
    @theuni Using the top commit at https://github.com/sipa/bitcoin/commits/repmemusage I’ve done tests with reindexes and syncs from scratch, killing and restarting bitcoind at certain memory usage points. Then looked at (MAX_RSS - 280MB) / (last_reported_cache=_line).
  10. laanwj commented at 9:43 am on March 31, 2017: member
    utACK 7228ce8
  11. laanwj merged this on Mar 31, 2017
  12. laanwj closed this on Mar 31, 2017

  13. laanwj referenced this in commit 9ab26bf7e4 on Mar 31, 2017
  14. laanwj referenced this in commit 0b5e162b84 on Mar 31, 2017
  15. morcos commented at 12:01 pm on March 31, 2017: member
    @sipa I still object to this PR without at least some increase in the default dbcache. I believe that in the event the mempool is full and there is no loan available then performance will be unnecessarily poor. I think 300 MB (with this new calculation) is approximately the bare minimum we can achieve without likely exceeding it by accident when connecting a block. It is not appropriate for a default.
  16. laanwj removed this from the "For backport" column in a project

  17. sipa deleted the branch on Jun 23, 2017
  18. codablock referenced this in commit 9ab1787a83 on Sep 27, 2017
  19. codablock referenced this in commit cfeab93642 on Oct 12, 2017
  20. codablock referenced this in commit cfefd34f45 on Oct 24, 2017
  21. UdjinM6 referenced this in commit f10760c621 on Nov 8, 2017
  22. PastaPastaPasta referenced this in commit dbe610d8a6 on May 20, 2019
  23. DrahtBot locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-10-31 03:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me