Store orphan blocks in serialized form #3516

pull sipa wants to merge 1 commits into bitcoin:master from sipa:serorphans-head changing 1 files +41 −17
  1. sipa commented at 10:41 pm on January 11, 2014: member

    Apparently (on 64-bit systems), an average CBlock takes up about 8 times as much space as its serialized form. This results in high memory usage for all orphan blocks that are kept. Change this to retain them in serialized form instead.

    Tested by doing a (partial) sync under valgrind, which involved processing orphans.

  2. Store orphan blocks in serialized form da0fecffa7
  3. BitcoinPullTester commented at 11:16 pm on January 11, 2014: none
    Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/da0fecffa788a0a74121d554d3d76936ab96b39e for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.
  4. sipa referenced this in commit eab1ee1f25 on Jan 12, 2014
  5. laanwj commented at 10:51 am on January 12, 2014: member
    ACK on code changes (haven’t tested memory usage, but looks like a good idea) Currently trying a full sync…
  6. sipa commented at 12:58 pm on January 12, 2014: member
    I have a branch (serorphans) which is a merge of #3514 with this; it should provide some stronger memory usage guarantees, as it also limits the maximum number of orphans in memory.
  7. laanwj commented at 1:15 pm on January 12, 2014: member

    My client appears to be stuck at block 202978 on the mainnet chain For some reason I’m receiving the entire chain starting from 0000000000000062e33609a6653b870f76f3f61e6f0aa3d9100ad12d78926a79 (#241041) to #241869 as ORPHAN BLOCK Could of course be unrelated to this pull and simply a case of bad peers.

    Oh… and it’s already syncing again :)

    (still lots and lots of orphans though, if you’re interested in the debug.log, it’s here http://download.visucore.com/bitcoin/debug-3516.log.bz2)

  8. sipa commented at 1:19 pm on January 12, 2014: member

    No, receiving orphans is very expected and one of the reason why I’m working on headers-first.

    The problem is that any peer that announces some blocks, we ask for it, and if it’s something whose parent we don’t know, we ask for the parents - starting a pretty much full sync process from this peer, even if we were already syncing from another before. The problem is just that if you’re syncing from two peers at once, some ranges of blocks are bound to be downloaded in incorrect order.

    It’s certainly possible to improve this, and try to prevent starting a second sync from another peer if another is ongoing, but this sounds hard to do right and guarantee to deal well with all weird edge cases that can arise.

  9. laanwj commented at 1:35 pm on January 12, 2014: member
    Indeed. The amount of received orphans shouldn’t be affected by these changes. So all the orphans are good for testing, in this case.
  10. laanwj commented at 7:29 am on January 13, 2014: member
    I did a succesful full sync with this patch. The orphans picked up around #202978 were successfully written when it actually came to block #241041. ACK
  11. gavinandresen commented at 4:54 pm on January 13, 2014: contributor
    ACK (reviewed changes and compiled/sanity tested on OSX).
  12. gavinandresen referenced this in commit 266921e70f on Jan 13, 2014
  13. gavinandresen merged this on Jan 13, 2014
  14. gavinandresen closed this on Jan 13, 2014

  15. DrahtBot locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-19 09:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me