Performance fix for #6312 #7190

pull NicolasDorier wants to merge 13 commits into bitcoin:master from NicolasDorier:sequencenumbers changing 17 files +340 −123
  1. NicolasDorier commented at 7:53 am on December 9, 2015: contributor

    Build on #6312 commit added is 0559bb24e6e92a2d790c77707e464aa1e17919ca :

    • Remove lock time verification in CNB because the mempool is always finality coherent.
    • Improve removeForReorg by not calling CheckLockTime on every transaction in the mempool during a reorg.
  2. Reverse structure of IsFinalTx()
    Instead of checking if the transaction is in a category of cases to return true and failing otherwise, we return false if the tx is a failure case and otherwise return true. This removes an early-exit opportunity which will no longer be possible once sequence numbers are re-enabled, but has no change in behaviour.
    30392955af
  3. Get rid of CTxIn::IsFinal(), which is only used in one location and whose semantics will not survive revival of sequence numbers, and introduce CTxIn::SEQUENCE_FINAL constant instead. 998230b446
  4. Expand IsFinalTx() to the structure it will have when it becomes LockTime()
    Breaks up the modifications to IsFinalTx() / LockTime() for easier review. This commit does not change the logic of IsFinalTx().
    bf45b9fa5a
  5. Add rules--presently disabled--for using sequence numbers as a relative lock time
    In summary, the sequence number of each input in a transaction can be used to store a relative locktime: a delta value which is added to the height or time of the block which included the output being spent to generate a per-input lock time from the sequence number. If enforced, this would make consensus-enforced transaction replacement possible as replacing an input with a lower sequence number / relative lock time would allow the new transaction to make it on the chain before previous versions of the transaction become valid.
    
    This commit does not implement the transaction replacement logic. Nor does this commit make the relative lock time rules as consensus or policy rules. It merely adds new, but disabled functionality and unit tests of that functionality.
    deedf00b96
  6. Enable policy enforcing sequence numbers as a relative lock time in the mempool and transaction selection code, for the purpose of later supporting consensus-enforced transaction replacement
    Transactions that fail relative lock time checks will be rejected from the mempool and not included in generated blocks, making it easy to test the feature. However blocks including transactions containing "invalid" relative lock times will still be accepted; this is *not* the soft-fork required to actually enable relative lock times for production use.
    8110e73112
  7. Add AssertLockHeld(cs_main) to LockTime() function. 33ec593aa1
  8. Transform LockTime to a pure logic function, while not breaking existing code fc4bfb40bd
  9. Make LockTime function easier to use by not requiring the tip, and by making it responsible to calculate the right blockTime from flags
    LockTime() : the nCoinTime of a confirmed coin does not depend on MTP
    flag, so the nCoinTime of an unconfirmed also should not be.
    
    Addressing sipa's nits
    aa83819cac
  10. Make LockTime use a height map of inputs 6bf228e4c3
  11. Rename flags
    _SECONDS_FLAG to _TYPE_FLAG
    _DISABLED_FLAG to _DISABLE_FLAG
    bd5a760e69
  12. Explain the reason for unsigned cast of tx.nVersion ae5d345a62
  13. Fix minor bugs in miner_tests
    There was an extraneous tx being added to the mempool which spent the same inputs.  This only didn't cause a problem because locktimes stopped it from being selected for the template.
    
    In addition, since the merger of MedianTimePast, locked txs that were time based were not being correctly tested in the unit test.
    4a010348d8
  14. morcos commented at 11:17 am on December 9, 2015: member

    Any tx that has been in your mempool for longer than 10 blocks will have to be recalculated from scratch during a reorg. In addition any transaction which entered your mempool more recently than the reorg depth will have to recalculated, so you will always have to recalculate at least the txs that came in since the last blcok.

    I think if we are going to try to make reorgs fast, we should do it properly.

  15. Do not recalculate LockTime for a mempool entry in removeForReorg if the tip at the time of calculation is still in the main chain 2615613765
  16. NicolasDorier force-pushed on Dec 9, 2015
  17. NicolasDorier commented at 2:40 pm on December 9, 2015: contributor

    We could potentially scan the whole header chain instead of only 10, but I don’t think this is a good idea performance wise. I think this is better to only scan 10 and just recalculate locktime if it is more, rather than wasting time scanning to the genesis. I can change that though if you think it is indeed faster to scan to genesis. MempoolEntry older than 10 blocks are rather rare I would expect.

    In addition any transaction which entered your mempool more recently than the reorg depth will have to recalculated, so you will always have to recalculate at least the txs that came in since the last blcok.

    I don’t understand your point. Imagine tip = Block A, the tx1 comes in mempool. Then a reorg happens that replace BlockA with BlockB then tx1 will have, as expected, locktime invalidated. This must be done, and can’t be optimized, because the blockTime in LockTime(), MTP or not, will have changed.

    If I understand your point : yes, during a reorg, all transaction which came since the last block will be invalidated, and they should be.

  18. morcos commented at 2:52 pm on December 9, 2015: member
    Yes that is my point. However with the solution in #7187 it will not be necessary to relook up the inputs to the the tx and recalculate the sequence locks unless any of the inputs to the tx were in that most recent block, which seems likely to be considerably rarer.
    Technically if its a 1 block reorg, the blockTime won’t have changed (with MTP) but I’m not suggesting not rechecking that. I’m just saying you don’t have to recalculate the time/height at which the sequence locks are valid which is the expensive part. With your solution I think you will still be recalculating sequence locks for a significant fraction of the mempool on a reorg which is slow and destroys your utxo cache.
  19. NicolasDorier commented at 3:03 pm on December 9, 2015: contributor

    Technically if its a 1 block reorg, the blockTime won’t have changed (with MTP)

    The blockTime will have changed. The old tip (BlockA) was part of the median. When calculating LockTime of a mempool entry, we are assuming the tx will get in next block just after tip, so BlockA or BlockB are part of MTP calculation.

    I’m just saying you don’t have to recalculate the time/height at which the sequence locks are valid which is the expensive part.

    I need to. After a reorg, the “birthdate” of the inputs may have changed, and so, the minimum time/height (what you call LockPoint) to satisfy as well. Am I missing something ?

  20. morcos commented at 4:02 pm on December 9, 2015: member

    The blockTime will have changed. The old tip (BlockA) was part of the median.

    Yes sorry, you are right about that, anyway we both agreed it should be checked.

    I need to. After a reorg, the “birthdate” of the inputs may have changed, and so, the minimum time/height (what you call LockPoint) to satisfy as well. Am I missing something ?

    Yes you need to in your code. I’m saying my approach saves that.

  21. jtimon commented at 4:37 pm on December 11, 2015: contributor
    I’m having troubles understanding what is wrong with this solution for #7176. It is very nice that it doesn’t have to touch the 6-months old (and already reviewed) consensus code (#6312) further, and I think that makes it the less risky and less development-resources-wasteful option for #7176.
  22. NicolasDorier commented at 10:52 am on December 12, 2015: contributor

    I agree, with jtimon. What is wrong with this code is the I recheck more than I need to. In this code, if there is a 1 block reorg (block A replaced by block B), then all mempool transaction which came after BlockA will need to be rechecked. Morcos solution does not have this problem.

    I myself prefer not touching the code that was heavily reviewed though.

  23. laanwj added the label Mempool on Dec 14, 2015
  24. btcdrak commented at 7:31 pm on January 20, 2016: contributor
    @NicolasDorier this should be closed as the related ticket was also closed and superseded.
  25. NicolasDorier closed this on Jan 21, 2016

  26. DrahtBot locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-09-29 07:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me