rpc: Faster getblock using PureBlock #17529

1. 
   maflcko commented at 3:58 am on November 20, 2019: member

   This adds and uses a new transaction type PureTransaction. The underlying class has no transaction hash compiled in. So it is safe to use (and faster) wherever it compiles.

   The existing CMutableTransaction retains the behavior to calculate the tx hash on demand. Use of CMutableTransaction makes compile-time complexity analysis impossible.

   The existing CTransaction retains the behavior to cache the hash on initialization.

2. maflcko force-pushed on Nov 20, 2019
3. DrahtBot added the label Build system on Nov 20, 2019
4. DrahtBot added the label Consensus on Nov 20, 2019
5. DrahtBot added the label RPC/REST/ZMQ on Nov 20, 2019
6. DrahtBot added the label Tests on Nov 20, 2019
7. DrahtBot added the label Validation on Nov 20, 2019
8. 
   DrahtBot commented at 8:44 am on November 20, 2019: contributor

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Conflicts

   Reviewers, this pull request conflicts with the following ones:

   • #25694 (refactor: Make CTransaction constructor explicit by MarcoFalke)
   • #25665 (BResult improvements, allow returning separate value on failure by ryanofsky)
   • #25232 (rpc: Faster getblock API by AaronDewes)
   • #23599 ([WIP] DRAFT NOMERGE Tidy up RPCTxSerializationFlags by MarcoFalke)

   If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

9. maflcko removed the label Build system on Nov 20, 2019
10. maflcko removed the label Consensus on Nov 20, 2019
11. maflcko removed the label Tests on Nov 20, 2019
12. 
    maflcko commented at 4:12 pm on November 20, 2019: member

    So I did some measurements: While this is a clear performance win, it seems the performance is dominated by shuffling the data around multiple times:

    Before: disk -> deserialize -> hash -> serialize -> hex string -> json

    After: disk -> deserialize -> ____ -> serialize -> hex string -> json

    Hashing takes 10%-25% on intel CPUs. Didn’t try arm.

    master:

    faaa142412:

    

13. 
    maflcko commented at 4:22 pm on November 20, 2019: member

    I guess it would be smarter to see if the rpc serialization flags are matching the ones of the block on disk and then just serve it raw. See #13151
14. maflcko added the label Brainstorming on Nov 20, 2019
15. DrahtBot added the label Needs rebase on Dec 16, 2019
16. 
    laanwj commented at 12:17 pm on December 17, 2019: member

    I guess it would be smarter to see if the rpc serialization flags are matching the ones of the block on disk and then just serve it raw. See #13151

    If so, it would definitely be a simpler/smaller change!

17. refactor: Add PureTransaction and PureBlock (no tx hash)

    Those new types are currently unused. The underlying class has no
    transaction hash compiled in. So it is safe to use wherever it compiles.
    
    The existing CMutableTransaction retains the behavior to calculate the
    tx hash on demand. Use of CMutableTransaction makes compile-time
    complexity analysis impossible.
    
    The existing CTransaction retains the behavior to cache the hash on
    initialization.

    39f89b31f9
18. bench: Add PureBlock deserialization benchmark 31bf7f635d
19. zmq: Send blocks faster by serializing PureBlock b60e452aef
20. rpc: Faster getblock by using PureBlock 853cea80b9
21. maflcko force-pushed on Jul 25, 2022
22. DrahtBot removed the label Needs rebase on Jul 25, 2022
23. 
    DrahtBot commented at 12:17 pm on July 26, 2022: contributor

    🐙 This pull request conflicts with the target branch and needs rebase.

    Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a “draft”.

24. DrahtBot added the label Needs rebase on Jul 26, 2022
25. 
    achow101 commented at 6:04 pm on October 12, 2022: member

    Closing this as it has not had any activity in a while. If you are interested in continuing work on this, please leave a comment so that it can be reopened.
26. achow101 closed this on Oct 12, 2022

    ---

27. bitcoin deleted a comment on Dec 8, 2022
28. maflcko deleted the branch on Dec 8, 2022
29. bitcoin locked this on Dec 8, 2023

Contributors
maflcko DrahtBot laanwj achow101

Labels
Brainstorming RPC/REST/ZMQ Validation Needs rebase