Simplify hash.h interface using Spans #19326

pull sipa wants to merge 8 commits into bitcoin:master from sipa:202006_spanhashes changing 31 files +117 −120
  1. sipa commented at 2:29 am on June 19, 2020: member

    This makes use of the implicit constructions and conversions to Span introduced in #18468 to simplify the hash.h interface:

    • All functions that take a pointer and a length are changed to take a Span instead.
    • The Hash() and Hash160() functions are changed to take in “range” objects instead of begin/end iterators.
  2. fanquake added the label Refactoring on Jun 19, 2020
  3. sipa force-pushed on Jun 19, 2020
  4. sipa commented at 3:53 am on June 19, 2020: member
    @jb55 You may be interested in this.
  5. DrahtBot commented at 12:25 pm on June 19, 2020: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #19628 (net: change CNetAddr::ip to have flexible size by vasild)
    • #19601 (Refactoring CHashWriter & Get{Prevouts,Sequence,Outputs}Hash to SHA256 (Alternative to #18071) by JeremyRubin)
    • #19245 ([WIP DONOTMERGE] Replace boost::filesystem with std::filesystem (in c++17) by kiminuo)
    • #19183 ([WIP DONOTMERGE] Replace boost with C++17 by MarcoFalke)
    • #18985 (bloom: use Span instead of std::vector for insert and contains [ZAP3] by jb55)
    • #18071 (Refactoring CHashWriter & Get{Prevouts,Sequence,Outputs}Hash to SHA256 by JeremyRubin)
    • #17977 (Implement BIP 340-342 validation (Schnorr/taproot/tapscript) by sipa)
    • #16841 (Replace GetScriptForWitness with GetScriptForDestination by meshcollider)
    • #10102 ([experimental] Multiprocess bitcoin by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  6. jb55 commented at 3:12 pm on June 19, 2020: member
    yup this will clean up some of my PRs. Concept ACK (I would almost say utACK but I just woke up).
  7. DrahtBot added the label Needs rebase on Jun 21, 2020
  8. jonatack commented at 5:42 pm on June 25, 2020: member
    Concept ACK. Read the code, built/tested/running bitcoind, need to do more detailed code review stepping through the commits.
  9. practicalswift commented at 9:12 pm on June 25, 2020: contributor
    Concept ACK: Span<const unsigned char> as a drop-in replacement for const std::vector<unsigned char>& as parameter type is safe use of Span :)
  10. jonatack commented at 8:16 am on June 26, 2020: member
    ACK 974381f1c53ff0674e22a40d9553f962fa0df108 a useful clear abstraction and nice cleanup. Code review, compared the implementation of as_(writable_)bytes to that shown in https://en.cppreference.com/w/cpp/container/span/as_bytes (if you retouch, s/to_/as_/ in the 45316c20 commit message). Built in various ways, ran tests and benches locally, ran bitcoind without issues for a day.
  11. sipa force-pushed on Jun 26, 2020
  12. sipa commented at 8:39 pm on June 26, 2020: member
    Rebased and addressed @jonatack’s comment.
  13. DrahtBot removed the label Needs rebase on Jun 26, 2020
  14. sipa force-pushed on Jun 26, 2020
  15. jonatack commented at 12:00 pm on June 28, 2020: member
    re-ACK fc7c518
  16. DrahtBot added the label Needs rebase on Jun 28, 2020
  17. sipa force-pushed on Jul 1, 2020
  18. DrahtBot removed the label Needs rebase on Jul 1, 2020
  19. jonatack commented at 3:50 am on July 8, 2020: member
    Code review re-ACK 675d6fd per git range-diff abdfd2d fc7c518 675d6fd
  20. in src/span.h:213 in 4679ef69f1 outdated
    207@@ -206,4 +208,20 @@ T& SpanPopBack(Span<T>& span)
    208     return back;
    209 }
    210 
    211+//! Mimick C++20's std::as_bytes (with unsigned char instead of std::byte)
    212+template<typename T>
    213+Span<const unsigned char> AsUnsignedChars(Span<T> in) { return {reinterpret_cast<const unsigned char*>(in.data()), in.size_bytes()}; }
    


    ryanofsky commented at 9:38 pm on July 16, 2020:

    In commit “Make uint256 Span-convertible by adding ::data()” (5d1f1d93154f3cee384e121963940141603e01f7)

    I think it’d be better to avoid doing potentially unsafe reinterpret_casts when really we only need to cast char pointers to unsigned char pointers.

    My concern is that providing this function might make it too easy to write code that looks like it is hashing vector<uint256> or Optional<uint256> or uint256* data, but actually hashing the outer pointer or container objects instead.

    I think we could provide a more limited convenience function, similar to our current CharCast function that’s just restricted to chars and won’t reinterpret arbitrary spans of objects.

    Experimenting a little, the following seems to work well:

    0inline unsigned char* UnsignedCharCast(char* c) { return (unsigned char*)c; }
    1inline unsigned char* UnsignedCharCast(unsigned char* c) { return c; }
    2inline const unsigned char* UnsignedCharCast(const char* c) { return (unsigned char*)c; }
    3inline const unsigned char* UnsignedCharCast(const unsigned char* c) { return c; }
    4
    5//! Safely convert a character array into a span of unsigned chars.
    6template<typename T>
    7Span<const unsigned char> UnsignedCharSpan(const T& in) { auto span = MakeSpan(in); return {UnsignedCharCast(span.data()), span.size_bytes()}; }
    

    Another advantage of this is allows replacing AsUnsignedChars(MakeSpan(in)) with simpler UnsignedCharSpan(in) everywhere.


    sipa commented at 5:01 am on July 21, 2020:

    Agree, that seems like both a more convenient and slightly safer construction. I’ve implemented it slightly differently so that it remains constexpr, and supports everything MakeSpan does (including passing temporaries, and non-const return when appropriate).

    In commit “Make uint256 Span-convertible by adding ::data()” (5d1f1d9)

    I think you mean a different commit?

    My concern is that providing this function might make it too easy to write code that looks like it is hashing vector<uint256> or Optional<uint256> or uint256* data, but actually hashing the outer pointer or container objects instead.

    I don’t think that would be a large concern. Those data types don’t have data() and size() methods, so no implicit conversion to Span can happen. You can of course explicitly construct one, but you’d need to use something like AsUnsignedChars(Span<std::vector<uint256>(&vec, 1)), which is hopefully obviously accessing the vector’s byte representation instead.

  21. in src/test/script_standard_tests.cpp:219 in b92907e0b8 outdated
    215@@ -216,7 +216,7 @@ BOOST_AUTO_TEST_CASE(script_standard_ExtractDestination)
    216     s << OP_0 << ToByteVector(pubkey.GetID());
    217     BOOST_CHECK(ExtractDestination(s, address));
    218     WitnessV0KeyHash keyhash;
    219-    CHash160().Write(pubkey).Finalize(keyhash.begin());
    220+    CHash160().Write(pubkey).Finalize(keyhash);
    


    ryanofsky commented at 10:04 pm on July 16, 2020:

    In commit “Make CHash256/CHash160 output to Span” (b92907e0b89210ecf25baa70702550f643d53525)

    This commit doesn’t seem to compile. Also not sure reason scriptHash.begin() is changing to scriptHash.data() above on line 102.


    sipa commented at 5:02 am on July 21, 2020:
    Fixed.
  22. ryanofsky approved
  23. ryanofsky commented at 10:12 pm on July 16, 2020: member
    Code review ACK 675d6fd044be08cd5dc7386b5574d845a74e9aae. I think intermediate commits might not compile, but comments below aren’t critical.
  24. laanwj commented at 2:49 pm on July 17, 2020: member
    ACK 675d6fd044be08cd5dc7386b5574d845a74e9aae
  25. MarcoFalke added the label Waiting for author on Jul 17, 2020
  26. sipa force-pushed on Jul 21, 2020
  27. sipa commented at 5:07 am on July 21, 2020: member
    Rebased, and addressed @ryanofsky’s comments.
  28. jonatack commented at 11:53 am on July 21, 2020: member
    Code review ACK 49fc016 per git range-diff caf1766 675d6fd 49fc016. Good improvements; the main change if I’m seeing things clearly is 4679ef6 As* Spans replaced by b0cf67f9 UChar* helper functions and the refactorings they enable, and moving commit “Make script/standard’s BaseHash Span-convertible” earlier in the commit order. Verified that each commit debug-builds cleanly.
  29. ryanofsky approved
  30. ryanofsky commented at 0:28 am on July 22, 2020: member
    Code review ACK 49fc016c78a476e687bc73adb5c32350c2e115e5. Since last review rebased added MakeUCharSpan function, rearranged commits to fix compile error, and dropped an unneeded test change
  31. sipa removed the label Waiting for author on Jul 22, 2020
  32. DrahtBot added the label Needs rebase on Jul 30, 2020
  33. scripted-diff: rename base_blob::data to m_data
    This is in preparation for exposing a ::data member function.
    
    -BEGIN VERIFY SCRIPT-
    sed -i "s/\([^.]\|other.\)data/\1m_data/g" src/uint256.h src/uint256.cpp
    -END VERIFY SCRIPT-
    131a2f0337
  34. Make uint256 Span-convertible by adding ::data() 567825049f
  35. Add MakeUCharSpan, to help constructing Span<[const] unsigned char>
    Based on a suggestion by Russell Yanofsky.
    e63dcc3a67
  36. Make script/standard's BaseHash Span-convertible 2a2182c387
  37. Make CHash256 and CHash160 consume Spans e549bf8a9a
  38. Make MurmurHash3 consume Spans 0ef97b1b10
  39. Make CHash256/CHash160 output to Span 02c4cc5c5d
  40. Make Hash[160] consume range-like objects 77c507358b
  41. sipa commented at 9:03 pm on July 30, 2020: member
    Rebased.
  42. sipa force-pushed on Jul 30, 2020
  43. DrahtBot removed the label Needs rebase on Jul 30, 2020
  44. laanwj commented at 2:54 pm on August 3, 2020: member
    re-ACK 77c507358bda9bd6c496f33e0f4418c0603bb08d
  45. jonatack commented at 3:14 pm on August 3, 2020: member
    Code review re-ACK 77c5073 per git range-diff 14ceddd 49fc016 77c5073
  46. laanwj merged this on Aug 3, 2020
  47. laanwj closed this on Aug 3, 2020

  48. sidhujag referenced this in commit 49ec455bbf on Aug 4, 2020
  49. Fabcien referenced this in commit 676c36c107 on Feb 4, 2021
  50. Fabcien referenced this in commit 6d6b676c0a on Feb 4, 2021
  51. Fabcien referenced this in commit bce6951264 on Feb 4, 2021
  52. Fabcien referenced this in commit c34f1d4c2f on Feb 4, 2021
  53. Fabcien referenced this in commit f494ab3669 on Feb 4, 2021
  54. Fabcien referenced this in commit dc26954562 on Feb 4, 2021
  55. Fabcien referenced this in commit 2417468301 on Feb 4, 2021
  56. Fabcien referenced this in commit d0c313ae91 on Feb 4, 2021
  57. kittywhiskers referenced this in commit 7f2c7ed82c on May 19, 2021
  58. kittywhiskers referenced this in commit f7e75e3352 on May 20, 2021
  59. kittywhiskers referenced this in commit 2dc3a1e161 on May 20, 2021
  60. kittywhiskers referenced this in commit 94b07c2bd9 on May 20, 2021
  61. kittywhiskers referenced this in commit 56f1b2d01c on May 20, 2021
  62. PastaPastaPasta referenced this in commit b76e7fec1f on May 21, 2021
  63. random-zebra referenced this in commit 56f10aaa51 on Jul 21, 2021
  64. DrahtBot locked this on Feb 15, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-18 15:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me