Add ChaCha20 encryption option (XOR) #15512

pull jonasschnelli wants to merge 2 commits into bitcoin:master from jonasschnelli:2019/03/chacha changing 7 files +232 −16
  1. jonasschnelli commented at 4:12 pm on March 1, 2019: contributor

    The current ChaCha20 implementation does not support message encryption (it can only output the keystream which is sufficient for the RNG).

    This PR adds the actual XORing of the plaintext with the keystream in order to return the desired ciphertext.

    Required for v2 message transport protocol.

  2. laanwj added the label P2P on Mar 2, 2019
  3. laanwj added the label Utils/log/libs on Mar 2, 2019
  4. laanwj removed the label P2P on Mar 2, 2019
  5. jonasschnelli force-pushed on Mar 3, 2019
  6. DrahtBot commented at 10:36 am on March 5, 2019: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #15649 (Add ChaCha20Poly1305@Bitcoin AEAD by jonasschnelli)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  7. jonasschnelli added this to the "Blockers" column in a project

  8. in src/crypto/chacha20.cpp:106 in b05bb64dc6 outdated
    101@@ -100,6 +102,10 @@ void ChaCha20::Output(unsigned char* c, size_t bytes)
    102 
    103     for (;;) {
    104         if (bytes < 64) {
    105+            if (m != nullptr) {
    106+                for (i = 0;i < bytes;++i) tmp[i] = m[i];
    


    ryanofsky commented at 6:17 pm on March 22, 2019:
    A comment would be helpful here: “If m has fewer than 64 bytes available, copy m to tmp and read from tmp instead”

    ryanofsky commented at 6:18 pm on March 22, 2019:
    Just memcpy(tmp, m, bytes) might be simpler and easy to read than this for loop.

    jonasschnelli commented at 8:07 pm on March 25, 2019:
    I’m not to familiar with compiler optimisations and stuff but I just checked the reference implementation by DJB and some other C based ChaCha implementations and all of them seems to use the byte per byte assignment. I prefer to leave it as it is.

    jonasschnelli commented at 8:43 pm on March 25, 2019:
    Added.
  9. in src/crypto/chacha20.cpp:156 in b05bb64dc6 outdated
    151@@ -146,6 +152,25 @@ void ChaCha20::Output(unsigned char* c, size_t bytes)
    152         x14 += j14;
    153         x15 += j15;
    154 
    155+        if (m != nullptr) {
    156+            x0 = XOR(x0, ReadLE32(m + 0));
    


    ryanofsky commented at 6:20 pm on March 22, 2019:
    Not sure what the benefit of defining XOR macro is. Isn’t x0 ^= ReadLE32(m + 0) more readable and less error-prone than x0 = XOR(x0, ReadLE32(m + 0))?

    jonasschnelli commented at 8:44 pm on March 25, 2019:
    Thanks for the review! Your right. Changed.
  10. in src/crypto/chacha20.h:24 in b05bb64dc6 outdated
    19@@ -20,7 +20,9 @@ class ChaCha20
    20     void SetKey(const unsigned char* key, size_t keylen);
    21     void SetIV(uint64_t iv);
    22     void Seek(uint64_t pos);
    23-    void Output(unsigned char* output, size_t bytes);
    24+
    25+    // if <input> is provided, the ciphertext will be written to <output> otherwise the current keystream
    


    ryanofsky commented at 6:25 pm on March 22, 2019:
    This comment seems imply that output is not written if input is null. Maybe rephrase: “If <input> is non-null, the specified number of <bytes> from <input> will encrypted and written to <output>, otherwise just the keystream will be written to <output> (which may be useful for random number generation). Either way, the number of bytes written to <output> will be <bytes>.”

    jonasschnelli commented at 8:43 pm on March 25, 2019:
    Valid point. Fixed.
  11. in src/test/crypto_tests.cpp:201 in 4325da919a outdated
    197     rng.Seek(seek);
    198     std::vector<unsigned char> out = ParseHex(hexout);
    199     std::vector<unsigned char> outres;
    200     outres.resize(out.size());
    201-    rng.Output(outres.data(), outres.size());
    202+
    


    ryanofsky commented at 6:36 pm on March 22, 2019:
    Maybe assert(hex_message.empty() || m.size() == out.size())

    jonasschnelli commented at 8:43 pm on March 25, 2019:
    Added.
  12. in src/test/crypto_tests.cpp:203 in 4325da919a outdated
    199     std::vector<unsigned char> outres;
    200     outres.resize(out.size());
    201-    rng.Output(outres.data(), outres.size());
    202+
    203+    // perform the ChaCha20 round(s), if message is provided it will output the encrypted ciphertext otherwise the keystream
    204+    rng.Output(hex_message.empty() ? nullptr : m.data(), outres.data(), outres.size());
    


    ryanofsky commented at 6:50 pm on March 22, 2019:
    It seems like a useful additional test would be call rng.Output twice if hex_message is nonempty, once with input and once without, and confirm that xoring the two outputs yields the input again.

    jonasschnelli commented at 8:43 pm on March 25, 2019:
    Good point. Added.
  13. ryanofsky approved
  14. ryanofsky commented at 6:53 pm on March 22, 2019: member
    utACK 4325da919a042dfb4eed87b6d038dbb236959e5b. I don’t know very much about chacha20 or this method of encryption, but this seems to do what is described.
  15. jonasschnelli force-pushed on Mar 25, 2019
  16. laanwj referenced this in commit 208406038c on Mar 27, 2019
  17. in src/crypto/chacha20.cpp:74 in 9d3ea6a776 outdated
    70@@ -71,7 +71,7 @@ void ChaCha20::Seek(uint64_t pos)
    71     input[13] = pos >> 32;
    72 }
    73 
    74-void ChaCha20::Output(unsigned char* c, size_t bytes)
    75+void ChaCha20::Output(const unsigned char* m, unsigned char* c, size_t bytes)
    


    sipa commented at 5:48 pm on March 27, 2019:

    I think it would be better to duplicate the logic here and turn the stream cipher version into a separate function.

    As is, it introduces an unnecessary branch in the output version.


    jonasschnelli commented at 8:12 pm on March 28, 2019:
    Good point. Adapted the duplication approach.

    jnewbery commented at 3:58 pm on May 3, 2019:

    What’s the reasoning here? Performance?

    How does the version in the refrence implementation compare?

    0void ECRYPT_keystream_bytes(ECRYPT_ctx *x,u8 *stream,u32 bytes)
    1{
    2  u32 i;
    3  for (i = 0;i < bytes;++i) stream[i] = 0;
    4  ECRYPT_encrypt_bytes(x,stream,stream,bytes);
    5}
    

    ie setting Output(c, bytes) to call Crypt(c, c, bytes)?


    jonasschnelli commented at 8:45 pm on May 3, 2019:
    I had it in the initial version like @jnewbery just proposed. After @sipa recommended to separate it, I thought that a strict separation may be beneficial for verification and later optimizations. But maybe @sipa can clarify…
  18. jonasschnelli force-pushed on Mar 28, 2019
  19. in src/crypto/chacha20.cpp:261 in 7f2028f04c outdated
    256+        x12 += j12;
    257+        x13 += j13;
    258+        x14 += j14;
    259+        x15 += j15;
    260+
    261+        if (m != nullptr) {
    


    sipa commented at 9:28 pm on March 28, 2019:
    No need for this if.

    jonasschnelli commented at 8:58 am on March 29, 2019:
    Yes. Fixed.
  20. jonasschnelli force-pushed on Mar 29, 2019
  21. ryanofsky approved
  22. ryanofsky commented at 4:54 pm on March 29, 2019: member
    utACK 427b49dcdbfb25153ee4228dda49aa07db16d59c. Changes since last review: splitting Output/Crypt functions, dropping comment describing function arguments, adding comment describing m/tmp copy, dropping XOR macro, and making suggested test changes.
  23. in src/crypto/chacha20.h:24 in 427b49dcdb outdated
    19@@ -20,7 +20,9 @@ class ChaCha20
    20     void SetKey(const unsigned char* key, size_t keylen);
    21     void SetIV(uint64_t iv);
    22     void Seek(uint64_t pos);
    23-    void Output(unsigned char* output, size_t bytes);
    24+
    


    jnewbery commented at 3:59 pm on May 3, 2019:
    nit: why the extra newline?
  24. in src/crypto/chacha20.h:31 in 427b49dcdb outdated
    19@@ -20,7 +20,9 @@ class ChaCha20
    20     void SetKey(const unsigned char* key, size_t keylen);
    21     void SetIV(uint64_t iv);
    22     void Seek(uint64_t pos);
    23-    void Output(unsigned char* output, size_t bytes);
    24+
    25+    void Output(unsigned char* c, size_t bytes);
    26+    void Crypt(const unsigned char* input, unsigned char* output, size_t bytes);
    


    jnewbery commented at 4:02 pm on May 3, 2019:

    Now that this class can encrypt plaintext, the class comment should be updated from “A PRNG class for ChaCha20.”

    Really, I’d like to see more commenting for this class in general (ie note that SetIV sets the nonce, Seek sets the counter, that the counter is 64 bytes and the nonce is 64 bytes as in the original ChaCha20 spec instead of 32 byte counter/96 byte nonce as in the IETF spec).

    (there are a couple of good resources on the different counter/nonce sizes here: https://libsodium.gitbook.io/doc/advanced/stream_ciphers/chacha20 and here: http://loup-vaillant.fr/tutorials/chacha20-design)


    jonasschnelli commented at 6:32 pm on May 3, 2019:

    The IETF has a slightly modified recommendation AFAIK. We use Bernstein-Vanilly 64bit nonce and 64bit block counter.

    • Changed the class comment
    • Added some function comments
  25. in src/crypto/chacha20.h:24 in 427b49dcdb outdated
    19@@ -20,7 +20,9 @@ class ChaCha20
    20     void SetKey(const unsigned char* key, size_t keylen);
    21     void SetIV(uint64_t iv);
    22     void Seek(uint64_t pos);
    23-    void Output(unsigned char* output, size_t bytes);
    24+
    25+    void Output(unsigned char* c, size_t bytes);
    


    jnewbery commented at 4:06 pm on May 3, 2019:
    nit: This function should be renamed Keystream now that the class is not just used for PRNG

    jonasschnelli commented at 6:27 pm on May 3, 2019:
    Renamed to Keystream
  26. in src/test/crypto_tests.cpp:209 in 427b49dcdb outdated
    206+        rng.Crypt(m.data(), outres.data(), outres.size());
    207+    } else {
    208+        rng.Output(outres.data(), outres.size());
    209+    }
    210     BOOST_CHECK(out == outres);
    211+    if (!hex_message.empty()) {
    


    jnewbery commented at 4:08 pm on May 3, 2019:
    This test seems redundant to me. How could it ever fail if the test above succeeds?

    ryanofsky commented at 5:03 pm on May 3, 2019:

    re: #15512 (review)

    Not sure exactly what you want to get rid of, but if you’re talking about the out == outres check below, I suggested it and think it’d be better to keep, because it means if the test passes you can be reassured that Crypt() and Output() results are consistent without manually verifying 4c616 ^ 6e2e3 == 224f5… in the test cases.

  27. jnewbery commented at 4:17 pm on May 3, 2019: member

    Implementation and tests look good and match the reference implementation here: https://cr.yp.to/streamciphers/timings/estreambench/submissions/salsa20/chacha8/merged/chacha.c and test vector here: https://tools.ietf.org/html/rfc7539#section-2.4.2

    I have a few minor comments inline, and would be curious about the answer to: #15512 (review)

  28. jonasschnelli force-pushed on May 3, 2019
  29. Add ChaCha20 encryption option (XOR) 2bc2b8b49a
  30. jonasschnelli force-pushed on May 3, 2019
  31. in src/bench/chacha20.cpp:25 in 66d682bfc3 outdated
    20+    ctx.SetIV(0);
    21+    ctx.Seek(0);
    22+    std::vector<uint8_t> in(buffersize,0);
    23+    std::vector<uint8_t> out(buffersize,0);
    24+    while (state.KeepRunning())
    25+        ctx.Crypt(in.data(), out.data(), in.size());
    


    jnewbery commented at 7:17 pm on May 3, 2019:
    micronit: same line or braces for while code block please! Same for while block in HASH() below.

    jonasschnelli commented at 8:53 pm on May 3, 2019:
    Fixed
  32. in src/bench/chacha20.cpp:28 in 66d682bfc3 outdated
    23+    std::vector<uint8_t> out(buffersize,0);
    24+    while (state.KeepRunning())
    25+        ctx.Crypt(in.data(), out.data(), in.size());
    26+}
    27+
    28+static void HASH(benchmark::State& state, size_t buffersize)
    


    jnewbery commented at 7:18 pm on May 3, 2019:
    Is a reason you’ve added these CHash256 benchmarks to chacha20.cpp?

    jonasschnelli commented at 8:43 pm on May 3, 2019:
    I added if for comparison and the impact on the networking… though I think it belong to an extra commit in #15649 Will remove it from here.
  33. jnewbery commented at 7:20 pm on May 3, 2019: member

    Looks great. Thanks for adding the comments to chacha20.h!

    Just a couple of nits in the bench file. I’m still curious about the code duplication between Keystream() and Crypt().

  34. Add ChaCha20 bench 2dfe275171
  35. jonasschnelli force-pushed on May 3, 2019
  36. jonasschnelli force-pushed on May 3, 2019
  37. jnewbery commented at 9:01 pm on May 3, 2019: member

    Looks good. utACK 2dfe2751713c814aea53b5a7563eb74ad1baea00.

    In general, it’s better to have less code duplication, so I’d like to hear from @sipa his reasoning for #15512 (review)

  38. sipa commented at 9:09 pm on May 3, 2019: member
    @jnewbery I just thought that it’d be preferable not to burden the RNG code with branches that are only used for encryption. In cryptographic code like this, I don’t care too much about duplication, as it isn’t code that subject to many possible future changes.
  39. jnewbery commented at 2:36 pm on May 6, 2019: member

    ok, my aesthetic preference is for less duplication, either by having a single Keystream/Crypt function, or by calling Keystream() from Crypt() with a zero’ed message. As sipa points out though, this code is unlikely to be modified much in future, so it’s not a big deal either way.

    utACK 2dfe2751713c814aea53b5a7563eb74ad1baea00

  40. ryanofsky approved
  41. ryanofsky commented at 8:59 pm on May 7, 2019: member

    utACK 2dfe2751713c814aea53b5a7563eb74ad1baea00. Changes since last review are just renaming the Crypt method, adding comments, and simplifying the benchmark.

    I just thought that it’d be preferable not to burden the RNG code with branches that are only used for encryption.

    I think the duplication is fine, but it’s unclear if the concern with branches is about readability or performance. If it’s performance, you could give the previous Output() function an additional template<bool use_input> template argument, and have Crypt() and Keystream() both call it inlined.

  42. sipa commented at 0:33 am on May 10, 2019: member
    utACK 2dfe2751713c814aea53b5a7563eb74ad1baea00
  43. jonasschnelli merged this on May 10, 2019
  44. jonasschnelli closed this on May 10, 2019

  45. jonasschnelli referenced this in commit 695141bf7a on May 10, 2019
  46. jonasschnelli removed this from the "Blockers" column in a project

  47. sidhujag referenced this in commit 49cf9b1d60 on May 10, 2019
  48. PastaPastaPasta referenced this in commit a9ecbcba62 on Jun 19, 2019
  49. PastaPastaPasta referenced this in commit 0766c55cb7 on Jun 19, 2019
  50. in src/crypto/chacha20.cpp:300 in 2dfe275171
    295+        WriteLE32(c + 56, x14);
    296+        WriteLE32(c + 60, x15);
    297+
    298+        if (bytes <= 64) {
    299+            if (bytes < 64) {
    300+                for (i = 0;i < bytes;++i) ctarget[i] = c[i];
    


    Kixunil commented at 1:01 pm on July 2, 2019:
    Isn’t this no-op? When bytes < 64 line 215 get’s executed which makes ctarget and c equal pointers, therefore it’s copying a value to itself.

    jonasschnelli commented at 9:39 am on July 3, 2019:
    I don’t think so. If we do less then 64 bytes, we execute the round on tmp (because we do 64bytes always, see line 216, c points to tmp if <64 bytes). So in the case of <64 bytes, c points to the temporary 64byte buffer and ctarget points to the function provided c argument.

    Kixunil commented at 11:05 am on July 4, 2019:
    Ah, yes, missed that. Thanks and sorry for bothering.
  51. in src/crypto/chacha20.h:21 in 2dfe275171
    17@@ -17,10 +18,17 @@ class ChaCha20
    18 public:
    19     ChaCha20();
    20     ChaCha20(const unsigned char* key, size_t keylen);
    21-    void SetKey(const unsigned char* key, size_t keylen);
    22-    void SetIV(uint64_t iv);
    23-    void Seek(uint64_t pos);
    24-    void Output(unsigned char* output, size_t bytes);
    25+    void SetKey(const unsigned char* key, size_t keylen); //!< set key with flexible keylength; 256bit recommended */
    


    Kixunil commented at 1:04 pm on July 2, 2019:
    I like the renaming. Would be nice if the comments included information about pointers (e.g. “key must be non-null, pointer isn’t stored”).

    jonasschnelli commented at 9:39 am on July 3, 2019:
    Yes. That would have been useful. Feel free to PR.
  52. laanwj referenced this in commit 28d1353f48 on Jul 11, 2019
  53. sidhujag referenced this in commit aae6de0fc9 on Jul 11, 2019
  54. PastaPastaPasta referenced this in commit ca860ce337 on Jul 18, 2019
  55. PastaPastaPasta referenced this in commit 19879c544f on Jul 18, 2019
  56. PastaPastaPasta referenced this in commit bf737f4b91 on Jul 18, 2019
  57. PastaPastaPasta referenced this in commit 1bf6613178 on Jul 18, 2019
  58. PastaPastaPasta referenced this in commit 21ace66291 on Jul 23, 2019
  59. PastaPastaPasta referenced this in commit 4493671b87 on Jul 23, 2019
  60. PastaPastaPasta referenced this in commit 0840ce3a92 on Jul 23, 2019
  61. PastaPastaPasta referenced this in commit 28ed243821 on Aug 6, 2019
  62. PastaPastaPasta referenced this in commit e5f34cab48 on Aug 6, 2019
  63. PastaPastaPasta referenced this in commit 8214c765d4 on Aug 6, 2019
  64. barrystyle referenced this in commit 9a67bc54bc on Jan 22, 2020
  65. barrystyle referenced this in commit 1ffc186995 on Jan 22, 2020
  66. barrystyle referenced this in commit 7ebc414298 on Jan 22, 2020
  67. jasonbcox referenced this in commit 8b2c2a2813 on Sep 21, 2020
  68. random-zebra referenced this in commit 93f43f0f81 on Apr 14, 2021
  69. dhruv added this to the "Done" column in a project

  70. MarcoFalke locked this on Dec 16, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-06-29 16:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me