Serialization improvements step 3 (compression.h) #17957

pull sipa wants to merge 2 commits into bitcoin:master from sipa:202001_noncastserial_3 changing 6 files +52 −45
  1. sipa commented at 3:44 pm on January 18, 2020: member

    This is the next piece of the puzzle from #10785. It includes:

    • The FORMATTER_METHODS macro, similar to SERIALIZE_METHODS, for defining a formatter with a unified serialization/deserialization implementation.
    • Updating compression.h to consist of 3 formatters, rather than old-style wrappers (ScriptCompression, AmountCompression, TxOutCompression).
  2. Add FORMATTER_METHODS, similar to SERIALIZE_METHODS, but for formatters ca34c5cba5
  3. sipa force-pushed on Jan 18, 2020
  4. sipa renamed this:
    Serialization improvements step 3 (vectors & compression.h)
    Serialization improvements step 3 (compression.h)
    on Jan 18, 2020
  5. sipa commented at 3:47 pm on January 18, 2020: member
    Ping @ryanofsky, as you volunteered to review further PRs in this series. I’ve made an additional change compared to the old big PR, namely the introduction of FORMATTER_METHODS (and then rewriting SERIALIZE_METHODS in function of FORMATTER_METHODS, which follows pretty naturally).
  6. sipa requested review from ryanofsky on Jan 18, 2020
  7. sipa force-pushed on Jan 18, 2020
  8. DrahtBot added the label Tests on Jan 18, 2020
  9. DrahtBot added the label UTXO Db and Indexes on Jan 18, 2020
  10. practicalswift commented at 5:27 pm on January 18, 2020: contributor

    https://github.com/bitcoin/bitcoin/blob/fd691da3b032e439936672e084b7c49086ec9ebc/src/coins.h#L59-L65

    A signed integer overflow will take place on L62 in case of serialising a coin with a very large nHeight value.

    It wasn’t introduced in this PR, but since you’re making changes to the coin serialization/deserialization functions here it might be worth fixing in the same go :)

    Also the Unserialize function should probably be adjusted to not accept block_heights above (std::numeric_limits<uint32_t>::max() - 1) / 2.

    https://github.com/bitcoin/bitcoin/blob/fd691da3b032e439936672e084b7c49086ec9ebc/src/coins.h#L67-L74

  11. sipa commented at 5:37 pm on January 18, 2020: member
    @practicalswift How about we address that when coins.h itself is updated to the new serialization framework (here it’s just adjusting for changes in compression.h)?
  12. laanwj added this to the "Blockers" column in a project

  13. in src/undo.h:35 in fd691da3b0 outdated
    31@@ -32,7 +32,7 @@ class TxInUndoSerializer
    32             // Required to maintain compatibility with older undo format.
    33             ::Serialize(s, (unsigned char)0);
    34         }
    35-        ::Serialize(s, CTxOutCompressor(REF(txout->out)));
    36+        ::Serialize(s, Using<TxOutCompression>(REF(txout->out)));
    


    ryanofsky commented at 6:00 pm on January 21, 2020:

    In commit “Convert compression.h to new serialization framework” (fd691da3b032e439936672e084b7c49086ec9ebc)

    Can you drop uses of REF macro here and below?


    sipa commented at 4:30 am on January 22, 2020:
    Indeed, done.
  14. in src/serialize.h:246 in ca34c5cba5 outdated
    246+        Unser(s, *this);                                                            \
    247     }                                                                               \
    248-    template<typename Stream, typename Type, typename Operation>                    \
    249-    static inline void SerializationOps(Type& obj, Stream& s, Operation ser_action) \
    250-
    251+    FORMATTER_METHODS(cls, obj)
    


    ryanofsky commented at 6:10 pm on January 21, 2020:

    In commit “Add FORMATTER_METHODS, similar to SERIALIZE_METHODS, but for formatters” (ca34c5cba5fbb9b046b074a234f06ecf6ed5d610)

    It’s a little surprising to see SERIALIZE_METHODS depend on FORMATTER_METHODS, but I guess it saves some repetition, and it seems reasonable


    sipa commented at 4:33 am on January 22, 2020:

    I think it makes sense: every class gets the ability to function as a formatter. Both FORMATTER_METHODS and SERIALIZE_METHODS help do that; with the first it’s for an arbitrary type, with the latter it’s for objects of the class it’s being defined it. The latter then also hooks up the Serialize/Unserialize functions to the self-formatter.

    Do you suggest documenting this better, or have some other suggestion?


    ryanofsky commented at 3:03 pm on January 22, 2020:

    It’s a little surprising to see SERIALIZE_METHODS depend on FORMATTER_METHODS, but I guess it saves some repetition, and it seems reasonable

    I think it makes sense: every class gets the ability to function as a formatter. Both FORMATTER_METHODS and SERIALIZE_METHODS help do that; with the first it’s for an arbitrary type, with the latter it’s for objects of the class it’s being defined it. The latter then also hooks up the Serialize/Unserialize functions to the self-formatter.

    I think it’s fine. A formatter class with a formatter methods is a class that can serialize and deserialize objects in a particular format. A serializable type with serialize methods is a class whose instances can serialize and deserialize themselves. I wasn’t expecting serializeable objects to now be able to format other objects of the same type, and I couldn’t think of a use for that, but I think it’s harmless.

    Do you suggest documenting this better, or have some other suggestion?

    I guess the suggestion would be to keep the new FORMATTER_METHODS macro exactly like you have it here, but revert the changes to the SERIALIZE_METHODS macro, so it isn’t affected by this PR. But only if you want to have a stronger distinction between formatter classes and serializable types. I’m fine either way.

  15. ryanofsky approved
  16. ryanofsky commented at 6:11 pm on January 21, 2020: member
    Code review ACK fd691da3b032e439936672e084b7c49086ec9ebc
  17. Convert compression.h to new serialization framework 4de934b9b5
  18. sipa force-pushed on Jan 22, 2020
  19. ryanofsky approved
  20. ryanofsky commented at 3:23 pm on January 22, 2020: member
    Code review ACK 4de934b9b5b4be1bac8fe205f4ee9a79e772dc34. Only change since last review is removing REF usages
  21. laanwj commented at 3:45 pm on January 23, 2020: member
    code review ACK 4de934b9b5b4be1bac8fe205f4ee9a79e772dc34
  22. laanwj referenced this in commit c1607b5df4 on Jan 29, 2020
  23. laanwj merged this on Jan 29, 2020
  24. laanwj closed this on Jan 29, 2020

  25. laanwj removed this from the "Blockers" column in a project

  26. sidhujag referenced this in commit bd1262fde3 on Feb 1, 2020
  27. sidhujag referenced this in commit 502bae9ef1 on Nov 10, 2020
  28. Fabcien referenced this in commit dd133aede6 on Dec 8, 2020
  29. Fabcien referenced this in commit bfbd8ef56e on Dec 8, 2020
  30. kittywhiskers referenced this in commit 3e05e60f84 on Mar 10, 2021
  31. kittywhiskers referenced this in commit 6e129990b0 on Mar 16, 2021
  32. kittywhiskers referenced this in commit 0b834f5a30 on Mar 16, 2021
  33. PastaPastaPasta referenced this in commit bfe5971359 on Apr 18, 2021
  34. UdjinM6 referenced this in commit 20b71700dc on May 14, 2021
  35. kittywhiskers referenced this in commit 5472f8d167 on May 20, 2021
  36. furszy referenced this in commit 5c93f159bc on Jul 5, 2021
  37. random-zebra referenced this in commit b4751e10ce on Aug 11, 2021
  38. DrahtBot locked this on Feb 15, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-17 09:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me