cmake: compact generated JSON C++ headers #35065

l0rinc commented at 3:06 PM on April 12, 2026: contributor

Problem

The current generated JSON headers are larger a noisier than they need to be.

For example, build/src/test/data/sighash.json.h contains lines like:

'\x5b','\x0a','\x09','\x5b','\x22','\x72','\x61','\x77',

That file is 1.43 MiB across 26,319 lines before this change.

Fix

This PR keeps the embedded data the same and only changes its C++ representation.

Across all generated JSON headers touched by this generator, total generated header size drops from 4.42 MiB to 2.55 MiB, and total line count drops from 81,653 to 20,824.

Considerations

_hex was already investigated in #31861. It was rejected because very large generated headers made that approach impractical for the compiler.

The generated *raw headers were originally part of this PR but since vscode forced the constexpr removal (causing compiler is out of heap space), it was reverted.

Before and after

JSON header sample

Before:

inline constexpr char detail_sighash_bytes[] {
'\x5b','\x0a','\x09','\x5b','\x22','\x72','\x61','\x77',
'\x5f','\x74','\x72','\x61','\x6e','\x73','\x61','\x63',

After:

inline constexpr char detail_sighash_bytes[] {
"\x5b\x0a\x09\x5b\x22\x72\x61\x77\x5f\x74\x72\x61\x6e\x73\x61\x63",

Generated header sizes

<details> <summary>Test JSON headers</summary>

Header	Before size	Before lines	After size	After lines	Size reduction	Line reduction
`build/src/test/data/base58_encode_decode.json.h`	15.04 KiB	274	8.74 KiB	75	41.9%	72.6%
`build/src/test/data/bip341_wallet_vectors.json.h`	204.10 KiB	3,671	117.37 KiB	924	42.5%	74.8%
`build/src/test/data/blockfilters.json.h`	124.19 KiB	2,236	71.44 KiB	565	42.5%	74.7%
`build/src/test/data/key_io_invalid.json.h`	36.37 KiB	658	20.98 KiB	171	42.3%	74.0%
`build/src/test/data/key_io_valid.json.h`	119.55 KiB	2,152	68.77 KiB	544	42.5%	74.7%
`build/src/test/data/script_tests.json.h`	1.49 MiB	27,480	878.82 KiB	6,876	42.5%	75.0%
`build/src/test/data/sighash.json.h`	1.43 MiB	26,319	841.67 KiB	6,586	42.5%	75.0%
`build/src/test/data/tx_invalid.json.h`	371.86 KiB	6,685	213.74 KiB	1,678	42.5%	74.9%
`build/src/test/data/tx_valid.json.h`	602.26 KiB	10,824	346.11 KiB	2,712	42.5%	74.9%
Total	4.36 MiB	80,299	2.51 MiB	20,131	42.5%	74.9%

</details>

<details> <summary>Univalue JSON headers</summary>

Header	Before size	Before lines	After size	After lines	Size reduction	Line reduction
`build/src/univalue/test/fail1.json.h`	700 B	17	480 B	11	31.4%	35.3%
`build/src/univalue/test/fail10.json.h`	619 B	16	433 B	10	30.0%	37.5%
`build/src/univalue/test/fail11.json.h`	412 B	12	314 B	9	23.8%	25.0%
`build/src/univalue/test/fail12.json.h`	426 B	12	322 B	9	24.4%	25.0%
`build/src/univalue/test/fail13.json.h`	512 B	14	373 B	10	27.1%	28.6%
`build/src/univalue/test/fail14.json.h`	426 B	12	322 B	9	24.4%	25.0%
`build/src/univalue/test/fail15.json.h`	448 B	13	337 B	10	24.8%	23.1%
`build/src/univalue/test/fail16.json.h`	263 B	10	230 B	9	12.5%	10.0%
`build/src/univalue/test/fail17.json.h`	448 B	13	337 B	10	24.8%	23.1%
`build/src/univalue/test/fail18.json.h`	7.42 KiB	138	4.34 KiB	41	41.5%	70.3%
`build/src/univalue/test/fail19.json.h`	362 B	11	286 B	9	21.0%	18.2%
`build/src/univalue/test/fail2.json.h`	323 B	11	262 B	9	18.9%	18.2%
`build/src/univalue/test/fail20.json.h`	369 B	11	290 B	9	21.4%	18.2%
`build/src/univalue/test/fail21.json.h`	434 B	13	327 B	10	24.7%	23.1%
`build/src/univalue/test/fail22.json.h`	441 B	13	333 B	10	24.5%	23.1%
`build/src/univalue/test/fail23.json.h`	348 B	11	278 B	9	20.1%	18.2%
`build/src/univalue/test/fail24.json.h`	320 B	11	262 B	9	18.1%	18.2%
`build/src/univalue/test/fail25.json.h`	412 B	12	314 B	9	23.8%	25.0%
`build/src/univalue/test/fail26.json.h`	476 B	13	353 B	10	25.8%	23.1%
`build/src/univalue/test/fail27.json.h`	305 B	10	254 B	9	16.7%	10.0%
`build/src/univalue/test/fail28.json.h`	312 B	10	258 B	9	17.3%	10.0%
`build/src/univalue/test/fail29.json.h`	234 B	9	214 B	9	8.5%	0.0%
`build/src/univalue/test/fail3.json.h`	465 B	13	345 B	10	25.8%	23.1%
`build/src/univalue/test/fail30.json.h`	241 B	9	218 B	9	9.5%	0.0%
`build/src/univalue/test/fail31.json.h`	255 B	9	226 B	9	11.4%	0.0%
`build/src/univalue/test/fail32.json.h`	491 B	14	361 B	10	26.5%	28.6%
`build/src/univalue/test/fail33.json.h`	291 B	10	246 B	9	15.5%	10.0%
`build/src/univalue/test/fail34.json.h`	277 B	10	238 B	9	14.1%	10.0%
`build/src/univalue/test/fail35.json.h`	405 B	12	310 B	9	23.5%	25.0%
`build/src/univalue/test/fail36.json.h`	255 B	9	226 B	9	11.4%	0.0%
`build/src/univalue/test/fail37.json.h`	305 B	10	254 B	9	16.7%	10.0%
`build/src/univalue/test/fail38.json.h`	284 B	10	242 B	9	14.8%	10.0%
`build/src/univalue/test/fail39.json.h`	284 B	10	242 B	9	14.8%	10.0%
`build/src/univalue/test/fail4.json.h`	316 B	11	258 B	9	18.4%	18.2%
`build/src/univalue/test/fail40.json.h`	255 B	9	226 B	9	11.4%	0.0%
`build/src/univalue/test/fail41.json.h`	255 B	9	226 B	9	11.4%	0.0%
`build/src/univalue/test/fail42.json.h`	469 B	13	349 B	10	25.6%	23.1%
`build/src/univalue/test/fail44.json.h`	555 B	15	397 B	10	28.5%	33.3%
`build/src/univalue/test/fail45.json.h`	7.35 KiB	137	4.30 KiB	41	41.5%	70.1%
`build/src/univalue/test/fail5.json.h`	373 B	12	290 B	9	22.3%	25.0%
`build/src/univalue/test/fail6.json.h`	387 B	12	298 B	9	23.0%	25.0%
`build/src/univalue/test/fail7.json.h`	387 B	12	298 B	9	23.0%	25.0%
`build/src/univalue/test/fail8.json.h`	316 B	11	258 B	9	18.4%	18.2%
`build/src/univalue/test/fail9.json.h`	358 B	11	282 B	9	21.2%	18.2%
`build/src/univalue/test/pass1.json.h`	10.22 KiB	189	5.95 KiB	54	41.8%	71.4%
`build/src/univalue/test/pass2.json.h`	7.43 KiB	138	4.34 KiB	41	41.5%	70.3%
`build/src/univalue/test/pass3.json.h`	1.23 KiB	27	798 B	13	36.5%	51.9%
`build/src/univalue/test/pass4.json.h`	7.33 KiB	137	4.29 KiB	41	41.5%	70.1%
`build/src/univalue/test/round1.json.h`	2.15 KiB	44	1.31 KiB	17	39.0%	61.4%
`build/src/univalue/test/round2.json.h`	341 B	11	274 B	9	19.6%	18.2%
`build/src/univalue/test/round3.json.h`	412 B	12	314 B	9	23.8%	25.0%
`build/src/univalue/test/round4.json.h`	220 B	9	206 B	9	6.4%	0.0%
`build/src/univalue/test/round5.json.h`	241 B	9	218 B	9	9.5%	0.0%
`build/src/univalue/test/round6.json.h`	248 B	9	222 B	9	10.5%	0.0%
`build/src/univalue/test/round7.json.h`	241 B	9	218 B	9	9.5%	0.0%
Total	60.22 KiB	1,354	38.80 KiB	693	35.6%	48.8%

</details>

<details><summary>Verifier</summary>

diff --git a/.gitattributes b/.gitattributes
index c9cf4a7d9c..d5db483a68 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -1 +1,4 @@
 src/clientversion.cpp export-subst
+
+# Keep generated test fixture bytes stable across platforms.
+src/**/test/**/*.json text eol=lf
diff --git a/src/test/CMakeLists.txt b/src/test/CMakeLists.txt
index 1365d6c147..7c259b13c4 100644
--- a/src/test/CMakeLists.txt
+++ b/src/test/CMakeLists.txt
@@ -48,6 +48,7 @@ add_executable(test_bitcoin
   flatfile_tests.cpp
   fs_tests.cpp
   getarg_tests.cpp
+  generated_data_tests.cpp
   hash_tests.cpp
   headers_sync_chainwork_tests.cpp
   httpserver_tests.cpp
diff --git a/src/test/generated_data_tests.cpp b/src/test/generated_data_tests.cpp
new file mode 100644
index 0000000000..977fa65ff9
--- /dev/null
+++ b/src/test/generated_data_tests.cpp
@@ -0,0 +1,39 @@
+// Copyright (c) 2026-present The Bitcoin Core developers
+// Distributed under the MIT software license, see the accompanying
+// file COPYING or https://opensource.org/license/mit/.
+
+#include <hash.h>
+#include <test/data/asmap.raw.h>
+#include <test/data/base58_encode_decode.json.h>
+#include <test/data/bip341_wallet_vectors.json.h>
+#include <test/data/blockfilters.json.h>
+#include <test/data/key_io_invalid.json.h>
+#include <test/data/key_io_valid.json.h>
+#include <test/data/script_tests.json.h>
+#include <test/data/sighash.json.h>
+#include <test/data/tx_invalid.json.h>
+#include <test/data/tx_valid.json.h>
+
+#include <boost/test/unit_test.hpp>
+
+BOOST_AUTO_TEST_SUITE(generated_data_tests)
+
+BOOST_AUTO_TEST_CASE(generated_json_hashes)
+{
+    BOOST_CHECK_EQUAL(Hash(json_tests::base58_encode_decode).ToString(), "3464e087c4a232deca45b36066cd792ea0d5c0e939d9bdaeae150932e8b4d1aa");
+    BOOST_CHECK_EQUAL(Hash(json_tests::bip341_wallet_vectors).ToString(), "33d50c7241ab5af1c5333bc27957fdc6cd50509885407fed75b2112a6e7dc57f");
+    BOOST_CHECK_EQUAL(Hash(json_tests::blockfilters).ToString(), "36c2521eb8c57bdca6d1321b24000189c13dedd4220488f704eb7098fbb3431f");
+    BOOST_CHECK_EQUAL(Hash(json_tests::key_io_invalid).ToString(), "0e1426518372b33529aaae5c1d258e0fffd0713d8b78fbd4022d2b1cf07af67a");
+    BOOST_CHECK_EQUAL(Hash(json_tests::key_io_valid).ToString(), "3be490f046325c99817bc12e784c5201fd8fc0d7b3704f5833ef6b28147ac083");
+    BOOST_CHECK_EQUAL(Hash(json_tests::script_tests).ToString(), "bf01a48b8c8ce43ca954faab23639214054806be7beb7193f981033084ae31d3");
+    BOOST_CHECK_EQUAL(Hash(json_tests::sighash).ToString(), "1c8ec220e5caf4c31b8d19b905ba89d887915dcb3bd94222428012346e495791");
+    BOOST_CHECK_EQUAL(Hash(json_tests::tx_invalid).ToString(), "9bca2afd7c06d030a2a208f711b21204beff48fc1458a41d6e1b1bdf9b7f0753");
+    BOOST_CHECK_EQUAL(Hash(json_tests::tx_valid).ToString(), "a57d330e62b365198b8f474e876ddcff5c0306535db8de18ed18d66639f7cdb4");
+}
+
+BOOST_AUTO_TEST_CASE(generated_raw_hashes)
+{
+    BOOST_CHECK_EQUAL(Hash(test::data::asmap).ToString(), "bafc9da308f45179443bd1d22325400ac9104f741522d003e3fac86700f68895");
+}
+
+BOOST_AUTO_TEST_SUITE_END()
diff --git a/src/univalue/CMakeLists.txt b/src/univalue/CMakeLists.txt
index 83cb4f1226..9c0798ba6b 100644
--- a/src/univalue/CMakeLists.txt
+++ b/src/univalue/CMakeLists.txt
@@ -79,6 +79,7 @@ if(BUILD_TESTS)
   target_link_libraries(unitester
     PRIVATE
       core_interface
+      bitcoin_crypto
       univalue
   )
   add_test(NAME univalue_test
diff --git a/src/univalue/test/unitester.cpp b/src/univalue/test/unitester.cpp
index e82ea74dec..356409c001 100644
--- a/src/univalue/test/unitester.cpp
+++ b/src/univalue/test/unitester.cpp
@@ -3,8 +3,9 @@
 // Distributed under the MIT software license, see the accompanying
 // file COPYING or https://opensource.org/licenses/mit-license.php.
 
+#include <hash.h>
+#include <uint256.h>
 #include <univalue.h>
-
 #include <univalue/test/fail1.json.h>
 #include <univalue/test/fail10.json.h>
 #include <univalue/test/fail11.json.h>
@@ -65,7 +66,6 @@
 #include <cassert>
 #include <string>
 #include <string_view>
-#include <tuple>
 
 static std::string rtrim(std::string s)
 {
@@ -97,64 +97,71 @@ static void runtest(std::string filename, const std::string& jdata)
     }
 }
 
-#define TEST_FILE(name) {#name, json_tests::name}
-inline constexpr std::array tests{std::to_array<std::tuple<std::string_view, std::string_view>>({
-    TEST_FILE(fail1),
-    TEST_FILE(fail10),
-    TEST_FILE(fail11),
-    TEST_FILE(fail12),
-    TEST_FILE(fail13),
-    TEST_FILE(fail14),
-    TEST_FILE(fail15),
-    TEST_FILE(fail16),
-    TEST_FILE(fail17),
-    TEST_FILE(fail18),
-    TEST_FILE(fail19),
-    TEST_FILE(fail2),
-    TEST_FILE(fail20),
-    TEST_FILE(fail21),
-    TEST_FILE(fail22),
-    TEST_FILE(fail23),
-    TEST_FILE(fail24),
-    TEST_FILE(fail25),
-    TEST_FILE(fail26),
-    TEST_FILE(fail27),
-    TEST_FILE(fail28),
-    TEST_FILE(fail29),
-    TEST_FILE(fail3),
-    TEST_FILE(fail30),
-    TEST_FILE(fail31),
-    TEST_FILE(fail32),
-    TEST_FILE(fail33),
-    TEST_FILE(fail34),
-    TEST_FILE(fail35),
-    TEST_FILE(fail36),
-    TEST_FILE(fail37),
-    TEST_FILE(fail38), // invalid unicode: only first half of surrogate pair
-    TEST_FILE(fail39), // invalid unicode: only second half of surrogate pair
-    TEST_FILE(fail4),  // extra comma
-    TEST_FILE(fail40), // invalid unicode: broken UTF-8
-    TEST_FILE(fail41), // invalid unicode: unfinished UTF-8
-    TEST_FILE(fail42), // valid json with garbage following a nul byte
-    TEST_FILE(fail44), // unterminated string
-    TEST_FILE(fail45), // nested beyond max depth
-    TEST_FILE(fail5),
-    TEST_FILE(fail6),
-    TEST_FILE(fail7),
-    TEST_FILE(fail8),
-    TEST_FILE(fail9), // extra comma
-    TEST_FILE(pass1),
-    TEST_FILE(pass2),
-    TEST_FILE(pass3),
-    TEST_FILE(pass4),
-    TEST_FILE(round1), // round-trip test
-    TEST_FILE(round2), // unicode
-    TEST_FILE(round3), // bare string
-    TEST_FILE(round4), // bare number
-    TEST_FILE(round5), // bare true
-    TEST_FILE(round6), // bare false
-    TEST_FILE(round7), // bare null
+struct JsonTestFile {
+    std::string_view m_name;
+    std::string_view m_json;
+    uint256 m_hash;
+};
+
+#define TEST_FILE(name, hash) {#name, json_tests::name, uint256{hash}}
+inline constexpr std::array tests{std::to_array<JsonTestFile>({
+    TEST_FILE(fail1, "e0f82e4680643632e245987c17f5c8508c309ba869032475791c1e535aa4f773"),
+    TEST_FILE(fail10, "a30cd52fd587eb8b8f35eba22cc17b31b3fb74c788522c5c029c418de105b634"),
+    TEST_FILE(fail11, "37be2924ae7502d3cfc92d413ba05895c46369b16a8ff18d954b86034bf1b135"),
+    TEST_FILE(fail12, "c5281bb0466417dace87dc703b9592f2e0b435aab487de6268da887b21aaf229"),
+    TEST_FILE(fail13, "d4137bf03139b250b947335f747bb2b5803bc868348a601272dcd8cfdb989229"),
+    TEST_FILE(fail14, "6da1d5568eed135ba6b17489567d48ecdb156ba0860603a50918e39060a432ec"),
+    TEST_FILE(fail15, "c7cadece87f8e825ece56a78f03ed6764524b3a2fc09dafce2cab2936fb4af70"),
+    TEST_FILE(fail16, "948f997b2a956a52f0845c11b5d63960f962fa8459c0c05cf6a73d55791ade74"),
+    TEST_FILE(fail17, "24d62847007c39070c34240caea5fc8c9bf850ed9f475d68823cb7dd1ab3d822"),
+    TEST_FILE(fail18, "18aa4a7a4de9e837772891a312fda21665b637f2f4992af91b88ce78eff88fdc"),
+    TEST_FILE(fail19, "9576320a1804c939892d0ebd83a072bf1ee1c39f742743ff4e9444986fd87339"),
+    TEST_FILE(fail2, "34afbbc8884020d1c9303b27da7123d6275b8f67be1ae870446d2726b494ef57"),
+    TEST_FILE(fail20, "8bd57a8e6029b156ff714bbc043bcca0590bb99814fc5ee6144a268353706f7f"),
+    TEST_FILE(fail21, "bde07e3d8c27d080a56be7d98551ae51b83937afe2169859940ab2f5fea38951"),
+    TEST_FILE(fail22, "4fe9183dd1213d003abaab4c19538dc32f79e1c76d6f53c4d71aa4ab55f445df"),
+    TEST_FILE(fail23, "ae0d6ae670c4eba95fa29cd6fe11e775d50df9fa2e96a834f0936871d49d0d98"),
+    TEST_FILE(fail24, "5af434ab4d1ba58072024ef9014991e5a449c271a506514183e897eef057a3ad"),
+    TEST_FILE(fail25, "0decf087bdabb155d878bdd3345fbf265abf42afedd57c9b239f74df5699d61b"),
+    TEST_FILE(fail26, "439cc28321005948cea51f46feab7163399d4cf5dd38f674ee6a9a566b0e73f5"),
+    TEST_FILE(fail27, "1ab59cf7db51e5e202483189e54dcf71f6af5b2cd98bef382599dd9e8559905d"),
+    TEST_FILE(fail28, "1b7b3639160510a03644589f08620a46ea0eddc39e3232dbb48e594cb3e64ebc"),
+    TEST_FILE(fail29, "36009c5adf306d19deb61b47fea1f5521ffc88e330b15562efbace4cff14e5e9"),
+    TEST_FILE(fail3, "e3932ec34900cbaf8deffe0c30518f093f12357040588aeaecd36fa9da532349"),
+    TEST_FILE(fail30, "f24a8d5907ea9c0b1a11d26af592da5ee519b961e893d1a87ca54106fc7bb682"),
+    TEST_FILE(fail31, "b5d1aed6b342b6d83f0bd742f3dc9c9b73ef471f1e0c250fa643b8ef114bc024"),
+    TEST_FILE(fail32, "05866b2ea527c3051800c686db79e4301d57f829b81e552da8344c9d8e22a002"),
+    TEST_FILE(fail33, "9e5bc56ba54d9fa79fb20fa3e593e7a7a302e27e87fc06a2ad91a688398722b2"),
+    TEST_FILE(fail34, "7595e58ece14e1d221134939812f9f28a29cfcd33cb769e12197b2c4a4dc71df"),
+    TEST_FILE(fail35, "ee34cb04a3b8b2b1a1527853543dc883d52bfaa8182fd3f6b674ff87ce3f84ef"),
+    TEST_FILE(fail36, "e1ba01f0e846b0c245734b57008c79bbdbc25d927de20fcf13de5063c8f49827"),
+    TEST_FILE(fail37, "8136afe807387834823c52397c37f844a4da49eb8dde7983ff61a5d885498fb2"),
+    TEST_FILE(fail38, "fd7d1a9323f9a7319296061f583141a9b66a124d359c574a07fcbc35c71a7f1b"), // invalid unicode: only first half of surrogate pair
+    TEST_FILE(fail39, "9c0a12e6bd2f9990c40adedc3fa8c66fb1f04c1077e8c29234ddb9844e2efea8"), // invalid unicode: only second half of surrogate pair
+    TEST_FILE(fail4, "c5f146fcf53ded562304975d59fceab85261d9248fb81a87972dad33d590e2be"),  // extra comma
+    TEST_FILE(fail40, "3536599d731d948e18ebdba2b8697a2ca91be61f136ed3006740a7873a9bafa6"), // invalid unicode: broken UTF-8
+    TEST_FILE(fail41, "b43765c94831853135bd555226eafa05d13a2d82185983a618ab2f0d377f99d3"), // invalid unicode: unfinished UTF-8
+    TEST_FILE(fail42, "cefee7600bf1f1e4be37d90eaba25ab4df92f6baafc10e561d27c616e912fdf4"), // valid json with garbage following a nul byte
+    TEST_FILE(fail44, "1c7565a64b1d8414817308f1a024af43cca08ba04b6b3a9b1c391e33c814c4d9"), // unterminated string
+    TEST_FILE(fail45, "d3aa46106bd6175e7e12f3b8bc86b822488cbd26176b4f69367ae29ce8f5a252"), // nested beyond max depth
+    TEST_FILE(fail5, "f95b0550c9493f23ec8594f23d0c5d98314c75051c789caba7c88e4a8c74469f"),
+    TEST_FILE(fail6, "1e42c4a1e8cc57eae7b2b7b44a887e0dd630d40a2e9f4603b413baf25d373506"),
+    TEST_FILE(fail7, "de38958db5a14fda0df7fc0730c39946dce00a5d3dbafabd3f1cbac68806dee7"),
+    TEST_FILE(fail8, "d5471441ddc4135ff10b5e13152382df57e7145662f6fd49b4a69195ac9018c9"),
+    TEST_FILE(fail9, "83dd66f3a61a734a334e66aa87e97a798bacdeb83e2bf9424a100931dda007c6"), // extra comma
+    TEST_FILE(pass1, "2e74b3d556161ae19d386800875ffd357cea8b99ab1b5c22c47fd2943d1764e2"),
+    TEST_FILE(pass2, "545bef2b9ee7dd9b691658c4fee84105e38dcf11e2d2072f33732c02491dc233"),
+    TEST_FILE(pass3, "ab3fbf8b8270184cd830cfa86fb35b4c3640aacf365a89b0c20b50ad4b221990"),
+    TEST_FILE(pass4, "688029b9c5a27684bbd4314c1e7cd2963b4111466625e3b0b77adc18f2788905"),
+    TEST_FILE(round1, "24e129f5bc2e9d00c85f828df276fcc2a131280e3850e5a31de3b703b2d8763c"), // round-trip test
+    TEST_FILE(round2, "c5e7e0d7ef2aef83cd4e1cc2a890aeb68f68a28126a5be9677dec7fe252a23f6"), // unicode
+    TEST_FILE(round3, "9ae2477d227d633e6522ae55aabf25385c5a20dfc145ba216e0f8d5af2641a4a"), // bare string
+    TEST_FILE(round4, "00c75f89035cd96c22f45f2c053e5a9df73abedaf267e14445bd807f3c6d87ae"), // bare number
+    TEST_FILE(round5, "6aa6234800c5418643ec288f3f26eaafe96f5a5b88d8e79b75ce170dc1562067"), // bare true
+    TEST_FILE(round6, "e62982a1c766caa39d735878c8401d60a8e0d40f55fe1b8e79f2a71434166627"), // bare false
+    TEST_FILE(round7, "34c2d1516cc623a5764ac8360184e13ec73ae886591f7264a6177312f1daed4e"), // bare null
 })};
+#undef TEST_FILE
 
 // Test \u handling
 void unescape_unicode_test()
@@ -188,7 +195,8 @@ void no_nul_test()
 
 int main(int argc, char* argv[])
 {
-    for (const auto& [file, json] : tests) {
+    for (const auto& [file, json, hash] : tests) {
+        assert(Hash(json) == hash);
         runtest(std::string{file}, std::string{json});
     }

</details>

DrahtBot added the label Build system on Apr 12, 2026

DrahtBot commented at 3:07 PM on April 12, 2026: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process. A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#32220 (cmake: Get rid of undocumented BITCOIN_GENBUILD_NO_GIT environment variable by hebasto)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

l0rinc force-pushed on Apr 12, 2026

DrahtBot added the label CI failed on Apr 12, 2026

l0rinc force-pushed on Apr 12, 2026

fanquake commented at 1:10 AM on April 13, 2026: member

Build FAILED.

"D:\a\bitcoin\bitcoin\build\ALL_BUILD.vcxproj" (default target) (1) ->
"D:\a\bitcoin\bitcoin\build\src\bench\bench_bitcoin.vcxproj" (default target) (4) ->
"D:\a\bitcoin\bitcoin\build\src\bitcoin_node.vcxproj" (default target) (10) ->
(ClCompile target) -> 
  D:\a\bitcoin\bitcoin\build\src\node\data\ip_asn.dat.h(45621,1): error C1060: compiler is out of heap space [D:\a\bitcoin\bitcoin\build\src\bitcoin_node.vcxproj]

    0 Warning(s)
    1 Error(s)

maflcko commented at 5:58 AM on April 13, 2026: member

Across all generated raw and JSON headers touched by these two generators, total generated header size drops from 43.17 MiB to 12.38 MiB (71.3% smaller), and total line count drops from 396,633 to 99,585 (74.9% fewer lines).

Not sure this is worth it. Storage is incredibly cheap, and the maximum benefit here would be 43 MiB. If this really was an issue for anyone, I am sure there are lower hanging fruits inside the build system to reduce the temporary storage footprint while compiling Bitcoin Core.

l0rinc commented at 7:21 AM on April 13, 2026: contributor

Not sure this is worth it

I'm still debugging the Windows failure, but why would it not be worth making the generated files smaller, if we can make it more "readable" and easier to parse at the same time?

maflcko commented at 7:26 AM on April 13, 2026: member

The files are inherently not readable, because they encode json as hex. I am not sure what audience you are targeting. If you speak about compilers, then you are making it worse, because you turn a compile-time property into a run-time one. I just don't see the point

l0rinc force-pushed on Apr 13, 2026

l0rinc commented at 7:40 AM on April 13, 2026: contributor

Thanks for the quick review, I have dropped the raw byte conversion, kept the JSON ones that don't have the mentioned constexpr to const conversion drawback - should pass CI as far as I can tell.

l0rinc renamed this:
~~cmake: compact generated raw and JSON C++ headers~~
cmake: compact generated JSON C++ headers
on Apr 13, 2026

l0rinc force-pushed on Apr 13, 2026

l0rinc marked this as ready for review on Apr 13, 2026

DrahtBot removed the label CI failed on Apr 13, 2026

hebasto commented at 3:03 PM on April 13, 2026: member

Problem

The current generated JSON headers are larger a noisier than they need to be.

It would be nice to explain why this is a problem.

cmake: compact generated JSON headers

The generated JSON headers currently emit one initializer per byte.
That makes the generated source much larger than it needs to be.

Emit concatenated byte string literals instead.
Build the exported `std::string_view` directly from the array pointer and size.

214d9f1664

l0rinc force-pushed on Apr 13, 2026

l0rinc commented at 6:13 PM on April 13, 2026: contributor

It would be nice to explain why this is a problem

This is just a refactor to avoid generated source bloat. It is not a big deal, but the generator does not get more complicated, we keep constexpr, and the result is easier to search, parse, compile, and restore from ccache.

I pushed a new version and dropped the verifier commit. I moved that context into the PR description instead so the PR stays focused on the actual change.

If this is still not seen as a clear improvement, I can just close it. Thanks for the feedback.

maflcko commented at 6:00 AM on April 14, 2026: member

the result is easier to search, parse, compile, and restore from ccache.

Without a benchmark, I'd doubt this is true, apart from rounding errors.

l0rinc commented at 6:18 AM on April 14, 2026: contributor

Thanks for the comments, closing

l0rinc closed this on Apr 14, 2026