Delete nonreduced fuzz inputs #263

1. 
   maflcko commented at 6:58 am on March 12, 2026: contributor

   As per the usual process to avoid wasted CI resources and timeouts when CI runs on large and presumed irrelevant inputs.

   Normally, deletion of non-reduced fuzz inputs should happen after feature-freeze on the master Bitcoin Core branch, but before branch-off, so that the latest release branch retains mostly valid fuzz inputs.

   Previous: #239

   To “reproduce”

   Install a fresh VM, as explained in the bash script’s doc, and run it:

   0apt update && apt install curl -y
   1curl -fLO https://raw.githubusercontent.com/bitcoin-core/qa-assets/b30853a993f2fdc2edb8608b4544164d05312428/delete_nonreduced_fuzz_inputs.sh
   2bash delete_nonreduced_fuzz_inputs.sh
   

   To “test”

   • Keep an eye on coverage stats, to ensure it doesn’t drop
   • Re-run the script, to ensure it is “reproducible” to some extent
   • Anything else you think is important to test or review

   CI

   CI should pass, except for a lint failure, which should light up on any changes like this pull request, which delete fuzz inputs.

2. Delete fuzz inputs bb5d5764fe
3. Reduced inputs for afl-cmin 7f41a31103
4. Reduced inputs for fuzzer a606c66831
5. Reduced inputs for fuzzer,address,undefined,integer a39e31a912
6. 
   maflcko commented at 2:32 pm on March 12, 2026: contributor

   Storage device usage (du -sh ./fuzz_corpora/)

   6.8G -> 2.7G

   Determinism

   • ~196k fuzz input files deleted:

   0git diff origin/main a39e31a912bb4d097e6af375ae15c629563bd8c9 --stat | tail -1
   1 196312 files changed, 592128 deletions(-)
   

   • Cross-diff with a second run of the script: 3k fuzz input files

   0git diff --no-renames --stat HEAD a39e31a912bb4d097e6af375ae15c629563bd8c9 | tail -1 
   1 3837 files changed, 438 insertions(+), 3295 deletions(-)
   

   Coverage

   main: https://drahtbot.space/host_reports/DrahtBot/reports/coverage_fuzz/monotree/b97abdcdf1396f2c/687c9922722cfd01/fuzz.coverage/index.html

   this pull: https://drahtbot.space/host_reports/DrahtBot/reports/coverage_fuzz/monotree/b97abdcdf1396f2c/a39e31a912bb4d09/fuzz.coverage/index.html

7. 
   maflcko commented at 2:34 pm on March 12, 2026: contributor

   Looks like a few lines of coverage went away. Possibly due to https://github.com/bitcoin/bitcoin/issues/29018 :(

   Still, I guess this should be good to go.

8. 
   sipa commented at 2:53 pm on March 12, 2026: contributor

   To see the number of files deleted per test:

     0$ git diff --raw --stat upstream/main | fgrep 'fuzz_corpora/' | fgrep '00000000000 D' | cut -d / -f 2 | sort | uniq -c | sort -g
     1     14 span
     2     17 uint256_deserialize
     3     20 uint160_deserialize
     4     21 blockheader_deserialize
     5     22 crypto_aes256
     6     23 inv_deserialize
     7     23 threadpool
     8     24 fee_rate_deserialize
     9     24 secp256k1_ec_seckey_import_export_der
    10     25 out_point_deserialize
    11     27 snapshotmetadata_deserialize
    12     29 flatfile
    13     34 protocol
    14     36 fee_rate
    15     39 crypto_common
    16     41 crypto_poly1305
    17     47 base64_encode_decode
    18     47 wallet_fees
    19     48 block_header
    20     48 kitchen_sink
    21     49 block_file_info_deserialize
    22     49 locale
    23     49 muhash
    24     51 script_deserialize
    25     52 flat_file_pos_deserialize
    26     53 base32_encode_decode
    27     53 bech32_roundtrip
    28     54 blocktransactionsrequest_deserialize
    29     54 float
    30     57 bloomfilter_deserialize
    31     59 random
    32     60 key_io
    33     60 parse_iso8601
    34     61 key_origin_info_deserialize
    35     61 tx_out
    36     62 diskblockindex_deserialize
    37     62 secp256k1_ecdsa_signature_parse_der_lax
    38     66 blocklocator_deserialize
    39     67 crypto_hkdf_hmac_sha256_l32
    40     69 http_request
    41     69 parse_hd_keypath
    42     69 pub_key_deserialize
    43     70 chain
    44     71 key
    45     71 tx_in_deserialize
    46     79 fees
    47     80 ellswift_roundtrip
    48     80 messageheader_deserialize
    49     81 block_filter_deserialize
    50     82 socks5
    51     88 bip324_ecdh
    52     88 merkle_block_deserialize
    53     90 script_parsing
    54     92 crypto_poly1305_split
    55     93 clusterlin_components
    56    100 partial_merkle_tree_deserialize
    57    102 feefrac
    58    103 clusterlin_chunking
    59    104 bech32_random_decode
    60    106 clusterlin_postlinearize
    61    106 tx_in
    62    107 checkqueue
    63    107 crypto_aes256cbc
    64    108 base58_encode_decode
    65    108 base58check_encode_decode
    66    108 feefrac_div_fallback
    67    108 rolling_bloom_filter
    68    109 chacha20_split_crypt
    69    110 clusterlin_postlinearize_moved_leaf
    70    112 difference_formatter
    71    118 feefrac_mul_div
    72    122 crypto_fschacha20
    73    124 script_descriptor_cache
    74    125 txoutcompressor_deserialize
    75    125 wallet_bdb_parser
    76    128 chacha20_split_keystream
    77    129 crypto_chacha20
    78    129 scriptnum_ops
    79    130 build_and_compare_feerate_diagram
    80    134 crypto_aeadchacha20poly1305
    81    135 clusterlin_fix_linearization
    82    135 num3072_mul
    83    135 overflow
    84    141 coin_grinder_is_optimal
    85    142 clusterlin_ancestor_finder
    86    142 crypto_fschacha20poly1305
    87    143 coins_deserialize
    88    149 bnb_finds_min_waste
    89    155 timeoffsets
    90    156 addr_info_deserialize
    91    157 crypto_diff_fuzz_chacha20
    92    165 pcp_request_port_map
    93    167 integer
    94    169 clusterlin_make_connected
    95    169 crypto
    96    172 clusterlin_linearization_chunking
    97    173 parse_numbers
    98    177 clusterlin_simple_finder
    99    178 addition_overflow
   100    190 natpmp_request_port_map
   101    192 clusterlin_depgraph_serialization
   102    195 num3072_inv
   103    199 netaddr_deserialize
   104    200 service_deserialize
   105    207 block_deserialize
   106    211 multiplication_overflow
   107    212 txundo_deserialize
   108    213 versionbits
   109    214 bip324_cipher_roundtrip
   110    215 cuckoocache
   111    228 netaddress
   112    232 message
   113    234 clusterlin_merge
   114    235 pow_transition
   115    243 blocktransactions_deserialize
   116    247 coinselection_srd
   117    249 net_permissions
   118    258 block_index_tree
   119    261 script_ops
   120    266 prefilled_transaction_deserialize
   121    275 golomb_rice
   122    275 p2p_transport_serialization
   123    276 netbase_dns_lookup
   124    279 p2p_transport_bidirectional
   125    281 blockmerkleroot
   126    284 blockfilter
   127    287 block_header_and_short_txids_deserialize
   128    288 parse_script
   129    289 prevector
   130    290 merkle
   131    293 clusterlin_simple_linearize
   132    306 address_deserialize
   133    310 hex
   134    319 pow
   135    337 coincontrol
   136    342 buffered_file
   137    347 asmap_direct
   138    354 decode_tx
   139    358 minisketch
   140    363 asmap
   141    363 p2p_transport_bidirectional_v1v2
   142    364 merkleblock
   143    367 blockundo_deserialize
   144    372 torcontrol
   145    385 policy_estimator_io
   146    394 autofile
   147    394 str_printf
   148    398 script_sigcache
   149    417 primitives_transaction
   150    417 txrequest
   151    420 vecdeque
   152    434 headers_sync_state
   153    436 clusterlin_depgraph_sim
   154    447 utxo_snapshot
   155    462 coinselection_bnb
   156    470 clusterlin_search_finder
   157    470 p2p_transport_bidirectional_v2
   158    490 coinselection_knapsack
   159    506 coinscache_sim
   160    514 script_interpreter
   161    537 txorphan
   162    554 block
   163    556 load_external_block_file
   164    574 sighash_cache
   165    589 node_eviction
   166    592 i2p
   167    595 clusterlin_postlinearize_tree
   168    605 block_index
   169    618 txorphan_protected
   170    636 system
   171    644 utxo_snapshot_invalid
   172    650 coin_grinder
   173    650 crypter
   174    652 mini_miner_selection
   175    659 p2p_headers_presync
   176    666 psbt_output_deserialize
   177    690 bloom_filter
   178    750 rbf
   179    811 pool_resource
   180    845 wallet_create_transaction
   181    851 script
   182    862 bitdeque
   183    872 mini_miner
   184    872 txorphanage_sim
   185    942 script_format
   186    951 utxo_total_supply
   187   1005 transaction
   188   1021 partially_downloaded_block
   189   1048 string
   190   1090 package_rbf
   191   1167 policy_estimator
   192   1169 addrman_serdeser
   193   1173 clusterlin_sfl
   194   1205 txdownloadman
   195   1266 validation_load_mempool
   196   1268 txdownloadman_impl
   197   1283 signet
   198   1304 clusterlin_linearize
   199   1392 data_stream_addr_man
   200   1568 net
   201   1610 psbt_input_deserialize
   202   1668 tx_pool_standard
   203   1677 local_address
   204   1745 p2p_handshake
   205   1807 eval_script
   206   1819 bitset
   207   1827 miniscript_script
   208   1835 banman
   209   2001 addrman
   210   2025 psbt_base64_decode
   211   2120 tx_package_eval
   212   2125 miniscript_smart
   213   2174 ephemeral_package_eval
   214   2234 miniscript_stable
   215   2319 miniscript_string
   216   2590 partially_signed_transaction_deserialize
   217   2899 signature_checker
   218   3045 coins_view_overlay
   219   3280 parse_univalue
   220   3461 descriptor_parse
   221   3501 script_flags
   222   4255 mocked_descriptor_parse
   223   4407 txgraph
   224   4431 connman
   225   4436 script_sign
   226   5191 process_message
   227   6096 process_messages
   228   6682 psbt
   229   7401 tx_pool
   230   8444 coins_view
   231   9599 coins_view_db
   232  10740 scriptpubkeyman
   233  13530 rpc
   

9. 
   sipa commented at 2:58 pm on March 12, 2026: contributor

   Would it be useful to keep relatively new files around, e.g. not delete any files that were added to qa-assets less than 2 years ago, because those might be relevant to old maintained branches even if they are not relevant to the latest release anymore?
10. 
    maflcko commented at 3:19 pm on March 12, 2026: contributor

    Would it be useful to keep relatively new files around, e.g. not delete any files that were added to qa-assets less than 2 years ago, because those might be relevant to old maintained branches even if they are not relevant to the latest release anymore?

    Yeah, I guess that is possible. Though, my preference would probably be to just create a branch of qa-assets (and use that in the previous release)

    Otherwise, it will be difficult or inefficient to delete any nonreduced fuzz inputs at all, because:

    • #232 was done less than a year ago, so we couldn’t delete any for yet another year.
    • The 10k deleted scriptpubkeyman fuzz inputs were likely due to https://github.com/bitcoin/bitcoin/pull/34230, so there is little value in keeping them at all, even for previous releases. (Same for the other descriptor targets)
    • For the other fuzz targets I expect the majority of deleted inputs are actually reduced ones, so most of them should still be valid for previous releases alike.
11. 
    maflcko commented at 4:21 pm on March 12, 2026: contributor

    A more precise cross-diff shows that the larger diffs happen in the targets that are more non-deterministic:

      0# ( cd fuzz_corpora/ && for f in ./* ; do  count=$(git diff --no-renames --stat HEAD a39e31a912bb4d097e6af375ae15c629563bd8c9 $f | wc -l) && echo $count $f; done ) | sort -h 
      10 ./addition_overflow
      20 ./asmap_direct
      30 ./base32_encode_decode
      40 ./base64_encode_decode
      50 ./bech32_roundtrip
      60 ./bip324_cipher_roundtrip
      70 ./bip324_ecdh
      80 ./block_file_info_deserialize
      90 ./block_header
     100 ./block_header_and_short_txids_deserialize
     110 ./blockheader_deserialize
     120 ./blocklocator_deserialize
     130 ./blocktransactions_deserialize
     140 ./blocktransactionsrequest_deserialize
     150 ./bloomfilter_deserialize
     160 ./chacha20_split_crypt
     170 ./chain
     180 ./clusterlin_chunking
     190 ./clusterlin_components
     200 ./clusterlin_depgraph_serialization
     210 ./clusterlin_depgraph_sim
     220 ./clusterlin_postlinearize
     230 ./clusterlin_postlinearize_tree
     240 ./crypto_aeadchacha20poly1305
     250 ./crypto_aes256
     260 ./crypto_common
     270 ./crypto_fschacha20
     280 ./crypto_fschacha20poly1305
     290 ./crypto_hkdf_hmac_sha256_l32
     300 ./crypto_poly1305
     310 ./crypto_poly1305_split
     320 ./difference_formatter
     330 ./diskblockindex_deserialize
     340 ./ellswift_roundtrip
     350 ./fee_rate
     360 ./fee_rate_deserialize
     370 ./feefrac
     380 ./feefrac_div_fallback
     390 ./feefrac_mul_div
     400 ./flat_file_pos_deserialize
     410 ./flatfile
     420 ./float
     430 ./inv_deserialize
     440 ./kitchen_sink
     450 ./messageheader_deserialize
     460 ./muhash
     470 ./multiplication_overflow
     480 ./netaddr_deserialize
     490 ./netbase_dns_lookup
     500 ./num3072_inv
     510 ./num3072_mul
     520 ./out_point_deserialize
     530 ./parse_iso8601
     540 ./policy_estimator_io
     550 ./pow_transition
     560 ./prefilled_transaction_deserialize
     570 ./pub_key_deserialize
     580 ./script_deserialize
     590 ./script_parsing
     600 ./secp256k1_ec_seckey_import_export_der
     610 ./snapshotmetadata_deserialize
     620 ./span
     630 ./tx_in
     640 ./tx_out
     650 ./txoutcompressor_deserialize
     660 ./uint160_deserialize
     670 ./uint256_deserialize
     682 ./base58check_encode_decode
     692 ./bnb_finds_min_waste
     702 ./checkqueue
     712 ./clusterlin_linearize
     722 ./clusterlin_simple_finder
     732 ./clusterlin_simple_linearize
     742 ./coin_grinder
     752 ./coins_deserialize
     762 ./key_io
     772 ./key_origin_info_deserialize
     782 ./locale
     792 ./merkle
     802 ./merkle_block_deserialize
     812 ./merkleblock
     822 ./miniscript_string
     832 ./natpmp_request_port_map
     842 ./overflow
     852 ./parse_hd_keypath
     862 ./partial_merkle_tree_deserialize
     872 ./policy_estimator
     882 ./script_descriptor_cache
     892 ./service_deserialize
     902 ./sighash_cache
     912 ./socks5
     922 ./tx_in_deserialize
     932 ./txundo_deserialize
     943 ./addr_info_deserialize
     953 ./block_filter_deserialize
     963 ./blockfilter
     973 ./build_and_compare_feerate_diagram
     983 ./chacha20_split_keystream
     993 ./clusterlin_sfl
    1003 ./coin_grinder_is_optimal
    1013 ./coinselection_bnb
    1023 ./integer
    1033 ./netaddress
    1043 ./protocol
    1053 ./script_interpreter
    1063 ./script_sigcache
    1073 ./timeoffsets
    1083 ./transaction
    1093 ./txorphan_protected
    1103 ./txrequest
    1113 ./wallet_bdb_parser
    1124 ./asmap
    1134 ./base58_encode_decode
    1144 ./blockundo_deserialize
    1154 ./coins_view
    1164 ./crypto_aes256cbc
    1174 ./fees
    1184 ./hex
    1194 ./http_request
    1204 ./message
    1214 ./random
    1224 ./secp256k1_ecdsa_signature_parse_der_lax
    1235 ./address_deserialize
    1245 ./block_deserialize
    1255 ./buffered_file
    1265 ./minisketch
    1275 ./net
    1285 ./pow
    1295 ./scriptnum_ops
    1306 ./autofile
    1316 ./bech32_random_decode
    1326 ./blockmerkleroot
    1336 ./clusterlin_postlinearize_moved_leaf
    1346 ./decode_tx
    1356 ./local_address
    1366 ./node_eviction
    1376 ./script_ops
    1386 ./string
    1396 ./txorphan
    1406 ./txorphanage_sim
    1417 ./coinselection_knapsack
    1427 ./crypter
    1437 ./crypto
    1447 ./cuckoocache
    1457 ./i2p
    1467 ./parse_script
    1477 ./prevector
    1488 ./bloom_filter
    1498 ./net_permissions
    1508 ./psbt_output_deserialize
    1518 ./script
    1529 ./block
    1539 ./script_flags
    1549 ./script_format
    1559 ./str_printf
    1569 ./torcontrol
    15710 ./coinscache_sim
    15810 ./miniscript_script
    15910 ./miniscript_stable
    16011 ./clusterlin_make_connected
    16111 ./coins_view_overlay
    16211 ./parse_numbers
    16311 ./pcp_request_port_map
    16411 ./signature_checker
    16511 ./system
    16612 ./crypto_diff_fuzz_chacha20
    16713 ./coinselection_srd
    16813 ./golomb_rice
    16913 ./key
    17013 ./primitives_transaction
    17114 ./bitset
    17214 ./miniscript_smart
    17314 ./p2p_transport_serialization
    17414 ./partially_signed_transaction_deserialize
    17516 ./rolling_bloom_filter
    17617 ./crypto_chacha20
    17719 ./coincontrol
    17819 ./p2p_transport_bidirectional_v1v2
    17919 ./psbt_base64_decode
    18020 ./bitdeque
    18120 ./threadpool
    18221 ./banman
    18321 ./block_index_tree
    18421 ./eval_script
    18521 ./psbt_input_deserialize
    18621 ./signet
    18723 ./data_stream_addr_man
    18824 ./addrman
    18925 ./p2p_handshake
    19026 ./versionbits
    19130 ./p2p_transport_bidirectional_v2
    19230 ./process_messages
    19331 ./txdownloadman_impl
    19432 ./descriptor_parse
    19532 ./mocked_descriptor_parse
    19633 ./package_rbf
    19734 ./addrman_serdeser
    19834 ./parse_univalue
    19936 ./psbt
    20037 ./txdownloadman
    20138 ./vecdeque
    20239 ./wallet_fees
    20341 ./rbf
    20443 ./validation_load_mempool
    20545 ./coins_view_db
    20648 ./mini_miner
    20748 ./partially_downloaded_block
    20848 ./script_sign
    20950 ./headers_sync_state
    21051 ./pool_resource
    21151 ./tx_pool
    21267 ./tx_pool_standard
    21373 ./load_external_block_file
    21478 ./tx_package_eval
    215100 ./txgraph
    216109 ./block_index
    217112 ./ephemeral_package_eval
    218122 ./p2p_transport_bidirectional
    219151 ./wallet_create_transaction
    220154 ./utxo_snapshot
    221156 ./scriptpubkeyman
    222157 ./p2p_headers_presync
    223165 ./connman
    224168 ./process_message
    225213 ./utxo_total_supply
    226240 ./rpc
    227324 ./utxo_snapshot_invalid
    

12. 
    sipa commented at 5:04 pm on March 12, 2026: contributor

    Yeah, I guess that is possible. Though, my preference would probably be to just create a branch of qa-assets (and use that in the previous release)

    Yeah, that’s a possibility too. Alternatively, the delete_nonreduced script could run merging with multiple releases/branches.

    It was just an idea I wanted to throw out there, but to be clear it’s not an objection to this PR or even a suggestion to address right now.

13. 
    maflcko commented at 5:18 pm on March 12, 2026: contributor

    Alternatively, the delete_nonreduced script could run merging with multiple releases/branches.

    That’d probably be ideal. Just creating a branch, or just merging against master will not supply new fuzz inputs for older releases, whereas merging against the previous branches will do that, while also deleting nonreduced inputs that are “unused” on all branches.

14. 
    maflcko commented at 3:23 pm on March 13, 2026: contributor

    rfm?
15. 
    dergoegge commented at 3:25 pm on March 13, 2026: member

    Are we concerned about the valgrind failure here?

    https://github.com/bitcoin-core/qa-assets/actions/runs/22990374949/job/66749878719?pr=263

     0Run wallet_create_transaction with args ['valgrind', '--quiet', '--error-exitcode=1', '/home/runner/work/_temp/build/bin/fuzz', PosixPath('/home/runner/work/_temp/ci/scratch/qa-assets/fuzz_corpora/wallet_create_transaction')]==6501== Conditional jump or move depends on uninitialised value(s)
     1==6501==    at 0x781E44: wallet::SelectCoins(wallet::CWallet const&, wallet::CoinsResult&, wallet::CoinsResult const&, long const&, wallet::CCoinControl const&, wallet::CoinSelectionParams const&) (spend.cpp:843)
     2==6501==    by 0x78767C: wallet::CreateTransactionInternal(wallet::CWallet&, std::vector<wallet::CRecipient, std::allocator<wallet::CRecipient> > const&, std::optional<unsigned int>, wallet::CCoinControl const&, bool) (spend.cpp:1212)
     3==6501==    by 0x78AA7A: wallet::CreateTransaction(wallet::CWallet&, std::vector<wallet::CRecipient, std::allocator<wallet::CRecipient> > const&, std::optional<unsigned int>, wallet::CCoinControl const&, bool) (spend.cpp:1457)
     4==6501==    by 0x581BE3: wallet::(anonymous namespace)::wallet_create_transaction_fuzz_target(std::span<unsigned char const, 18446744073709551615ul>) (spend.cpp:101)
     5==6501==    by 0x58B81F: operator() (std_function.h:591)
     6==6501==    by 0x58B81F: test_one_input(std::span<unsigned char const, 18446744073709551615ul>) (fuzz.cpp:88)
     7==6501==    by 0x2DF8CF: main (fuzz.cpp:264)
     8==6501== 
     9
    10==6501== Conditional jump or move depends on uninitialised value(s)
    11==6501==    at 0x781E44: wallet::SelectCoins(wallet::CWallet const&, wallet::CoinsResult&, wallet::CoinsResult const&, long const&, wallet::CCoinControl const&, wallet::CoinSelectionParams const&) (spend.cpp:843)
    12==6501==    by 0x78767C: wallet::CreateTransactionInternal(wallet::CWallet&, std::vector<wallet::CRecipient, std::allocator<wallet::CRecipient> > const&, std::optional<unsigned int>, wallet::CCoinControl const&, bool) (spend.cpp:1212)
    13==6501==    by 0x78AA7A: wallet::CreateTransaction(wallet::CWallet&, std::vector<wallet::CRecipient, std::allocator<wallet::CRecipient> > const&, std::optional<unsigned int>, wallet::CCoinControl const&, bool) (spend.cpp:1457)
    14==6501==    by 0x581BE3: wallet::(anonymous namespace)::wallet_create_transaction_fuzz_target(std::span<unsigned char const, 18446744073709551615ul>) (spend.cpp:101)
    15==6501==    by 0x58B81F: operator() (std_function.h:591)
    16==6501==    by 0x58B81F: test_one_input(std::span<unsigned char const, 18446744073709551615ul>) (fuzz.cpp:88)
    17==6501==    by 0x2DF8CF: main (fuzz.cpp:264)
    18==6501== 
    

16. 
    maflcko commented at 3:41 pm on March 13, 2026: contributor

    I think this is just a known upstream valgrind bug. A possible temporary workaround is in https://github.com/bitcoin/bitcoin/pull/34589
17. dergoegge merged this on Mar 13, 2026
18. dergoegge closed this on Mar 13, 2026

    ---

Contributors
maflcko sipa dergoegge