utxo_total_supply extremely slow #158

issue murchandamus openend this issue on November 7, 2023
  1. murchandamus commented at 3:01 pm on November 7, 2023: contributor

    While I had noticed utxo_total_supply being extremely slow on a merge before, my nightly fuzzer has been on the utxo_total_supply target for 6h51m at this point, even though the limit on the fuzzer is 3600 seconds. It seems to me that either it is stuck in an endless loop, or just extremely slow.

    image

    My […]/qa-assets-active-fuzzing/fuzz_seed_corpus/utxo_total_supply directory has 3250 seeds.

    It looks to me like it hasn’t even loaded my corpus yet: image

    Here are two of the worst offenders:

    slow-unit-ec3af16aeb3b967f9b63edc7266a71f52825c616

    0echo "egV+iYmJCy56C0HRX/7+cP7+/v////////7+/v7+/v7+/v7+/v78/v7+/v7+/v7+/r7+/v5cXFxcXFxcXFwpKSkpKSkpKf7+/v7+/v7+/v6+/v4pKSkpKSkpKSkpXClcXP7+JiAOWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZsDQ/J55ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWbA1PyeewVkoWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVkODg4OWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZsDQ/J55ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWbA1PyeewVkoWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVkODg4OWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlQWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVmwND8nnllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZsDU/J57BWShZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZDg4ODllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVmwND8nnllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZsDU/J57BWShZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVBZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWf7+/v7+/v7+C0HRX/7+/v7+/v5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWbA0PyeeWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZsDQ/J55ZWVlZWVn+/v7+/v6+WVlZWVlZWVlZWVlZWVlZWVlZWVlZ/v7+/v741y4LQVlZWVlZWVlZWVlZsDQ/J57BWShZWVkODg7RDg==" | base64 -d > slow-unit-ec3af16aeb3b967f9b63edc7266a71f52825c616

    /slow-unit-9c08c09c1d630fade4f699fdd1eeaa53325dcce3

    0echo "JiAOWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZDg4ODllZWVlZWVlZWVlZWVlZWVlZWVlZWVkZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWTJZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVkODg4OWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlTWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZDg4ODllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVkgDllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZGVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVkyWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZDg4ODllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVkODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVxcXFxcXFxcXFxcXFxcXFxcXFwIvLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8XFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcvLy8t7y8vLy8XAj19Ts7Ox0mJiYmOzs7JiYmJiYmJiYmJiYmJiYmJiYmJiYmXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXAi8vLy8vLy8vLy8vLy8vLy8vLy8vLxZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVmwOD8nnllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZDg4OWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWbA4PyeeWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVmwNT8nnsFZKFlZWQ4ODg4=" | base64 -d > slow-repro-9c08c09c
  2. maflcko commented at 3:13 pm on November 7, 2023: contributor
    What is tail ./fuzz-*.log ?
  3. murchandamus commented at 3:20 pm on November 7, 2023: contributor

    Sorry, was working on adding that. See OP, here’s fuzz-0.log:

     0INFO: Running with entropic power schedule (0xFF, 100).
 1INFO: Seed: 3732496129
 2INFO: Loaded 1 modules   (360765 inline 8-bit counters): 360765 [0x561b6de24550, 0x561b6de7c68d), 
 3INFO: Loaded 1 PC tables (360765 PCs): 360765 [0x561b6de7c690,0x561b6e3fda60), 
 4INFO:     3247 files found in ../qa-assets-active-fuzzing/fuzz_seed_corpus/utxo_total_supply
 5INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
 6INFO: seed corpus: files: 3247 min: 1b max: 4194303b total: 11985543b rss: 145Mb
 7[#16](/bitcoin-core-qa-assets/16/) pulse  cov: 12694 ft: 12725 corp: 5/5b exec/s: 8 rss: 203Mb
 8[#32](/bitcoin-core-qa-assets/32/) pulse  cov: 12694 ft: 12764 corp: 8/8b exec/s: 8 rss: 261Mb
 9[#64](/bitcoin-core-qa-assets/64/) pulse  cov: 12698 ft: 12822 corp: 14/18b exec/s: 8 rss: 314Mb
10[#128](/bitcoin-core-qa-assets/128/)    pulse  cov: 12859 ft: 17302 corp: 28/57b exec/s: 7 rss: 316Mb
11[#256](/bitcoin-core-qa-assets/256/)    pulse  cov: 13075 ft: 21971 corp: 61/179b exec/s: 6 rss: 316Mb
12[#512](/bitcoin-core-qa-assets/512/)    pulse  cov: 13323 ft: 30130 corp: 209/1145b exec/s: 5 rss: 316Mb
13[#1024](/bitcoin-core-qa-assets/1024/)   pulse  cov: 13360 ft: 35448 corp: 466/3858b exec/s: 3 rss: 316Mb
14[#2048](/bitcoin-core-qa-assets/2048/)   pulse  cov: 13450 ft: 53069 corp: 1156/25Kb exec/s: 1 rss: 316Mb
15Slowest unit: 10 s:
16artifact_prefix='./'; Test unit written to ./slow-unit-e1eb74194ce0614d81ac490af1a6583a66d1d2c4
17Base64: EQj19Ts7OyYmJiYmOzs7JiYmJiYmJiYmJiYmJiYmJiYmJiYmXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXAi8vLy8eby8vLy8vLy8vLy8YmxvY2u8vLy8vLy8vLy8vLy8vLy8vLy8vLxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFy8vLy8vLy8vLxcXFxcDlwJ
18Slowest unit: 12 s:
19artifact_prefix='./'; Test unit written to ./slow-unit-cb6bebf823aa3326525d2cc1aa6cbf58b487970d
20Base64: Jrm5uSa5VlZWVlZWVlZWVlZWVlYmJiYmJiwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsJiYB/yYmJoBWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlYmJiYmJiYmJiYmJiYmJiYmJFsmQUEl
21Slowest unit: 14 s:
22artifact_prefix='./'; Test unit written to ./slow-unit-b625b9409852885d51fdba131cf90164fc8178d1
23Base64: CwcKxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFoaGhoaGhoaGhoaGhoaGhoaGhoaGhocXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcWhoaGhoaGhoaGhoaGhoaGhoaGhoaGhocXFxSOhxcXFIwoa
24Slowest unit: 16 s:
25artifact_prefix='./'; Test unit written to ./slow-unit-a9e98ee3db1eaac45162745e9e950c48097250e0
26Base64: BUOPj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+PjwVDj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+P
27Slowest unit: 18 s:
28artifact_prefix='./'; Test unit written to ./slow-unit-802520d1fc4cc129532de6b468e78d9e59160aaa
29Slowest unit: 20 s:
30artifact_prefix='./'; Test unit written to ./slow-unit-e20f483cbf3d4b7449bac7dda8c5a46121e9c139
31Slowest unit: 23 s:
32artifact_prefix='./'; Test unit written to ./slow-unit-1ef5e38d73fdf03b9c54ca5634a10b5a16fe91e2
33Slowest unit: 26 s:
34artifact_prefix='./'; Test unit written to ./slow-unit-2c01e1724129117881e5b8860ec15c6cc0ac852c
35Slowest unit: 29 s:
36artifact_prefix='./'; Test unit written to ./slow-unit-346622c0f7de92ac1969ae91f8b030d0dc2ddf0a
37Slowest unit: 32 s:
38artifact_prefix='./'; Test unit written to ./slow-unit-993881544bb66c65ae6a206250eacbb655511ad7
39Slowest unit: 36 s:
40artifact_prefix='./'; Test unit written to ./slow-unit-6901a6b6a7915ed339cc6e5d0c7f93045ed7da2f
41Slowest unit: 40 s:
42artifact_prefix='./'; Test unit written to ./slow-unit-49b3b886c91ccb51bf356eef02d51dda4af2ee1a
43Slowest unit: 46 s:
44artifact_prefix='./'; Test unit written to ./slow-unit-9ab2a6b61a346db624def4553bf9841a8aeb2712
45Slowest unit: 61 s:
46artifact_prefix='./'; Test unit written to ./slow-unit-75d0af5eeadc76bd8866cec81a5f60a942f904e7
47Slowest unit: 75 s:
48artifact_prefix='./'; Test unit written to ./slow-unit-973e0cccb7e5db0a29874b2122691a26de0558f6
49Slowest unit: 87 s:
50artifact_prefix='./'; Test unit written to ./slow-unit-bc0fe6f6e4a65c9da5be25be7d5536bad6f8e24c
51Slowest unit: 113 s:
52artifact_prefix='./'; Test unit written to ./slow-unit-e92f8cda91b50f98d77dcb7e56b3b41fceb5e78b
53Slowest unit: 127 s:
54artifact_prefix='./'; Test unit written to ./slow-unit-59a4542a4657116270ab3457d83c217c1996d7c8
55Slowest unit: 140 s:
56artifact_prefix='./'; Test unit written to ./slow-unit-82b80732dc85e5c11f5a8255b82d31f3f13c9244
57Slowest unit: 156 s:
58artifact_prefix='./'; Test unit written to ./slow-unit-280ddb2c92b53aa298cb88d11c77ccd22e1de7ab
59Slowest unit: 182 s:
60artifact_prefix='./'; Test unit written to ./slow-unit-9c08c09c1d630fade4f699fdd1eeaa53325dcce3
61Slowest unit: 211 s:
62artifact_prefix='./'; Test unit written to ./slow-unit-ec3af16aeb3b967f9b63edc7266a71f52825c616
  4. maflcko commented at 3:54 pm on November 7, 2023: contributor

    Ah, so I guess this is expected, because it is loading all fuzz inputs. (The same would happen on merge or on -runs=1).

    This should be fixed the next time you pull from current Bitcoin Core master.

    Leave a comment here, if not.

  6. murchandamus commented at 5:55 pm on November 7, 2023: contributor
    I see, I’ll update to the latest upstream/master and try fuzzing on utxo_total_supply a bit
  7. murchandamus commented at 8:25 pm on November 7, 2023: contributor 
     0➜  qa-fuzz git:(fuzz-master) ✗ FUZZ=utxo_total_supply src/test/fuzz/fuzz -runs=1 ../qa-assets-active-fuzzing/fuzz_seed_corpus/utxo_total_supply
 1INFO: Running with entropic power schedule (0xFF, 100).
 2INFO: Seed: 2198204795
 3INFO: Loaded 1 modules   (361490 inline 8-bit counters): 361490 [0x55b85189bf90, 0x55b8518f43a2), 
 4INFO: Loaded 1 PC tables (361490 PCs): 361490 [0x55b8518f43a8,0x55b851e784c8), 
 5INFO:     3247 files found in ../qa-assets-active-fuzzing/fuzz_seed_corpus/utxo_total_supply
 6INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
 7INFO: seed corpus: files: 3247 min: 1b max: 4194303b total: 11985543b rss: 146Mb
 8[#32](/bitcoin-core-qa-assets/32/)	pulse  cov: 12753 ft: 12865 corp: 21/21b exec/s: 10 rss: 261Mb
 9[#64](/bitcoin-core-qa-assets/64/)	pulse  cov: 12767 ft: 12939 corp: 37/47b exec/s: 10 rss: 313Mb
10[#128](/bitcoin-core-qa-assets/128/)	pulse  cov: 12931 ft: 17427 corp: 59/107b exec/s: 9 rss: 318Mb
11[#256](/bitcoin-core-qa-assets/256/)	pulse  cov: 13150 ft: 22006 corp: 104/266b exec/s: 8 rss: 318Mb
12[#512](/bitcoin-core-qa-assets/512/)	pulse  cov: 13401 ft: 30279 corp: 251/1233b exec/s: 8 rss: 318Mb
13[#1024](/bitcoin-core-qa-assets/1024/)	pulse  cov: 13444 ft: 35515 corp: 529/4212b exec/s: 8 rss: 318Mb
14[#2048](/bitcoin-core-qa-assets/2048/)	pulse  cov: 13538 ft: 53104 corp: 1220/25Kb exec/s: 4 rss: 318Mb
15[#3249](/bitcoin-core-qa-assets/3249/)	INITED cov: 14240 ft: 63278 corp: 1644/151Kb exec/s: 0 rss: 409Mb
16[#3249](/bitcoin-core-qa-assets/3249/)	DONE   cov: 14240 ft: 63278 corp: 1644/151Kb lim: 12691 exec/s: 0 rss: 409Mb
17Done 3249 runs in 4797 second(s)

    That’s better, but still a bit slow. Given that the fuzz test changed, I would expect that the runtime should already be reduced but even so. :-/

  8. maflcko commented at 7:59 am on November 8, 2023: contributor
    I don’t know how to reduce the runtime further. Maybe assumeutxo can be used? If not, the only alternative would be to remove the target.
  9. murchandamus commented at 5:49 pm on May 17, 2024: contributor

    I got a new faster computer. I fuzzed utxo_total_supply a bit, and still only got about 2 exec/s on that fuzz target. Then I fuzzed an hour each on a number of very short max_len (1 byte, 4 bytes, 16 bytes, 64 bytes, and 256 bytes), and built a new corpus from my fuzz seeds via set_cover_merge=1.

    Doing a single run of all seeds on the main branch takes me 217 seconds:

     0➜  qa-fuzz git:(fuzz-master) ✗ time FUZZ=utxo_total_supply src/test/fuzz/fuzz ../qa-assets/fuzz_seed_corpus/utxo_total_supply -runs=1 
 1INFO: Running with entropic power schedule (0xFF, 100).
 2INFO: Seed: 1937175595
 3INFO: Loaded 1 modules   (546442 inline 8-bit counters): 546442 [0x5e51a781cc60, 0x5e51a78a22ea), 
 4INFO: Loaded 1 PC tables (546442 PCs): 546442 [0x5e51a78a22f0,0x5e51a80f8b90), 
 5INFO:      981 files found in ../qa-assets/fuzz_seed_corpus/utxo_total_supply
 6INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 316441 bytes
 7INFO: seed corpus: files: 981 min: 1b max: 316441b total: 851769b rss: 130Mb
 8[#128](/bitcoin-core-qa-assets/128/)	pulse  cov: 15662 ft: 47334 corp: 123/2130b exec/s: 42 rss: 288Mb
 9[#256](/bitcoin-core-qa-assets/256/)	pulse  cov: 15739 ft: 54301 corp: 251/9253b exec/s: 17 rss: 290Mb
10[#512](/bitcoin-core-qa-assets/512/)	pulse  cov: 16422 ft: 64810 corp: 494/38Kb exec/s: 7 rss: 290Mb
11[#983](/bitcoin-core-qa-assets/983/)	INITED cov: 16510 ft: 68164 corp: 764/145Kb exec/s: 4 rss: 315Mb
12[#983](/bitcoin-core-qa-assets/983/)	DONE   cov: 16510 ft: 68164 corp: 764/145Kb lim: 16384 exec/s: 4 rss: 315Mb
13Done 983 runs in 217 second(s)
14FUZZ=utxo_total_supply src/test/fuzz/fuzz  -runs=1  145.60s user 47.53s system 88% cpu 3:37.46 total

    Doing a single run on my new corpus took only 101 seconds:

     0➜  qa-fuzz git:(fuzz-master) ✗ time FUZZ=utxo_total_supply src/test/fuzz/fuzz /tmp/utxo_total_supply -runs=1
 1INFO: Running with entropic power schedule (0xFF, 100).
 2INFO: Seed: 964102661
 3INFO: Loaded 1 modules   (546442 inline 8-bit counters): 546442 [0x5fb20591ec60, 0x5fb2059a42ea), 
 4INFO: Loaded 1 PC tables (546442 PCs): 546442 [0x5fb2059a42f0,0x5fb2061fab90), 
 5INFO:      444 files found in /tmp/utxo_total_supply
 6INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 131076 bytes
 7INFO: seed corpus: files: 444 min: 1b max: 131076b total: 263162b rss: 130Mb
 8[#128](/bitcoin-core-qa-assets/128/)	pulse  cov: 15709 ft: 53620 corp: 126/5616b exec/s: 14 rss: 273Mb
 9[#256](/bitcoin-core-qa-assets/256/)	pulse  cov: 16407 ft: 64414 corp: 251/23Kb exec/s: 5 rss: 273Mb
10[#446](/bitcoin-core-qa-assets/446/)	INITED cov: 16441 ft: 67734 corp: 434/125Kb exec/s: 4 rss: 286Mb
11[#446](/bitcoin-core-qa-assets/446/)	DONE   cov: 16441 ft: 67734 corp: 434/125Kb lim: 22550 exec/s: 4 rss: 286Mb
12Done 446 runs in 101 second(s)
13FUZZ=utxo_total_supply src/test/fuzz/fuzz /tmp/utxo_total_supply -runs=1  65.42s user 22.74s system 87% cpu 1:41.30 total

    I do notice that my new corpus only achieved a cov: 16441 vs the main branch getting cov: 16510, but I’m gonna fuzz a bit more with low max_len to see if I can speed up the corpus by replacing longer seeds with short stuff.

  10. dergoegge commented at 5:53 pm on May 17, 2024: member
    iirc utxo_total_supply is mostly bottle-necked by IO, are you using a ramdisk/tmpfs for /tmp? (that should help a little)
  11. murchandamus commented at 7:10 pm on May 17, 2024: contributor
    I was not, but either way, one being twice as fast as the other wouldn’t change much, should it?
  12. murchandamus commented at 1:49 pm on September 12, 2024: contributor
    I just retested this, and with a ram disk and the current set of fuzz inputs, it’s only 74 seconds on my machine. I think this is no longer a pressing concern.
  13. murchandamus closed this on Sep 12, 2024


murchandamus maflcko dergoegge

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin-core/qa-assets. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-10-30 01:25 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me