ci: Run fuzz target even if input folder is empty #27919

maflcko commented at 1:14 PM on June 20, 2023: member

This should catch trivial integer sanitizer bugs if the author and all reviewers forget to look for them.

maflcko force-pushed on Jun 20, 2023

DrahtBot commented at 1:14 PM on June 20, 2023: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	brunoerg, dergoegge

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

DrahtBot added the label Tests on Jun 20, 2023

dergoegge commented at 1:18 PM on June 20, 2023: member

Concept ACK

in test/fuzz/test_runner.py:272 in fa39e50e41 outdated

 270 |              os.path.join(build_dir, 'src', 'test', 'fuzz', 'fuzz'),
 271 |              '-runs=1',
 272 | -            corpus_path,
 273 |          ]
 274 | +        if not any(corpus_path.iterdir()) and empty_min_time:
 275 | +            args.append(f"-max_total_time={empty_min_time}")

dergoegge commented at 1:23 PM on June 20, 2023:

this would still include runs=1 which for an empty corpus would only generate 1 input.

Run process_messages with args ['/home/dergoegge/workspace/bitcoin/src/test/fuzz/fuzz', '-runs=1', '-max_total_time=10']INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 2455678762
INFO: Loaded 1 modules   (350793 inline 8-bit counters): 350793 [0x55c579a854b0, 0x55c579adaef9),
INFO: Loaded 1 PC tables (350793 PCs): 350793 [0x55c579adaf00,0x55c57a035390),
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: A corpus is not provided, starting from an empty corpus
[#2](/bitcoin-bitcoin/2/)      INITED cov: 2324 ft: 2325 corp: 1/1b exec/s: 0 rss: 115Mb
[#2](/bitcoin-bitcoin/2/)      DONE   cov: 2324 ft: 2325 corp: 1/1b lim: 4 exec/s: 0 rss: 115Mb
Done 2 runs in 0 second(s)

maflcko force-pushed on Jun 20, 2023

dergoegge commented at 2:03 PM on June 20, 2023: member

Tested it locally, seems to work. We should also see this work in the CI here shortly for the utxo_total_supply, wallet_fees and coincontrol targets.

brunoerg commented at 2:16 PM on June 20, 2023: contributor

Strong Concept ACK

dergoegge approved

dergoegge commented at 2:21 PM on June 20, 2023: member

ACK fae0c9955ed8254665c9afbaf940f1f4295bfa29

It is working and found the wallet_fees bug.

We should probably wait with merging until #27917 is in.

DrahtBot added the label CI failed on Jun 20, 2023

dergoegge commented at 3:52 PM on June 20, 2023: member

#27917 was merged, so this should have a green CI after rebase.

brunoerg commented at 4:11 PM on June 20, 2023: contributor

Tested locally and it got wallet_fees bug.

brunoerg approved

brunoerg commented at 4:12 PM on June 20, 2023: contributor

ACK fae0c9955ed8254665c9afbaf940f1f4295bfa29

Gonna re-ack as soon as it got rebased

ci: Run fuzz target even if input folder is empty 0000f55293

maflcko force-pushed on Jun 20, 2023

brunoerg commented at 4:21 PM on June 20, 2023: contributor

reACK 0000f552937ee787d25c8fd0af3278ea94889216

DrahtBot requested review from dergoegge on Jun 20, 2023

DrahtBot removed the label CI failed on Jun 20, 2023

in ci/test/06_script_b.sh:173 in 0000f55293

 169 | @@ -170,5 +170,5 @@ if [ "${RUN_TIDY}" = "true" ]; then
 170 |  fi
 171 |  
 172 |  if [ "$RUN_FUZZ_TESTS" = "true" ]; then
 173 | -  bash -c "LD_LIBRARY_PATH=${DEPENDS_DIR}/${HOST}/lib test/fuzz/test_runner.py ${FUZZ_TESTS_CONFIG} $MAKEJOBS -l DEBUG ${DIR_FUZZ_IN}"
 174 | +  bash -c "LD_LIBRARY_PATH=${DEPENDS_DIR}/${HOST}/lib test/fuzz/test_runner.py ${FUZZ_TESTS_CONFIG} $MAKEJOBS -l DEBUG ${DIR_FUZZ_IN} --empty_min_time=60"

dergoegge commented at 8:44 AM on June 21, 2023:

We can probably play around with this value but for now 60s seems fine to me.

maflcko commented at 8:52 AM on June 21, 2023:

Yeah, I think it should be 10 seconds or more, but less than 2 minutes? With 1 minute, even with 60 missing fuzz input folders, the overhead will be 1 hour for all runs until fuzz inputs are provided, which seems fine.

dergoegge approved

dergoegge commented at 8:44 AM on June 21, 2023: member

reACK 0000f552937ee787d25c8fd0af3278ea94889216

fanquake merged this on Jun 21, 2023

fanquake closed this on Jun 21, 2023

maflcko deleted the branch on Jun 21, 2023

sidhujag referenced this in commit 02c33b9e56 on Jun 21, 2023

bitcoin locked this on Jun 20, 2024