fuzz: Avoid OOM in transaction fuzz target #27921

maflcko commented at 4:16 PM on June 20, 2023: member

To test: FUZZ=transaction /usr/bin/time -f '%Us %MkB' ./src/test/fuzz/fuzz ../btc_qa_assets/fuzz_seed_corpus/transaction/9dc22b51df0af05ee5a595beefb0ce291feb6b99

Before: 0.72s 249636kB After: 0.30s 92128kB

DrahtBot commented at 4:16 PM on June 20, 2023: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	dergoegge

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

DrahtBot added the label Tests on Jun 20, 2023

maflcko commented at 4:25 PM on June 20, 2023: member

This may also help with:

==16162== ERROR: libFuzzer: out-of-memory (used: 2055Mb; limit: 2048Mb)
   To change the out-of-memory limit use -rss_limit_mb=<N>
MS: 0 ; base unit: 0000000000000000000000000000000000000000
artifact_prefix='./'; Test unit written to ./oom-9dc22b51df0af05ee5a595beefb0ce291feb6b99
SUMMARY: libFuzzer: out-of-memory
Target "/tmp/cirrus-build/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz -runs=1 /tmp/cirrus-build/bitcoin-core/ci/scratch/qa-assets/fuzz_seed_corpus/transaction" failed with exit code 71

https://cirrus-ci.com/task/5750870453780480?logs=ci#L9068

maflcko renamed this:
~~fuzz: Avoid timeout in transaction fuzz target~~
fuzz: Avoid OOM in transaction fuzz target
on Jun 21, 2023

fuzz: Avoid OOM in transaction fuzz target

Also fix bug where the json object is reused between two calls.

fa31c4daac

maflcko force-pushed on Jun 21, 2023

maflcko commented at 6:51 AM on June 21, 2023: member

Checked locally that this fixes the CI task native_fuzz_with_msan

dergoegge approved

dergoegge commented at 10:43 AM on June 21, 2023: member

utACK fa31c4daac5629d14360bbe9b2cd98db4c083989

fanquake commented at 2:33 PM on June 21, 2023: member

reconfirmed that this OOM's, and ran into another issue while doing so:

Run transaction with args ['/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz', '-runs=1', PosixPath('/home/ubuntu/ci_scratch/ci/scratch/qa-assets/fuzz_seed_corpus/transaction')]INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 2714625807
INFO: Loaded 1 modules   (406370 inline 8-bit counters): 406370 [0x55a28ea008e0, 0x55a28ea63c42), 
INFO: Loaded 1 PC tables (406370 PCs): 406370 [0x55a28ea63c48,0x55a28f097268), 
INFO:     3510 files found in /home/ubuntu/ci_scratch/ci/scratch/qa-assets/fuzz_seed_corpus/transaction
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
INFO: seed corpus: files: 3510 min: 1b max: 1855780b total: 192527534b rss: 112Mb
[#1024](/bitcoin-bitcoin/1024/)	pulse  cov: 3002 ft: 6917 corp: 312/16Kb exec/s: 341 rss: 136Mb
[#2048](/bitcoin-bitcoin/2048/)	pulse  cov: 3427 ft: 14596 corp: 845/367Kb exec/s: 75 rss: 138Mb
Slowest unit: 11 s:
artifact_prefix='./'; Test unit written to ./slow-unit-be16ac19bb5b0840ca16f3dfd3280eb9a3031e73
Slowest unit: 15 s:
artifact_prefix='./'; Test unit written to ./slow-unit-96435df971eba2c9c4bb075d3f3528eb3d82cd46
Slowest unit: 23 s:
artifact_prefix='./'; Test unit written to ./slow-unit-639c027d00c0206287054e862ca5f6c9fade1320
Slowest unit: 35 s:
artifact_prefix='./'; Test unit written to ./slow-unit-9ee82a6a83f031c81d2512050ee5ecdc9f3eb527
Slowest unit: 45 s:
artifact_prefix='./'; Test unit written to ./slow-unit-91ee2766360efce6d700daa1eacd872616f6e434
Slowest unit: 72 s:
artifact_prefix='./'; Test unit written to ./slow-unit-ae198ef8388da086d18ebbbcb42cca642faceda7
==30091== ERROR: libFuzzer: out-of-memory (used: 2075Mb; limit: 2048Mb)
   To change the out-of-memory limit use -rss_limit_mb=<N>

MS: 0 ; base unit: 0000000000000000000000000000000000000000
artifact_prefix='./'; Test unit written to ./oom-3ab0860b24a11833b3dec26b47a0a9b8fcc5fcd4
SUMMARY: libFuzzer: out-of-memory

INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 2714625807
INFO: Loaded 1 modules   (406370 inline 8-bit counters): 406370 [0x55a28ea008e0, 0x55a28ea63c42), 
INFO: Loaded 1 PC tables (406370 PCs): 406370 [0x55a28ea63c48,0x55a28f097268), 
INFO:     3510 files found in /home/ubuntu/ci_scratch/ci/scratch/qa-assets/fuzz_seed_corpus/transaction
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
INFO: seed corpus: files: 3510 min: 1b max: 1855780b total: 192527534b rss: 112Mb
[#1024](/bitcoin-bitcoin/1024/)	pulse  cov: 3002 ft: 6917 corp: 312/16Kb exec/s: 341 rss: 136Mb
[#2048](/bitcoin-bitcoin/2048/)	pulse  cov: 3427 ft: 14596 corp: 845/367Kb exec/s: 75 rss: 138Mb
Slowest unit: 11 s:
artifact_prefix='./'; Test unit written to ./slow-unit-be16ac19bb5b0840ca16f3dfd3280eb9a3031e73
Slowest unit: 15 s:
artifact_prefix='./'; Test unit written to ./slow-unit-96435df971eba2c9c4bb075d3f3528eb3d82cd46
Slowest unit: 23 s:
artifact_prefix='./'; Test unit written to ./slow-unit-639c027d00c0206287054e862ca5f6c9fade1320
Slowest unit: 35 s:
artifact_prefix='./'; Test unit written to ./slow-unit-9ee82a6a83f031c81d2512050ee5ecdc9f3eb527
Slowest unit: 45 s:
artifact_prefix='./'; Test unit written to ./slow-unit-91ee2766360efce6d700daa1eacd872616f6e434
Slowest unit: 72 s:
artifact_prefix='./'; Test unit written to ./slow-unit-ae198ef8388da086d18ebbbcb42cca642faceda7
==30091== ERROR: libFuzzer: out-of-memory (used: 2075Mb; limit: 2048Mb)
   To change the out-of-memory limit use -rss_limit_mb=<N>

MS: 0 ; base unit: 0000000000000000000000000000000000000000
artifact_prefix='./'; Test unit written to ./oom-3ab0860b24a11833b3dec26b47a0a9b8fcc5fcd4
SUMMARY: libFuzzer: out-of-memory

Traceback (most recent call last):
  File "/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 298, in run_once
    result.check_returncode()
  File "/usr/lib/python3.10/subprocess.py", line 456, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '['/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz', '-runs=1', PosixPath('/home/ubuntu/ci_scratch/ci/scratch/qa-assets/fuzz_seed_corpus/transaction')]' returned non-zero exit status 71.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 322, in <module>
    main()
  File "/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 183, in main
    run_once(
  File "/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 304, in run_once
    logging.info("Target \"{}\" failed with exit code {}".format(" ".join(result.args), e.returncode))
TypeError: sequence item 2: expected str instance, PosixPath found

real	195m7.994s

maflcko commented at 3:00 PM on June 21, 2023: member

" ".join(result.args)
TypeError: sequence item 2: expected str instance, PosixPath found

Yeah, you can replace this with " ".join(str(result.args)), if you want.

maflcko commented at 3:02 PM on June 21, 2023: member

An alternative to avoid TypeError would be to rewrite the fuzz/test_runner script to a rust script from python.

fanquake merged this on Jun 21, 2023

fanquake closed this on Jun 21, 2023

maflcko deleted the branch on Jun 21, 2023

sidhujag referenced this in commit 966db682ca on Jun 22, 2023

bitcoin locked this on Jun 20, 2024