fuzz: Avoid OOM in transaction fuzz target

maflcko commented at 4:16 pm on June 20, 2023: member

To test: FUZZ=transaction /usr/bin/time -f '%Us %MkB' ./src/test/fuzz/fuzz ../btc_qa_assets/fuzz_seed_corpus/transaction/9dc22b51df0af05ee5a595beefb0ce291feb6b99

Before: 0.72s 249636kB After: 0.30s 92128kB

DrahtBot commented at 4:16 pm on June 20, 2023: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	dergoegge

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

DrahtBot added the label Tests on Jun 20, 2023

maflcko commented at 4:25 pm on June 20, 2023: member

This may also help with:

0==16162== ERROR: libFuzzer: out-of-memory (used: 2055Mb; limit: 2048Mb)
1   To change the out-of-memory limit use -rss_limit_mb=<N>
2MS: 0 ; base unit: 0000000000000000000000000000000000000000
3artifact_prefix='./'; Test unit written to ./oom-9dc22b51df0af05ee5a595beefb0ce291feb6b99
4SUMMARY: libFuzzer: out-of-memory
5Target "/tmp/cirrus-build/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz -runs=1 /tmp/cirrus-build/bitcoin-core/ci/scratch/qa-assets/fuzz_seed_corpus/transaction" failed with exit code 71

https://cirrus-ci.com/task/5750870453780480?logs=ci#L9068

maflcko renamed this:
~~fuzz: Avoid timeout in transaction fuzz target~~
fuzz: Avoid OOM in transaction fuzz target
on Jun 21, 2023

fuzz: Avoid OOM in transaction fuzz target

Also fix bug where the json object is reused between two calls.

fa31c4daac

maflcko force-pushed on Jun 21, 2023

maflcko commented at 6:51 am on June 21, 2023: member

Checked locally that this fixes the CI task native_fuzz_with_msan

dergoegge approved

dergoegge commented at 10:43 am on June 21, 2023: member

utACK fa31c4daac5629d14360bbe9b2cd98db4c083989

fanquake commented at 2:33 pm on June 21, 2023: member

reconfirmed that this OOM’s, and ran into another issue while doing so:

 0Run transaction with args ['/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz', '-runs=1', PosixPath('/home/ubuntu/ci_scratch/ci/scratch/qa-assets/fuzz_seed_corpus/transaction')]INFO: Running with entropic power schedule (0xFF, 100).
 1INFO: Seed: 2714625807
 2INFO: Loaded 1 modules   (406370 inline 8-bit counters): 406370 [0x55a28ea008e0, 0x55a28ea63c42), 
 3INFO: Loaded 1 PC tables (406370 PCs): 406370 [0x55a28ea63c48,0x55a28f097268), 
 4INFO:     3510 files found in /home/ubuntu/ci_scratch/ci/scratch/qa-assets/fuzz_seed_corpus/transaction
 5INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
 6INFO: seed corpus: files: 3510 min: 1b max: 1855780b total: 192527534b rss: 112Mb
 7[#1024](/bitcoin-bitcoin/1024/)	pulse  cov: 3002 ft: 6917 corp: 312/16Kb exec/s: 341 rss: 136Mb
 8[#2048](/bitcoin-bitcoin/2048/)	pulse  cov: 3427 ft: 14596 corp: 845/367Kb exec/s: 75 rss: 138Mb
 9Slowest unit: 11 s:
10artifact_prefix='./'; Test unit written to ./slow-unit-be16ac19bb5b0840ca16f3dfd3280eb9a3031e73
11Slowest unit: 15 s:
12artifact_prefix='./'; Test unit written to ./slow-unit-96435df971eba2c9c4bb075d3f3528eb3d82cd46
13Slowest unit: 23 s:
14artifact_prefix='./'; Test unit written to ./slow-unit-639c027d00c0206287054e862ca5f6c9fade1320
15Slowest unit: 35 s:
16artifact_prefix='./'; Test unit written to ./slow-unit-9ee82a6a83f031c81d2512050ee5ecdc9f3eb527
17Slowest unit: 45 s:
18artifact_prefix='./'; Test unit written to ./slow-unit-91ee2766360efce6d700daa1eacd872616f6e434
19Slowest unit: 72 s:
20artifact_prefix='./'; Test unit written to ./slow-unit-ae198ef8388da086d18ebbbcb42cca642faceda7
21==30091== ERROR: libFuzzer: out-of-memory (used: 2075Mb; limit: 2048Mb)
22   To change the out-of-memory limit use -rss_limit_mb=<N>
23
24MS: 0 ; base unit: 0000000000000000000000000000000000000000
25artifact_prefix='./'; Test unit written to ./oom-3ab0860b24a11833b3dec26b47a0a9b8fcc5fcd4
26SUMMARY: libFuzzer: out-of-memory
27
28INFO: Running with entropic power schedule (0xFF, 100).
29INFO: Seed: 2714625807
30INFO: Loaded 1 modules   (406370 inline 8-bit counters): 406370 [0x55a28ea008e0, 0x55a28ea63c42), 
31INFO: Loaded 1 PC tables (406370 PCs): 406370 [0x55a28ea63c48,0x55a28f097268), 
32INFO:     3510 files found in /home/ubuntu/ci_scratch/ci/scratch/qa-assets/fuzz_seed_corpus/transaction
33INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
34INFO: seed corpus: files: 3510 min: 1b max: 1855780b total: 192527534b rss: 112Mb
35[#1024](/bitcoin-bitcoin/1024/)	pulse  cov: 3002 ft: 6917 corp: 312/16Kb exec/s: 341 rss: 136Mb
36[#2048](/bitcoin-bitcoin/2048/)	pulse  cov: 3427 ft: 14596 corp: 845/367Kb exec/s: 75 rss: 138Mb
37Slowest unit: 11 s:
38artifact_prefix='./'; Test unit written to ./slow-unit-be16ac19bb5b0840ca16f3dfd3280eb9a3031e73
39Slowest unit: 15 s:
40artifact_prefix='./'; Test unit written to ./slow-unit-96435df971eba2c9c4bb075d3f3528eb3d82cd46
41Slowest unit: 23 s:
42artifact_prefix='./'; Test unit written to ./slow-unit-639c027d00c0206287054e862ca5f6c9fade1320
43Slowest unit: 35 s:
44artifact_prefix='./'; Test unit written to ./slow-unit-9ee82a6a83f031c81d2512050ee5ecdc9f3eb527
45Slowest unit: 45 s:
46artifact_prefix='./'; Test unit written to ./slow-unit-91ee2766360efce6d700daa1eacd872616f6e434
47Slowest unit: 72 s:
48artifact_prefix='./'; Test unit written to ./slow-unit-ae198ef8388da086d18ebbbcb42cca642faceda7
49==30091== ERROR: libFuzzer: out-of-memory (used: 2075Mb; limit: 2048Mb)
50   To change the out-of-memory limit use -rss_limit_mb=<N>
51
52MS: 0 ; base unit: 0000000000000000000000000000000000000000
53artifact_prefix='./'; Test unit written to ./oom-3ab0860b24a11833b3dec26b47a0a9b8fcc5fcd4
54SUMMARY: libFuzzer: out-of-memory
55
56Traceback (most recent call last):
57  File "/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 298, in run_once
58    result.check_returncode()
59  File "/usr/lib/python3.10/subprocess.py", line 456, in check_returncode
60    raise CalledProcessError(self.returncode, self.args, self.stdout,
61subprocess.CalledProcessError: Command '['/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz', '-runs=1', PosixPath('/home/ubuntu/ci_scratch/ci/scratch/qa-assets/fuzz_seed_corpus/transaction')]' returned non-zero exit status 71.
62
63During handling of the above exception, another exception occurred:
64
65Traceback (most recent call last):
66  File "/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 322, in <module>
67    main()
68  File "/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 183, in main
69    run_once(
70  File "/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 304, in run_once
71    logging.info("Target \"{}\" failed with exit code {}".format(" ".join(result.args), e.returncode))
72TypeError: sequence item 2: expected str instance, PosixPath found
73
74real	195m7.994s

maflcko commented at 3:00 pm on June 21, 2023: member

0" ".join(result.args)
1TypeError: sequence item 2: expected str instance, PosixPath found

Yeah, you can replace this with " ".join(str(result.args)), if you want.

maflcko commented at 3:02 pm on June 21, 2023: member

An alternative to avoid TypeError would be to rewrite the fuzz/test_runner script to a rust script from python.

fanquake merged this on Jun 21, 2023

fanquake closed this on Jun 21, 2023

maflcko deleted the branch on Jun 21, 2023

sidhujag referenced this in commit 966db682ca on Jun 22, 2023

bitcoin locked this on Jun 20, 2024

fuzz: Avoid OOM in transaction fuzz target #27921

Reviews