build: set boost cppflags with --enable-fuzz #27057

fanquake commented at 5:31 PM on February 7, 2023: member

Even though all other targets are disabled, we still need Boost CPPFLAGS (use_boost) to compile. This currently works everywhere, except on arm macOS (where the include path is non-standard), because generally, the Boost include path is generic, i.e /usr/include.

DrahtBot commented at 5:31 PM on February 7, 2023: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	hebasto
Stale ACK	jonatack

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

DrahtBot added the label Build system on Feb 7, 2023

in configure.ac:1452 in 7c95e6e92c outdated

1448 | @@ -1449,7 +1449,7 @@ if test "$use_natpmp" != "no"; then
1449 |    CPPFLAGS="$TEMP_CPPFLAGS"
1450 |  fi
1451 |  
1452 | -if test "$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_util$build_bitcoind$bitcoin_enable_qt$use_tests$use_bench" = "nononononononono"; then
1453 | +if test "$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_util$build_bitcoind$bitcoin_enable_qt$use_tests$use_bench$enable_fuzz" = "nononononononono"; then

maflcko commented at 5:48 PM on February 7, 2023:

missing no?

fanquake commented at 9:45 AM on February 8, 2023:

yea

jonatack commented at 5:57 PM on February 7, 2023: contributor

ACK 7c95e6e92c53da8564fae2bce919b2c53926ab00

Tested a fuzz build on an M1 running macOS 13.2 and ran a fuzzer on this branch, using the updated macOS configure documentation in #27056 with the following (suggest adding --enable-suppress-external-warnings to your configure):

brew install llvm
make clean && ./autogen.sh && ./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined --disable-asm CC=$(brew --prefix llvm)/bin/clang CXX=$(brew --prefix llvm)/bin/clang++ --enable-suppress-external-warnings && make -j11
FUZZ=process_message ./src/test/fuzz/fuzz

fanquake force-pushed on Feb 8, 2023

in configure.ac:1452 in 11caed2edb outdated

1448 | @@ -1449,7 +1449,7 @@ if test "$use_natpmp" != "no"; then
1449 |    CPPFLAGS="$TEMP_CPPFLAGS"
1450 |  fi
1451 |  
1452 | -if test "$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_util$build_bitcoind$bitcoin_enable_qt$use_tests$use_bench" = "nononononononono"; then
1453 | +if test "$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_util$build_bitcoind$bitcoin_enable_qt$use_tests$use_bench$enable_fuzz" = "nonononononononono"; then

hebasto commented at 12:03 PM on February 8, 2023:

Actually, the enable_fuzz_binary variable is responsible for building the test/fuzz/fuzz target, not enable_fuzz.

fanquake commented at 12:12 PM on February 8, 2023:

I wont make much difference, because they will both be set. Changed to enable_fuzz_binary.

hebasto commented at 2:04 PM on February 8, 2023:

I wont make much difference

It will. With my testing setup I have an error for 11caed2edbadc5ef2078550c2f323bf9855f5d8d:

$ make
Making all in src
make[1]: Entering directory '/home/hebasto/git/bitcoin/src'
make[2]: Entering directory '/home/hebasto/git/bitcoin/src'
  CXX      test/fuzz/fuzz-addition_overflow.o
  CXX      test/fuzz/fuzz-addrman.o
  CXX      test/fuzz/fuzz-asmap.o
  CXX      test/fuzz/fuzz-asmap_direct.o
  CXX      test/fuzz/fuzz-autofile.o
  CXX      test/fuzz/fuzz-banman.o
  CXX      test/fuzz/fuzz-base_encode_decode.o
  CXX      test/fuzz/fuzz-bitdeque.o
  CXX      test/fuzz/fuzz-block.o
  CXX      test/fuzz/fuzz-block_header.o
  CXX      test/fuzz/fuzz-blockfilter.o
  CXX      test/fuzz/fuzz-bloom_filter.o
  CXX      test/fuzz/fuzz-buffered_file.o
  CXX      test/fuzz/fuzz-chain.o
  CXX      test/fuzz/fuzz-checkqueue.o
  CXX      test/fuzz/fuzz-coins_view.o
In file included from ./validation.h:29,
                 from test/fuzz/block.cpp:14:
./txmempool.h:35:10: fatal error: boost/multi_index/hashed_index.hpp: No such file or directory
   35 | #include <boost/multi_index/hashed_index.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [Makefile:16020: test/fuzz/fuzz-block.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from ./validation.h:29,
                 from test/fuzz/coins_view.cpp:20:
./txmempool.h:35:10: fatal error: boost/multi_index/hashed_index.hpp: No such file or directory
   35 | #include <boost/multi_index/hashed_index.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [Makefile:16118: test/fuzz/fuzz-coins_view.o] Error 1
make[2]: Leaving directory '/home/hebasto/git/bitcoin/src'
make[1]: *** [Makefile:19229: all-recursive] Error 1
make[1]: Leaving directory '/home/hebasto/git/bitcoin/src'
make: *** [Makefile:815: all-recursive] Error 1

fanquake force-pushed on Feb 8, 2023

hebasto approved

hebasto commented at 2:00 PM on February 8, 2023: member

ACK 3f764796103a73236aaabe185f7c1ccd924187a1, tested on Ubuntu 22.04 with @theuni's patch and no system Boost package installed:

$ make -C depends NO_QT=1 NO_WALLET=1 NO_UPNP=1 NO_NATPMP=1 NO_ZMQ=1
$ ./autogen.sh
$ ./configure --without-utils --without-daemon --disable-tests -disable-bench CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site
$ make clean
$ make

in configure.ac:1454 in 3f76479610 outdated

1448 | @@ -1449,7 +1449,7 @@ if test "$use_natpmp" != "no"; then
1449 |    CPPFLAGS="$TEMP_CPPFLAGS"
1450 |  fi
1451 |  
1452 | -if test "$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_util$build_bitcoind$bitcoin_enable_qt$use_tests$use_bench" = "nononononononono"; then
1453 | +if test "$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_util$build_bitcoind$bitcoin_enable_qt$use_tests$use_bench$enable_fuzz_binary" = "nonononononononono"; then

hebasto commented at 2:01 PM on February 8, 2023:

Unrelated: Should the $build_bitcoin_tx be dropped from here after #26086?

fanquake commented at 3:32 PM on February 8, 2023:

Ok. I'll combine that into here while we are already changing the same line.

fanquake commented at 3:40 PM on February 8, 2023:

Added.

fanquake force-pushed on Feb 8, 2023

hebasto commented at 4:01 PM on February 8, 2023: member

Unrelated: Should the $build_bitcoin_tx be dropped from here after #26086?

Added.

Apparently, I was wrong. Tested f8b68c1f63f3a942712ef4eeebbdb231b3884f13:

$ ./autogen.sh
$ ./configure -q --without-utils --enable-util-tx --without-daemon --disable-tests -disable-bench --disable-fuzz-binary --without-libs CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site
$ make
...
  CXX      common/libbitcoin_common_a-interfaces.o
common/interfaces.cpp:8:10: fatal error: boost/signals2/connection.hpp: No such file or directory
    8 | #include <boost/signals2/connection.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
...

Sorry for the noise.

build: set boost cppflags with --enable-fuzz

Even though all other targets are disabled, we still need Boost CPPFLAGS
(use_boost) to compile. This currently works everywhere, except on arm
macOS (where the include path is pretty non-standard), because
generally, the Boost include path is generic, i.e `/usr/include`.

b03a98291b

fanquake force-pushed on Feb 8, 2023

hebasto approved

hebasto commented at 4:19 PM on February 8, 2023: member

ACK b03a98291bc950d33d8e00e6fa578a9e3f8a1852

fanquake merged this on Feb 8, 2023

fanquake closed this on Feb 8, 2023

sidhujag referenced this in commit 131c0d79d9 on Feb 8, 2023

fanquake deleted the branch on Feb 13, 2023

bitcoin locked this on Feb 13, 2024