build: actually set CXX and CXXFLAGS for native

fanquake commented at 8:02 am on November 19, 2021: member

Currently, ./build.sh doesn’t seem to respect CXX and CXXFLAGS. This becomes more obvious when you build depends with CC and CXX set, without another compiler installed. i.e when using g++-8 on Ubuntu Bionic, but not having g++ (GGC-7) installed. The native_b2 build will fail with:

0make -C depends/ -j9 CC=gcc-8 CXX=g++-8
1...
2/bitcoin/depends/sources/boost_1_71_0.tar.bz2: OK
3Preprocessing native_b2...
4Configuring native_b2...
5Building native_b2...
6B2_TOOLSET is gcc, but the 'gcc' command cannot be executed.
7Make sure 'gcc' is in PATH, or use a different toolset.

Instead, don’t set CXX or CXXFLAGS before calling build.sh. In this case, we either get the default CXX (g++), or, we’ll get the CXX set by the user. i.e:

 0make -C depends/ -j9 CC=gcc-8 CXX=g++-8
 1...
 2/bitcoin/depends/sources/boost_1_71_0.tar.bz2: OK
 3Preprocessing native_b2...
 4Configuring native_b2...
 5Building native_b2...
 6g++-8 --version
 7g++-8 (Ubuntu 8.4.0-1ubuntu1~18.04) 8.4.0
 8Copyright (C) 2018 Free Software Foundation, Inc.
 9This is free software; see the source for copying conditions.  There is NO
10warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
11
12g++-8 -x c++ -std=c++11 -O2 -s .... <snip>

fanquake added the label Build system on Nov 19, 2021

fanquake added the label DrahtBot Guix build requested on Nov 19, 2021

fanquake requested review from dongcarl on Nov 19, 2021

hebasto commented at 8:03 am on November 19, 2021: member

Concept ACK.

MarcoFalke commented at 9:20 am on November 19, 2021: member

Does this allow to simplify the msan ci with boost_cxxflags?

dongcarl commented at 6:06 pm on November 19, 2021: member

Actually, boost 1.71.0 (the version we use)’s ./build.sh does respect CXX env vars, see:

https://github.com/boostorg/build/blob/boost-1.71.0/src/engine/build.sh#L157

However, this was changed in https://github.com/boostorg/build/commit/7497f61dde3e38a098080449c2031bb87896e431, which was first included in boost 1.76.0

The reason why you’re seeing the current breakage in the first case is because our well-known var propagation in depends is limited to non-native packages, native packages do not yet get them: #18820 (comment).

However, I have not figured out why it works in the second case. My reading of boost’s build.sh as of 1.71.0 seems to indicate that the --cxx and --cxxflags would be ignored: https://github.com/boostorg/build/blob/04efaac148671479b514db882b1c520dcaafe89e/src/engine/build.sh#L149-L155

So perhaps there’s a CXX env var that’s set somewhere in depends that we don’t know about? Curious.

DrahtBot commented at 10:11 pm on November 20, 2021: member

Guix builds

File	commit fe03f7a37fd0ef05149161f6b95a25493e1fe38f(master)	commit 32f340ba7ad12008cb5bbde535e5eec9dd128a8f(master and this pull)
SHA256SUMS.part	`044bb745547def7a...`	`a540a026627d762c...`
*-aarch64-linux-gnu-debug.tar.gz	`09bbff302ad3331e...`	`cfa7d315f8340a9a...`
*-aarch64-linux-gnu.tar.gz	`430b9f97f423b06a...`	`260f643ece176933...`
*-arm-linux-gnueabihf-debug.tar.gz	`50a27653d6994b6a...`	`e78afb97501fdc1d...`
*-arm-linux-gnueabihf.tar.gz	`81a07a9da6f858a0...`	`6f1046d349c70172...`
*-osx-unsigned.dmg	`3e2009f5309e90c5...`	`3d76472b83cb69c5...`
*-osx-unsigned.tar.gz	`4b62c341efb32478...`	`f56fa9436b77af0f...`
*-osx64.tar.gz	`15db1c07fa4e9d30...`	`4ac9ee302dc308d2...`
*-powerpc64-linux-gnu-debug.tar.gz	`a1e42ccb04825279...`	`ac47d7d19548ee18...`
*-powerpc64-linux-gnu.tar.gz	`378f31682cfccc5f...`	`179d75cdfdef7b55...`
*-powerpc64le-linux-gnu-debug.tar.gz	`7a93b3ab8e90cf7e...`	`9e6f4edbb304f765...`
*-powerpc64le-linux-gnu.tar.gz	`a1277ecf841344cf...`	`730b64a10a2b07de...`
*-riscv64-linux-gnu-debug.tar.gz	`b43821a269135bd8...`	`b093538d744e2e23...`
*-riscv64-linux-gnu.tar.gz	`7e274b4336478453...`	`b85922e22e83b9e2...`
*-win-unsigned.tar.gz	`99e7f5561fe50fff...`	`2a506a2bb4ccd875...`
*-win64-debug.zip	`41e203dfc0daedec...`	`c09814d390c57bf3...`
*-win64-setup-unsigned.exe	`63dc48e24fa4c39a...`	`82057dfe19025a8e...`
*-win64.zip	`5535e45ef7494fba...`	`fc22d0050ea949c0...`
*-x86_64-linux-gnu-debug.tar.gz	`38eaad4f9ea9d1c6...`	`44fc15627569b7ee...`
*-x86_64-linux-gnu.tar.gz	`a33a93e0a8ac54a5...`	`94fae578176d774c...`
*.tar.gz	`f5fcc5155e947e72...`	`25d65c8d33f850ac...`
guix_build.log	`fb12e25d5ee0f080...`	`35ccd4e8cfcc4b8e...`
guix_build.log.diff		`57e93b4c9abcb153...`

DrahtBot removed the label DrahtBot Guix build requested on Nov 20, 2021

build: actually set CXX and CXXFLAGS for native_b2 build

Currently, ./build.sh doesn't seem to respect `CXX` and `CXXFLAGS`.
This becomes more obvious when you build depends with `CC` and `CXX` set,
without another compiler installed. i.e when using `g++-8` on Ubuntu
Bionic, but not having `g++` (GGC-7) installed. The native_b2 build will
fail with:

```bash
make -C depends/ -j9 CC=gcc-8 CXX=g++-8
...
/bitcoin/depends/sources/boost_1_71_0.tar.bz2: OK
Preprocessing native_b2...
Configuring native_b2...
Building native_b2...
B2_TOOLSET is gcc, but the 'gcc' command cannot be executed.
Make sure 'gcc' is in PATH, or use a different toolset.
```

Instead, don't set CXX or CXXFLAGS before calling `build.sh`.
In this case, we either get the default CXX (`g++`), or, we'll
get the CXX set by the user. i.e:
```bash
make -C depends/ -j9 CC=gcc-8 CXX=g++-8
...
/bitcoin/depends/sources/boost_1_71_0.tar.bz2: OK
Preprocessing native_b2...
Configuring native_b2...
Building native_b2...
g++-8 --version
g++-8 (Ubuntu 8.4.0-1ubuntu1~18.04) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

g++-8 -x c++ -std=c++11 -O2 -s .... <snip>
```

eaa42d049e

fanquake force-pushed on Nov 22, 2021

fanquake commented at 3:56 am on November 22, 2021: member

@dongcarl Thanks for having a look.

The reason why you’re seeing the current breakage in the first case is because our well-known var propagation in depends is limited to non-native packages, native packages do not yet get them: #18820 (comment).

I’m not sure about this. If I just remove the setting of CXX and CXXFLAGS before calling ./build.sh, the package ends up being built with the CXX set on the command line. I was fooled into thinking that our current Boost supported the --cxx options, as when I started using them, things were compiled as expected. However I think the actual fix here, was removing the the setting of CXX and CXXFLAGS, which was undermining the CXX and CXXFLAGS being set by the user.

I’ve updated this change to just drop CXX="$($(package)_cxx)" CXXFLAGS="$($(package)_cxxflags)", and everything now works as expected. If CXX is not set, you end up with CXX=g++, which is the current behaviour. However if CXX is set, that compiler will be used, as opposed to the build failure which happens now.

We can look at using --cxx if/when we move to a newer Boost, i.e: #23340.

hebasto commented at 11:16 am on November 22, 2021: member

This is still broken:

 0$ make --no-print-directory -C depends native_b2 CC=clang CXX=clang++
 1Extracting native_b2...
 2/home/hebasto/GitHub/bitcoin/depends/sources/boost_1_71_0.tar.bz2: OK
 3Preprocessing native_b2...
 4Configuring native_b2...
 5Building native_b2...
 6###
 7###
 8### Using 'gcc' toolset.
 9###
10###
11clang++ --version
12clang version 10.0.0-4ubuntu1 
13Target: x86_64-pc-linux-gnu
14Thread model: posix
15InstalledDir: /usr/bin
16###
17###
18clang++ -x c++ -std=c++11 -O2 -s -DNDEBUG builtins.cpp class.cpp command.cpp compile.cpp constants.cpp cwd.cpp debug.cpp debugger.cpp execcmd.cpp filesys.cpp frames.cpp function.cpp glob.cpp hash.cpp hcache.cpp hdrmacro.cpp headers.cpp jam.cpp jambase.cpp jamgram.cpp lists.cpp make.cpp make1.cpp md5.cpp mem.cpp modules.cpp native.cpp object.cpp option.cpp output.cpp parse.cpp pathsys.cpp regexp.cpp rules.cpp scan.cpp search.cpp strings.cpp subst.cpp sysinfo.cpp timestamp.cpp variable.cpp w32_getreg.cpp modules/order.cpp modules/path.cpp modules/property-set.cpp modules/regex.cpp modules/sequence.cpp modules/set.cpp execunix.cpp fileunix.cpp pathunix.cpp -o b2
19cp b2 bjam
20Staging native_b2...
21Postprocessing native_b2...
22Caching native_b2...

Note Using 'gcc' toolset.

Please consider an alternative #23571.

fanquake commented at 0:52 am on November 23, 2021: member

Please consider an alternative #23571.

Lets look at that for now.

fanquake closed this on Nov 23, 2021

DrahtBot locked this on Nov 23, 2022

fanquake deleted the branch on Nov 29, 2022

build: actually set CXX and CXXFLAGS for native_b2 build #23551

Guix builds