macOS Catalina #16387

New macOS releases have historically led to unpleasant surprises, so I decided to run the beta on an older machine. I installed both the beta of macOS and Xcode, and made the latter default using sudo xcode-select --switch /Applications/Xcode-beta.app/Contents/Developer.

Note that Apple switched the default shell to zsh: https://support.apple.com/kb/HT208050

Beta 3

TL&DR: binaries still work, but building QT from source doesn't and depends seems broken

The existing binary on my system kept working fine. I was also able to download and install v0.18 from bitcoincore.org.

Compling master (536590f358dc3d3e5821eba7f1009452ea93b205) from source source fails, see (log).

Using Homebrew with boost 1.70.0 and qt 5.13.0.

The tests produce a bunch of warnings:

test/util_threadnames_tests.cpp:64:5: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::check_frwd<boost::test_tools::tt_detail::equal_impl_frwd, unsigned long, int>' requested here
    BOOST_CHECK_EQUAL(names.size(), 100);
    ^
/usr/local/include/boost/test/tools/old/interface.hpp:154:45: note: expanded from macro 'BOOST_CHECK_EQUAL'
#define BOOST_CHECK_EQUAL( L, R )           BOOST_TEST_TOOL_IMPL( 0, \
                                            ^
/usr/local/include/boost/test/tools/old/interface.hpp:67:47: note: expanded from macro 'BOOST_TEST_TOOL_IMPL'
    BOOST_PP_IF( frwd_type, report_assertion, check_frwd ) (                    \

These warnings also occur in the Solaris problem in #15708.

QT produces warnings:

  CXX      qt/libbitcoinqt_a-bitcoinunits.o
qt/bitcoinamountfield.cpp:124:24: warning: 'width' is deprecated: Use QFontMetrics::horizontalAdvance [-Wdeprecated-declarations]
            int w = fm.width(BitcoinUnits::format(BitcoinUnits::BTC, BitcoinUnits::maxMoney(), false, BitcoinUnits::separatorAlways));
                       ^
/usr/local/Cellar/qt/5.13.0/lib/QtGui.framework/Headers/qfontmetrics.h:107:5: note: 'width' has been explicitly marked deprecated here
    QT_DEPRECATED_X("Use QFontMetrics::horizontalAdvance")
    ^
/usr/local/Cellar/qt/5.13.0/lib/QtCore.framework/Headers/qglobal.h:294:33: note: expanded from macro 'QT_DEPRECATED_X'
#  define QT_DEPRECATED_X(text) Q_DECL_DEPRECATED_X(text)
                                ^
/usr/local/Cellar/qt/5.13.0/lib/QtCore.framework/Headers/qcompilerdetection.h:649:55: note: expanded from macro 'Q_DECL_DEPRECATED_X'
#    define Q_DECL_DEPRECATED_X(text) __attribute__ ((__deprecated__(text)))
                                                      ^

And finally QT throws an error:

  CXX      qt/libbitcoinqt_a-networkstyle.o
qt/guiutil.cpp:365:14: error: no matching function for call to 'objc_msgSend'
    id app = objc_msgSend((id) objc_getClass("NSApplication"), sel_registerName("sharedApplication"));
             ^~~~~~~~~~~~
/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/objc/message.h:63:1: note: candidate function not viable: requires 0 arguments, but 2 were provided
objc_msgSend(void /* id self, SEL op, ... */ )
^
qt/guiutil.cpp:366:5: error: no matching function for call to 'objc_msgSend'
    objc_msgSend(app, sel_registerName("activateIgnoringOtherApps:"), YES);
    ^~~~~~~~~~~~
/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/objc/message.h:63:1: note: candidate function not viable: requires 0 arguments, but 3 were provided
objc_msgSend(void /* id self, SEL op, ... */ )
^
2 errors generated.
make[2]: *** [qt/libbitcoinqt_a-guiutil.o] Error 1

I can ./configure --without-gui and run the test suite.

I reinstalled all of homebrew, but still get the above warnings and errors.

Using depends instead, I'm unable to build protobuf (see log). When I strip out protobuf, Boost fails in the same way as #13601 (log). However in this case xcode-select --install doesn't make the issue go away, and the other workaround /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg no longer exists. See also this dicussion.

Regarding the Qt fix, it's only available in 5.11 https://doc.qt.io/qt-5/qfontmetrics.html#horizontalAdvance so nothing can be done now unless we add conditional directive for Qt 5.11 - which wouldn't be tested.

Regarding objc_msgSend, a similar issue can be found at https://stackoverflow.com/a/25853741, apparently some compiler flag changed so either fix that or cast the function pointer accordingly to ...objc/message.h:

 56  * These functions must be cast to an appropriate function pointer type
 57  * before being called.

@promag

Regarding objc_msgSend, ... apparently some compiler flag changed...

Correct. See #16720.

@hebasto thanks, that worked! I was able to compile and sync the chain.

I just compiled and ran Bitcoin QT master (1c11e53bb604835afd838f4520d3fdba25521c5b) on the macOS Catalina release. Other than the annoying build warnings shown above, it seems to work.

make check is broken for me. Could be something specific to my machine, but making an issue anyway: https://github.com/bitcoin-core/secp256k1/issues/674

When I strip out protobuf, Boost fails in the same way as #13601 (log). However in this case xcode-select --install doesn't make the issue go away, and the other workaround /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg no longer exists. See also this dicussion.

This should be fixed with #17118.

I'm also still getting an error on Catalina (on 561a7d304).

Running test/util/bitcoin-util-test.py...
/usr/local/bin/python3.7 ../test/util/bitcoin-util-test.py
Running test/util/rpcauth-test.py...
/usr/local/bin/python3.7 ../test/util/rpcauth-test.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.018s

OK
Running bench/bench_bitcoin -evals=1 -scaling=0...
bench/bench_bitcoin -evals=1 -scaling=0 > /dev/null
  CC       src/tests-tests.o
  CCLD     tests
  CC       src/exhaustive_tests-tests_exhaustive.o
  CCLD     exhaustive_tests
/Library/Developer/CommandLineTools/usr/bin/make  check-TESTS
./build-aux/test-driver: line 107: 33356 Segmentation fault: 11  "$@" > $log_file 2>&1
FAIL: tests
PASS: exhaustive_tests
============================================================================
Testsuite summary for libsecp256k1 0.1
============================================================================
# TOTAL: 2
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See ./test-suite.log
============================================================================
make[6]: *** [test-suite.log] Error 1
make[5]: *** [check-TESTS] Error 2
make[4]: *** [check-am] Error 2
make[3]: *** [check-local] Error 2
make[2]: *** [check-am] Error 2
make[1]: *** [check-recursive] Error 1
make: *** [check-recursive] Error 1

@meeDamian that's the same issue I found. You can work around it with ./configure CFLAGS="-fno-stack-check" though I have no idea if that's safe. See https://github.com/bitcoin-core/secp256k1/issues/674

@meeDamian that's the same issue I found. You can work around it with ./configure CFLAGS="-fno-stack-check" though I have no idea if that's safe. See bitcoin-core/secp256k1#674

Can confirm it worked, and I was finally able to successfully build Bitcoind 😊. Thank you Sjors!

@Sjors , can this be closed? I noticed it was still on the 0.19.0 milestone.

./configure CFLAGS="-fno-stack-check" though I have no idea if that's safe

I might be wrong, but from what I understand it's not: that disables all stack checks. So also stack protector cookies. If someone were to find a buffer overflow in bitcoin core, that'd make it trivial to exploit and gain RCE. There haven't been many historically; some in UPNP, one in the SOCKS5 handling, but never in P2P network code. Still, we would never consider disabling that for a release.

Milestone can be removed, because we compile the release against an older SDK.

The safest workaround is probably to use the 10.14 SDK:

./configure CFLAGS=-mmacosx-version-min=10.14

This problem magically solved itself with Xcode 11.2 (as discovered by @fanquake in bitcoin-core/secp256k1#674.