build: fix building libconsensus with reduced exports for Darwin targets #19522

pull fanquake wants to merge 6 commits into bitcoin:master from fanquake:libconsensus_visibility_clang changing 3 files +26 −246
  1. fanquake commented at 2:20 am on July 15, 2020: member

    Darwin targets do not have a protected visibility function attribute, see LLVM explanation. This means that the AX_GCC_FUNC_ATTRIBUTE check for visibility fails:

    0configure:24513: checking for __attribute__((visibility))
    1configure:24537: g++ -std=c++11 -o conftest -g -O2  -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS -DMAC_OSX -DOBJC_OLD_DISPATCH_PROTOTYPES=0  -Wl,-headerpad_max_install_names conftest.cpp  >&5
    2conftest.cpp:35:56: warning: target does not support 'protected' visibility; using 'default' [-Wunsupported-visibility]
    3                    int foo_pro( void ) __attribute__((visibility("protected")));
    4                                                       ^
    51 warning generated.
    6configure:24537: $? = 0
    7configure:24550: result: no
    

    This leads to EXPORT_SYMBOL being defined to nothing, as HAVE_FUNC_ATTRIBUTE_VISIBILITY is not defined, and when building with reduced exports, you end up with a libbitcoinconsensus.dylib that doesn’t export any _bitcoinconsensus_* symbols.

    0➜  git:(master) nm -C src/.libs/libbitcoinconsensus.dylib | rg _bitcoinconsensus_
    1➜  git:(master)
    

    We do have a second check for the visibility attribute, which works for Darwin as it’s only testing for default visibility, however the result of this check isn’t used at all. It was added in #4725, along with the --enable-reduce-exports option, however when libbitcoinconsensus was added in #5235, it used the results of the added AX_GCC_FUNC_ATTRIBUTE calls.

    This PR removes our usage of the AX_GCC_FUNC_ATTRIBUTE macro entirely, in favour of our own checks in configure. This meant adding a check for dllexport, which I’ve tested as working with both GCC and Clang when building for Windows. I haven’t added an equivalent check for dllimport, as we weren’t actually using the result of that check, we’re just testing that MSC_VER was defined before using.

    With these changes building a libbitcoinconsensus with reduced exports, when targeting Darwin, works as expected:

    0./autogen.sh
    1./configure --disable-tests --disable-bench --with-utils=no --with-daemon=no --with-gui=no --disable-wallet --with-libs=yes --enable-reduce-exports
    2make -j8
    3...
    4nm -C src/.libs/libbitcoinconsensus.dylib | rg _bitcoinconsensus_
    5000000000000a340 T _bitcoinconsensus_verify_script
    600000000000097e0 T _bitcoinconsensus_verify_script_with_amount
    7000000000000a3c0 T _bitcoinconsensus_version
    
    0>>> import ctypes
    1>>> consensus = ctypes.CDLL("src/.libs/libbitcoinconsensus.dylib")
    2>>> print(consensus.bitcoinconsensus_version())
    31
    4>>> exit()
    

    TODO: Modify a CI job to compile with –enable-reduce-exports and check for symbols in shared lib?

  2. fanquake added the label Build system on Jul 15, 2020
  3. fanquake added the label Needs gitian build on Jul 15, 2020
  4. fanquake added the label Needs Guix build on Jul 15, 2020
  5. fanquake commented at 2:59 am on July 15, 2020: member

    A PR is never truly open until a moment of realization right after hitting the big green button.

    This is actually something I broke in #7711. Updating the AX_GCC_FUNC_ATTRIBUTE macro reintroduced the checks for protected and internal, which had been removed in ee64c53c1fdcadf00138b7e6fdf44a1c433db9f8. However I take it for the past 4 years, no-one has built and used a libbitcoinconsensus for a Darwin target, with --enable-reduced-exports.

    The alternative approach here would be to re-remove the two visibility attributes we don’t care about from the AX_GCC_FUNC_ATTRIBUTE macro (rather than 7a600331a84fa7da0759f89deb309e60e5040773, 1381e9c8d327a86cbb5d2405d24b7ddd5b26fcd5 and 01bc0b6ca2b7b2d2b384f667e2abe7f1a7b9c8a5), noting that we’ve modified it from upstream, and then remove the unused visibility check from #4725. In that case, the remaining changes would still be relevant improvements.

    Big anti-props me.

  6. DrahtBot commented at 5:43 am on July 15, 2020: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #20487 (draft: Add syscall sandboxing using seccomp-bpf (Linux secure computing mode) by practicalswift)
    • #16546 (External signer support - Wallet Box edition by Sjors)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  7. DrahtBot commented at 8:07 pm on July 15, 2020: member

    Guix builds

    File commit f4de89edfa8be4501534fec0c662c650a4ce7ef2(master) commit 65ee660f071caa8ad0d5831c53bd8cd37bb6db89(master and this pull)
    *-aarch64-linux-gnu-debug.tar.gz 08cfaea788a07ceb... 363d65bf69cb4ed3...
    *-aarch64-linux-gnu.tar.gz f22c76019e0ccd7f... 9f4e6b10e84a4742...
    *-arm-linux-gnueabihf-debug.tar.gz 60a5d1855b55f0a8... fd5e5127f9a70f72...
    *-arm-linux-gnueabihf.tar.gz ebd8c0e33838e1fa... 4ad3bbead17991e9...
    *-riscv64-linux-gnu-debug.tar.gz 5bdbfcff97558737... 5fcd68e92c9ab05a...
    *-riscv64-linux-gnu.tar.gz a826ce80c8c40316... 93ad8954a8eb6852...
    *-win-unsigned.tar.gz f5adfa1b298f3ef6... 6f8d30b0d88ff7ba...
    *-win64-debug.zip 6c8a495cf2d815cf... 053acb2b03e6c22c...
    *-win64-setup-unsigned.exe 3c1135852587c8d0... 84b2797d2fadbd41...
    *-win64.zip 1e6c4a7013557b19... 160133580ec04756...
    *-x86_64-linux-gnu-debug.tar.gz b2f27029c311cc3b... 89a0fd504956290f...
    *-x86_64-linux-gnu.tar.gz e182a4c29c6961d0... c6d49aa87cc67fcb...
    *.tar.gz f142bfa8f69d3205... 10f65d5c9df2a708...
    guix_build.log 49d4490a44c4a052... 936b426e8678d14c...
    guix_build.log.diff ca1fd062c0261430...
  8. DrahtBot removed the label Needs Guix build on Jul 15, 2020
  9. DrahtBot commented at 6:57 pm on July 16, 2020: member

    Gitian builds

    File commit 3864219d4074d289799634378d85cccbcc2e6e56(master) commit d401e95bea488a51f571c44089014e94c6fe9d56(master and this pull)
    *-aarch64-linux-gnu-debug.tar.gz 234341edaaa03287... 1f56f09dfb88aaf3...
    *-aarch64-linux-gnu.tar.gz fe6c16d320115507... a6a1427684d33416...
    *-arm-linux-gnueabihf-debug.tar.gz 42a6b4a487d81b8f... 76c65185b837c05f...
    *-arm-linux-gnueabihf.tar.gz a47bf9a033950563... e4d223079b9f7e4c...
    *-osx-unsigned.dmg 6b39d6bb9fcba41e... 254fea74e349e72f...
    *-osx64.tar.gz 9de75640b54a62ad... f2a081a0f3f262f2...
    *-riscv64-linux-gnu-debug.tar.gz f2b02a01562d8c1e... 46146d5758fa1531...
    *-riscv64-linux-gnu.tar.gz 0270bb629d738fc0... ffb7f90a465edf7e...
    *-win64-debug.zip 38645188383170e2... 0db1f1e01357801b...
    *-win64-setup-unsigned.exe 273a380bf2f19bf4... bb3ad193464dd190...
    *-win64.zip bf93c8f5d2457a57... 36891f6dc2ea9fb5...
    *-x86_64-linux-gnu-debug.tar.gz 2c2a010fb9114dd5... 6d543a9d3ecca3ee...
    *-x86_64-linux-gnu.tar.gz 8b953c29cccb10d7... 655695eea728b52b...
    *.tar.gz 17eb6dc576244bd1... d07b584351d2cde9...
    bitcoin-core-linux-0.21-res.yml 8a85867b0f5145d2... 64c59fba43184855...
    bitcoin-core-osx-0.21-res.yml e6a4c03cdb1fa972... 1a37785727ed9201...
    bitcoin-core-win-0.21-res.yml 5e5f75dcf64ee726... 446e2e48863cefa4...
    linux-build.log 8747037a8d02910b... 1bdfecc908d058c3...
    osx-build.log 03a1f2575139de06... e64a3f107c6f9f35...
    win-build.log 54377010d59fa536... 68136fbbfe3df63c...
    bitcoin-core-linux-0.21-res.yml.diff d0bb7fce873c59ef...
    bitcoin-core-osx-0.21-res.yml.diff cc1f8f928b74e353...
    bitcoin-core-win-0.21-res.yml.diff 73ee2a3c66b5b403...
    linux-build.log.diff 795ad3d3b77eeffe...
    osx-build.log.diff 0483109ffe3f6558...
    win-build.log.diff 247cd0f8744e81c8...
  10. DrahtBot removed the label Needs gitian build on Jul 16, 2020
  11. fanquake force-pushed on Jul 21, 2020
  12. fanquake commented at 5:15 am on July 21, 2020: member
    Consensus from the latest build meeting was that removing the macro would be ok, so I’m going to leave this with the current approach. I’ve also modified the last commit to re-split the RE_CXXFLAGS, and conditionally output them and the RELDFLAGS at the end of configure.
  13. theuni commented at 7:24 pm on September 4, 2020: member

    Concept ACK, changes look good at a glance.

    Thanks for fixing this. The attribute check has been broken for years.

    Any reason for not just appending to CXXFLAGS/LDFLAGS for the configure output? I’m not sure that RE_FOO would anything to anyone other than reviewers of this pull request :)

  14. fanquake force-pushed on Sep 11, 2020
  15. fanquake commented at 6:33 am on September 11, 2020: member

    Any reason for not just appending to CXXFLAGS/LDFLAGS for the configure output? I’m not sure that RE_FOO would anything to anyone other than reviewers of this pull request :)

    Yea fair call. I’ve dropped the RE specifics and have added to the CXX and LD flags output.

  16. DrahtBot added the label Needs rebase on Sep 15, 2020
  17. DrahtBot commented at 11:39 am on September 15, 2020: member

    🐙 This pull request conflicts with the target branch and needs rebase.

    Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a “draft”.

  18. fanquake force-pushed on Sep 15, 2020
  19. fanquake removed the label Needs rebase on Sep 15, 2020
  20. in src/script/bitcoinconsensus.h:15 in 8959aec04a outdated
    11@@ -12,13 +12,13 @@
    12 #include <config/bitcoin-config.h>
    13   #if defined(_WIN32)
    14     #if defined(DLL_EXPORT)
    15-      #if defined(HAVE_FUNC_ATTRIBUTE_DLLEXPORT)
    16+      #if defined(HAVE_DLLEXPORT_ATTRIBUTE)
    


    laanwj commented at 7:35 am on October 7, 2020:
    We could also always assume __declspec(dllexport) to be present on windows? It seems something is very wrong with the compiler if it is not. I mean: two checks make sense here: test for HAVE_DLLEXPORT_ATTRIBUTE or test for _WIN32. Both is overkill.

    theuni commented at 8:32 pm on October 8, 2020:

    We could also always assume __declspec(dllexport) to be present on windows? It seems something is very wrong with the compiler if it is not.

    Agree. Also, this is our internal build, so we’re generally aware of what we support, and we wouldn’t support such a broken compiler.


    theuni commented at 8:37 pm on October 8, 2020:

    In that case, the dllexport can be moved to a Windows-only check in configure, which errors if it fails.

    Edit: Windows-only, and only if building the lib.

  21. build: remove duplicate visibility attribute detection
    We are already testing for this, and our test works correctly with a Darwin
    target, where the macro does not. Darwin targets do not support "protected"
    visibility.
    1624e17b54
  22. build: test for __declspec(dllexport) in configure
    This should work for GCC and Clang when building for Windows targets.
    7cd0a69664
  23. build: remove AX_GCC_FUNC_ATTRIBUTE test for dllimport
    The result of this test isn't currently used anywhere (we use dllimport based on
    MSC_VER in libconsensus).
    f054a089ec
  24. build: remove ax_gcc_func_attribute macro
    This is no-longer used.
    8f360e349e
  25. build: add building libconsensus to end-of-configure output 012bdec1b7
  26. fanquake force-pushed on Feb 12, 2021
  27. build: consolidate reduced export checks de4238f92f
  28. fanquake force-pushed on Feb 12, 2021
  29. laanwj commented at 10:11 am on February 12, 2021: member
    Code review ACK de4238f92f4c067f099663f68d9772105de81d75 nice diff +26 −246
  30. laanwj merged this on Feb 12, 2021
  31. laanwj closed this on Feb 12, 2021

  32. fanquake deleted the branch on Feb 12, 2021
  33. sidhujag referenced this in commit cd8f951240 on Feb 12, 2021
  34. DrahtBot locked this on Aug 16, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-05 19:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me