scripts: Add MACHO dylib checks to symbol-check.py

fanquake commented at 1:48 am on January 4, 2020: member

Based on #17857.

This adds dynamic library checks for MACHO executables to symbol-check.py. The script has been modified to function more like security-check.py. The error output is now also slightly different. i.e:

 0# Linux x86
 1bitcoin-cli: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4
 2bitcoin-cli: export of symbol vtable for std::basic_ios<char, std::char_traits<char> > not allowed
 3bitcoin-cli: NEEDED library libstdc++.so.6 is not allowed
 4bitcoin-cli: failed IMPORTED_SYMBOLS EXPORTED_SYMBOLS LIBRARY_DEPENDENCIES
 5
 6# RISCV (skips exported symbols checks)
 7bitcoin-tx: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4
 8bitcoin-tx: NEEDED library libstdc++.so.6 is not allowed
 9bitcoin-tx: failed IMPORTED_SYMBOLS LIBRARY_DEPENDENCIES
10
11# macOS
12Checking macOS dynamic libraries...
13libboost_filesystem.dylib is not in ALLOWED_LIBRARIES!
14bitcoind: failed DYNAMIC_LIBRARIES

Compared to v0.19.0.1 the macOS allowed dylibs has been slimmed down somewhat:

 0 src/qt/bitcoin-qt:
 1 /usr/lib/libSystem.B.dylib 
 2-/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration 
 3 /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit 
 4 /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation 
 5 /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices 
 6 /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit 
 7 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices 
 8 /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation 
 9-/System/Library/Frameworks/Security.framework/Versions/A/Security 
10-/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration 
11 /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics 
12-/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL 
13-/System/Library/Frameworks/AGL.framework/Versions/A/AGL 
14 /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon 
15 /usr/lib/libc++.1.dylib 
16-/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork 
17 /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText 
18 /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO 
19 /usr/lib/libobjc.A.dylib

scripts: fix check-symbols & check-security argument passing

The first argument in bin_PROGRAMS (bitcoind) was being silently
dropped and never passed into the check-security.py or check-symbols.py scripts.

This has been the case since the scripts were added to the makefile in
https://github.com/bitcoin/bitcoin/commit/f3d3eaf78eb51238d799d8f20a585550d1567719.

Example of the behavior:

```python
# touch a, touch b, touch c
# python3 args.py < a b c

import sys
if __name__ == '__main__':
    print(sys.argv)
    # ['args.py', 'b', 'c']

    # if you add some lines to "a",
    # you'll see them here..
    for line in sys.stdin:
        print(line)
```

76bf97213f

fanquake added the label macOS on Jan 4, 2020

fanquake added the label Scripts and tools on Jan 4, 2020

fanquake added the label Needs gitian build on Jan 4, 2020

fanquake requested review from laanwj on Jan 4, 2020

scripts: add MACHO dylib checking to symbol-check.py c491368d8c

fanquake force-pushed on Jan 4, 2020

DrahtBot commented at 8:35 am on January 5, 2020: member

Gitian builds

File	commit 593f5e239f277e3fd917737122c2aed3c615430c(master)	commit 486ab1b76f1185cf6f8b0eb4d591c4a12e03c2a4(master and this pull)
bitcoin-0.19.99-aarch64-linux-gnu-debug.tar.gz	`d459a67160e0d3b2...`	`ea5bfa09352ccddd...`
bitcoin-0.19.99-aarch64-linux-gnu.tar.gz	`d1cee0bb7645f3fd...`	`8043a7ce79778249...`
bitcoin-0.19.99-arm-linux-gnueabihf-debug.tar.gz	`b44a6ead5f25cced...`	`a65d7f98714e89b7...`
bitcoin-0.19.99-arm-linux-gnueabihf.tar.gz	`733c3789b6618e44...`	`b0fc43eb386fc8da...`
bitcoin-0.19.99-i686-pc-linux-gnu-debug.tar.gz	`b69435c1039fd1b6...`	`493071e2bd99d544...`
bitcoin-0.19.99-i686-pc-linux-gnu.tar.gz	`6559d07f24d3365a...`	`b967fa3911768267...`
bitcoin-0.19.99-osx-unsigned.dmg	`932ecce4107c3972...`	`aaa57c32f1b334ad...`
bitcoin-0.19.99-osx64.tar.gz	`9643a190f0171411...`	`e0e3344ab56d6f37...`
bitcoin-0.19.99-riscv64-linux-gnu-debug.tar.gz	`48408407e1569b74...`	`299c855e4de33d6d...`
bitcoin-0.19.99-riscv64-linux-gnu.tar.gz	`4f4beca5efeba401...`	`bc22c9276cb1b11f...`
bitcoin-0.19.99-win64-debug.zip	`3332f1ff742870ba...`	`80d907e22325b123...`
bitcoin-0.19.99-win64-setup-unsigned.exe	`3038f679f63aaace...`	`e3ce39c654bb6766...`
bitcoin-0.19.99-win64.zip	`36164b6cccc3031e...`	`c576b877ca1947cd...`
bitcoin-0.19.99-x86_64-linux-gnu-debug.tar.gz	`35f4572db7b0c185...`	`cfc2d75ded34fa63...`
bitcoin-0.19.99-x86_64-linux-gnu.tar.gz	`7e2a547cc37ef980...`	`c80ebe14e2a1af2e...`
bitcoin-0.19.99.tar.gz	`3c30932da888c25f...`	`a7f2938e431d0246...`
bitcoin-core-linux-0.20-res.yml	`2d9d45c467e8164e...`	`1161dc8a1baf3d94...`
bitcoin-core-osx-0.20-res.yml	`5ed70c6cfd3b1161...`	`c44e04fae41c3221...`
bitcoin-core-win-0.20-res.yml	`db482b904e4e51e8...`	`9de825de9a924bc6...`
linux-build.log	`b5193d2b6e9cf133...`	`f70cf3e1cc7ea40f...`
osx-build.log	`190a3f15b2f45195...`	`9d7377da796f5c73...`
win-build.log	`471ed21e9305257d...`	`30a221d7d9f74808...`
bitcoin-core-linux-0.20-res.yml.diff		`cf9754928fa6b345...`
bitcoin-core-osx-0.20-res.yml.diff		`63b09fe6d5c51df2...`
bitcoin-core-win-0.20-res.yml.diff		`e34c56a8a24b7c50...`
linux-build.log.diff		`0ccb6d5a67c8b95a...`
osx-build.log.diff		`9928b012cf38b48e...`
win-build.log.diff		`973af9236ee0a38d...`

DrahtBot removed the label Needs gitian build on Jan 5, 2020

in contrib/devtools/symbol-check.py:186 in c491368d8c

182+
183+def check_exported_symbols(filename) -> bool:
184+    cppfilt = CPPFilt()
185+    ok = True
186+    for sym,version,arch in read_symbols(filename, False):
187+        if arch == 'RISC-V' or sym in IGNORE_EXPORTS:

laanwj commented at 11:05 am on January 13, 2020:

might want to check arch=='RISC-V' at the beginning of this function, instead of for every symbol

edit: oh, read_symbols returns a separate arch per symbol? ok, that’s strange, but now I understand this

laanwj commented at 7:33 pm on January 22, 2020: member

ACK c491368d8cfddf3a5b6d574f10ed67492fcecbed

I don’t think my code-organizational remark #17863#pullrequestreview-341769161 is enough to hold this up, can always refactor this later.

laanwj referenced this in commit a3d198c93c on Jan 22, 2020

laanwj merged this on Jan 22, 2020

laanwj closed this on Jan 22, 2020

fanquake deleted the branch on Jan 23, 2020

sidhujag referenced this in commit 67671cbe66 on Jan 24, 2020

sidhujag referenced this in commit 4393ded651 on Nov 10, 2020

DrahtBot locked this on Feb 15, 2022

scripts: Add MACHO dylib checks to symbol-check.py #17863

Gitian builds