Gitian crash for ARM compiling BDB depends (Bionic host)

Sjors commented at 8:02 am on July 18, 2018: member

I was trying a Gitian build for master at 9cdb19fe6717b1003e01d997c096fb4ab534aa03? I used LXC (not Docker), but then crashes during depends make with bdb:

 0libtool: compile:  arm-linux-gnueabihf-g++ -c -I. -I../dist/./.. -I/home/ubuntu/build/bitcoin/depends/arm-linux-gnueabihf/include -D_GNU_SOURCE -D_REENTRANT -std=c++11 ../dist/./../cxx/cxx_multi.cpp  -fPIC -DPIC -o cxx_multi.o
 1libtool: compile:  arm-linux-gnueabihf-g++ -c -I. -I../dist/./.. -I/home/ubuntu/build/bitcoin/depends/arm-linux-gnueabihf/include -D_GNU_SOURCE -D_REENTRANT -std=c++11 ../dist/./../cxx/cxx_mpool.cpp  -fPIC -DPIC -o cxx_mpool.o
 2In file included from ../dist/./../cxx/cxx_mpool.cpp:1:0:
 3
 4
 5Internal compiler error: Error reporting routines re-entered.
 6./libtool --mode=compile arm-linux-gnueabihf-g++ -c -I. -I../dist/./.. -I/home/ubuntu/build/bitcoin/depends/arm-linux-gnueabihf/include     -D_GNU_SOURCE -D_REENTRANT -std=c++11     ../dist/./../cxx/cxx_seq.cpp
 7Please submit a full bug report,
 8with preprocessed source if appropriate.
 9See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
10Makefile:2032: recipe for target 'cxx_mpool.o' failed
11make[1]: *** [cxx_mpool.o] Error 1
12make[1]: *** Waiting for unfinished jobs....
13libtool: compile:  arm-linux-gnueabihf-g++ -c -I. -I../dist/./.. -I/home/ubuntu/build/bitcoin/depends/arm-linux-gnueabihf/include -D_GNU_SOURCE -D_REENTRANT -std=c++11 ../dist/./../cxx/cxx_seq.cpp  -fPIC -DPIC -o cxx_seq.o
14make[1]: Leaving directory '/home/ubuntu/build/bitcoin/depends/work/build/arm-linux-gnueabihf/bdb/4.8.30-8f2a4f0c933/build_unix'
15funcs.mk:242: recipe for target '/home/ubuntu/build/bitcoin/depends/work/build/arm-linux-gnueabihf/bdb/4.8.30-8f2a4f0c933/build_unix/.stamp_built' failed
16make: *** [/home/ubuntu/build/bitcoin/depends/work/build/arm-linux-gnueabihf/bdb/4.8.30-8f2a4f0c933/build_unix/.stamp_built] Error 2

Gitian worked when I tested an earlier version of #13604, on a Debian 9 host and before Bionic descriptors were merged. Perhaps the changes added before merge cause a problem, or perhaps there’s something about the Bionic descriptors (or host).

fanquake added the label Build system on Jul 18, 2018

ken2812221 commented at 8:50 am on July 18, 2018: contributor

I didn’t see this issue on Bionic host. Did you try build it again?

Sjors commented at 9:05 am on July 18, 2018: member

@ken2812221 I’m currently trying to build on the Bionic host directly. If that works, I’ll try Gitian again. Note that #13604 was merged after your #13171.

ken2812221 commented at 10:21 am on July 18, 2018: contributor

@Sjors I just did gitian build for master a few minutes ago, didn’t see any issue.

Sjors commented at 12:06 pm on July 18, 2018: member

I was able to cross-compile on the Bionic host (and the QT binary works on the device).

I ran Gitian again:

0python3 gitian-build.py --detach-sign --no-commit -b sjors -c 9cdb19fe6717b1003e01d997c096fb4ab534aa03 -j5

This time it’s happy.

Maybe it was just a fluke. However I added a bunch of dependencies on the host machine while writing #13702, so maybe that matters? g++-arm-linux-gnueabihf g++-aarch64-linux-gnu binutils-arm-linux-gnueabihf binutils-aarch64-linux-gnu g++-4.8-multilib gcc-4.8-multilib binutils-gold

Sjors commented at 1:27 pm on July 18, 2018: member

File checksums for 9cdb19f:

 0a48eb658c8b64ec68ce00388d7143c50e19715bb37b4bf150c0b3d6188663dd2  bitcoin-0.16.99-aarch64-linux-gnu-debug.tar.gz
 1794dd81be553fe37ea12e419170e4f37dac2956218f1427776965b187edaf76a  bitcoin-0.16.99-aarch64-linux-gnu.tar.gz
 2b5a71e7edf8673ad4347c50decfac8a2e3eb01c94e913f9e526edca40b41ec19  bitcoin-0.16.99-arm-linux-gnueabihf-debug.tar.gz
 35acc336f94331de16069890caa2d537ac27b10e6b41547c32c177589cc6cf7c4  bitcoin-0.16.99-arm-linux-gnueabihf.tar.gz
 4ac000fd5682bc9b8c7015ff45e6f9087b8c641185c2925edbee844f7d03ad0d8  bitcoin-0.16.99-i686-pc-linux-gnu-debug.tar.gz
 5d77bc0433d385ed2158029c12a4e4c90dccd0ff2c1c0653f4b037fd9f9ce7fbf  bitcoin-0.16.99-i686-pc-linux-gnu.tar.gz
 64f3d3d231c6ec3707125f4b57f3f8e599918883be6262354d29d49240110c92b  bitcoin-0.16.99-x86_64-linux-gnu-debug.tar.gz
 7fafaa884c1ab81eefed912684e6e2d4f7f7d8b2060aecb745e1e0d1c8f88c03d  bitcoin-0.16.99-x86_64-linux-gnu.tar.gz
 818f9d77399bb62b19ee58b22e667a6e327f8afad6c413244d9b19931fb03d9f7  src/bitcoin-0.16.99.tar.gz
 9d0c4d472f3f48fe4e6b5d2a407de5df3fee2dacffe3cba4532a08589b3b7a3db  bitcoin-linux-0.17-res.yml
10
11eef3f7d3135ec7f87370326c39f68f47b158e6f209a352b3c12af54a1cf44310  bitcoin-0.16.99-win-unsigned.tar.gz
12b1bab4827403cedd6005507234d94e5e9f0e9b9c16522d8a78df75585024fdaf  bitcoin-0.16.99-win32-debug.zip
13a7e0557b4f8490397a0835da0d0f76d801872804c3075a32baa4a27e73bc4038  bitcoin-0.16.99-win32-setup-unsigned.exe
1432b27d42c4abc2a2fd1c3c8f00f9d03ca15a7cc01ec83c7ccb4b394723abc84d  bitcoin-0.16.99-win32.zip
15e0ab4d9f85f30c0a757f22f98622be8f483e3fe06c8430af3ef2bf4bc1d2774e  bitcoin-0.16.99-win64-debug.zip
164a1eecb701594b24938da7da7a16243ca5653747383df52460b606d6717a9cbf  bitcoin-0.16.99-win64-setup-unsigned.exe
1725c90b9942493e5fb0bd61bcaf0ee96f73682066fadb6ad82e0966f2bfe21e14  bitcoin-0.16.99-win64.zip
1818f9d77399bb62b19ee58b22e667a6e327f8afad6c413244d9b19931fb03d9f7  src/bitcoin-0.16.99.tar.gz
19f853837ddbefb28d4c10e6adcf934840ab3abc553851ab71f01527b9682a99ab  bitcoin-win-0.17-res.yml
20
216113a75598aa0055c50651bbec589e57a3a0e7b880e0cf0d7a87e1a8ffed12ca  bitcoin-0.16.99-osx-unsigned.dmg
22f7968dd5a69168004ec854d0ed45f10ca703a7947cacdbc613fd1703d912267b  bitcoin-0.16.99-osx-unsigned.tar.gz
2307cc9b2de899fde2868109655ab1f35c13279c0b2ee4747e2fd7880124d1c6bf  bitcoin-0.16.99-osx64.tar.gz
2418f9d77399bb62b19ee58b22e667a6e327f8afad6c413244d9b19931fb03d9f7  src/bitcoin-0.16.99.tar.gz
2514decafee5700d7d4b5b1d442d7fbd5e1a71dea7d5d2b3b7c22ed046976e0c6d  bitcoin-osx-0.17-res.yml

laanwj commented at 3:45 pm on July 18, 2018: member

If it isn’t a cosmic ray or CPU overheating it’s a compiler bug.

theuni commented at 8:53 pm on July 18, 2018: member

GCC ICE’s in Gitian have historically been due to running out of mem. Maybe you’re running too many parallel jobs?

Sjors commented at 9:10 am on July 19, 2018: member

-j5 on a VM with 10 GB RAM and 4 vCPU’s. Happy to blame the cosmic ray if nobody else runs into it on a fresh machine.

jonasschnelli commented at 3:52 pm on July 30, 2018: contributor

Don’t you also need the --memory <4000 or something> for gbuild? Seems to be very likely OOM.

MarcoFalke commented at 5:08 pm on July 30, 2018: member

Closing for now. You can ask me to reopen, if this happens again.

MarcoFalke closed this on Jul 30, 2018

MarcoFalke locked this on Sep 8, 2021

Gitian crash for ARM compiling BDB depends (Bionic host) #13700