Parallel build does not work on OS X #7834

I'm building via a MacPorts portfile, but it's no difference to building via the command line. Here's the full log.

https://cloudup.com/co19esAfMhG

Note that a non-parallel build works just fine.

I'm not familiar with the MacPorts portfile parallel building. But a simple ./autogen, ./configure, make -j5 works for me.

You build log looks after a upstream issue.

I can reproduce something similar on master:

bitcoin-master$ make clean >/dev/null 2>&1; make -j8 -w all appbundle 2>&1 | grep error
  CXX      script/libbitcoinconsensus_la-script_error.lo
  CXX      script/libbitcoin_consensus_a-script_error.o
libtool:   error: 'lib/libunivalue_la-univalue_read.lo' is not a valid libtool object
bitcoin-master$ 

Well, the error for me is rather mysterious and odd.

:error:build Failed to build bitcoin: command execution failed
:debug:build Error code: CHILDSTATUS 24589 2
:debug:build Backtrace: command execution failed

What does that even mean?

I had never seen that before. Very likely something from MacPorts.

Two lines before this:

:info:build Exit code: 2
:error:build Failed to build bitcoin: command execution failed

Ie. make failed with error code 2. This is the same for me now on master now....

bitcoin-master pavel$ make clean >/dev/null 2>&1; make -j8 -w all appbundle >/dev/null 2>&1; echo $?
2
bitcoin-master pavel$ 

Without appbundle, it returns 0 here.

Interesting. So it seems the appbundle target is the non-parallelisable bit.

On 7 Apr 2016, at 20:03, paveljanik notifications@github.com wrote:

Two lines before this:

:info:build Exit code: 2 :error:build Failed to build bitcoin: command execution failed Ie. make failed with error code 2. This is the same for me now on master now....

bitcoin-master pavel$ make clean >/dev/null 2>&1; make -j8 -w all appbundle >/dev/null 2>&1; echo $? 2 bitcoin-master pavel$ Without appbundle, it returns 0 here.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub #7834 (comment)

The parallel build seems to be non-deterministically succeeding/failing.

On 7 Apr 2016, at 20:03, paveljanik notifications@github.com wrote:

Two lines before this:

:info:build Exit code: 2 :error:build Failed to build bitcoin: command execution failed Ie. make failed with error code 2. This is the same for me now on master now....

bitcoin-master pavel$ make clean >/dev/null 2>&1; make -j8 -w all appbundle >/dev/null 2>&1; echo $? 2 bitcoin-master pavel$ Without appbundle, it returns 0 here.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub #7834 (comment)

Looks like univalue is not parallel build safe when run from top level makefile with appbundle target. But it alone is parallel build safe here... @theuni Do you have any idea?

Interesting. I'm guessing we have overlapping recursive builds. I don't see anything that would lead to that though. @paveljanik: do you see the same problem running make -C src qt/bitcoin-qt from the top level?

All this said, I’ve now gotten parallel builds running fine on OS X 10.11, using my MacPorts Portfile. Not sure exactly what I did though.

On 8 Apr 2016, at 17:37, Cory Fields notifications@github.com wrote:

Interesting. I'm guessing we have overlapping recursive builds. I don't see anything that would lead to that though.

@paveljanik https://github.com/paveljanik: do you see the same problem running make -C src qt/bitcoin-qt from the top level?

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub #7834 (comment)

Ok, I see. I assumed that "-w all" was some make option for "warn all" that i didn't know. It's not..

"-w" means "show curdir", and "all" is a target.

So you're building with 2 targets: all and appbundle. In that case, yes, the recursive builds clash. There's not an easy fix for the problem because the command itself kinda doesn't make sense.

To fix, just drop the "all".

Ah 8) Thanks for explanation :-)

I just took a look at the macports file. Change: build.args appbundle to: build.target appbundle

That should allow parallel build to be re-enabled. Edit: It should also build much quicker.

Looks like the changes to the MacPorts file were made here. Parallel build working and enabled. Can this issue now be closed?

I don't know - can anyone try a parallel build (of master) on mac? I can't test this.

Confirmed parallel builds of master work fine on OS X.