Try also linking without -pthread and do not use it when not needed (clang on OS X)

paveljanik commented at 10:59 am on December 20, 2014: contributor

Labels: Build system, Mac, Priority Low clang on OS X doesn’t want -pthread option when linking and thus emits this on link phase:

0clang: warning: argument unused during compilation: '-pthread'

Our configure is using http://www.gnu.org/software/autoconf-archive/ax_pthread.html for pthread options detection but it doesn’t (yet?) try to run clang without -pthread in the current version (compare Savannah patch http://savannah.gnu.org/patch/?8186) on OS X. This PR adds this and configure thus tries it. After this change the configure output changes like this (diff against previous configure log):

0-checking whether pthreads work with -pthread... yes
1+checking whether pthreads work without any flags... yes

and the actual build log does not contain the above mentioned warnings at all.

The other possible approach to this is removing the darwin special case completely. This leads to configure trying -lpthreads before no options without success and ending in no options. Ie. the result is almost the same, but this way is safe (we can’t know other/old environments and someone could have libpthreads doing something else). @theuni please have a look.

jgarzik commented at 1:23 pm on December 20, 2014: contributor

“-pthreads” is a compiler and linker flag. On some platforms it enables various macros in C/C++ headers. It is the “I am using threads” command line switch.

As such, a poor test can silently fail (== appear to not need -pthread), resulting in code that is compiled as single-threaded code.

Silently compiling to single-threaded code is a far worse outcome than an annoying warning on compiler+platform combinations where -pthread is superfluous.

paveljanik commented at 2:30 pm on December 20, 2014: contributor

@jgarzik “-pthread” is a compiler and linker flag: no. This only applies to good old GNU gcc and GNU linker world only (compare pthreads(7) on GNU/Linux: On Linux, programs that use the Pthreads API should be compiled using cc -pthread.). But it is a bit different on OS X. You don’t need any compiler/linker flags because all system libs contain threads support. See https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/pthread.3.html#//apple_ref/doc/man/3/pthread for more details - especially the section INSTALLATION: “The default system libraries include pthread functions. No additional libraries or CFLAGS are necessary to use these interfaces.” “Silent compilation as single thread” is thus not possible on OS X and this PR is Darwin/OS X only anyway.

paveljanik force-pushed on Dec 20, 2014

jgarzik commented at 3:36 pm on December 20, 2014: contributor

@paveljanik I am well aware of that. My comment remains applicable. You must consider the case of building with gcc + third party libraries on OSX that trigger different thread/non-thread behavior at compile time.

The entire app stack on OSX needs to be verified free of MT behavior switching in headers, not just the system libs. e.g. Boost in particular is very sensitive on OSX, and is header-heavy.

paveljanik commented at 7:20 pm on December 20, 2014: contributor

But this is “solved” in ax_pthread.m4 already IMO, where it checks thread functionality (not completely though, of course) with the provided compiler/libs and auto-selects the needed flags. When I think more about it, ax_pthread.m4 macros should be changed to distinguish between compile time and linking time. Of course written for GNU world, where compile and link time are the same…

On OS X:

0Pavels-MacBook-Pro:tmp pavel$ gcc -pthread conftest.c

but:

0Pavels-MacBook-Pro:tmp pavel$ gcc -pthread -o conftest.o -c conftest.c
1Pavels-MacBook-Pro:tmp pavel$ gcc -pthread conftest.o
2clang: warning: argument unused during compilation: '-pthread'
3Pavels-MacBook-Pro:tmp pavel$

Ie. when clang is used to compile&link, it doesn’t emit warning. Compile alone is OK but link alone emits warning. I think this doesn’t have a solution right now and I should close this PR and report the problem to ax_pthread.m4 authors. Other opinions?

theuni commented at 6:51 pm on December 22, 2014: member

 0diff --git a/src/Makefile.am b/src/Makefile.am
 1index d6ac6e1..d7968bd 100644
 2--- a/src/Makefile.am
 3+++ b/src/Makefile.am
 4@@ -1,5 +1,6 @@
 5 DIST_SUBDIRS = secp256k1
 6-AM_LDFLAGS = $(PTHREAD_CFLAGS) $(LIBTOOL_LDFLAGS)
 7+AM_LDFLAGS = $(PTHREAD_LIBS) $(LIBTOOL_LDFLAGS)
 8+AM_CFLAGS = $(PTHREAD_CFLAGS)
 9
10
11 if EMBEDDED_LEVELDB

Isn’t that all that needs to be done? The m4 macro already does all kinds of checking for us, we just weren’t using the output correctly. Clang is perfectly happy to eat “-pthreads” for compile, it just doesn’t like it in the link-line.

paveljanik commented at 7:25 pm on December 22, 2014: contributor

@theuni yes, this is all to be done. Thank you! Will you PR it or should I use it here?

jgarzik commented at 7:29 pm on December 22, 2014: contributor

+1 @theuni

theuni commented at 9:43 pm on December 22, 2014: member

@paveljanik You’re welcome to just take that here, to keep the discussion in one place.

Use correct compile and link time options for pthreads. 86d295e2bd

paveljanik force-pushed on Dec 22, 2014

paveljanik commented at 10:12 pm on December 22, 2014: contributor

Done. Thank you again.

Print configured PTHREAD_* values (for easier debugging) e7f114cd87

paveljanik commented at 11:09 pm on December 22, 2014: contributor

Travis failed: https://travis-ci.org/bitcoin/bitcoin/jobs/44878072 on ARM.

Looks like -pthread is missing when linking or something similar… But can’t test easily. Adding debugging prints to configure.

paveljanik commented at 11:12 pm on December 22, 2014: contributor

Yup, confirmed:

0configure: The contents of configured PTHREAD_CFLAGS: -pthread
1configure: The contents of configured PTHREAD_LIBS: 
2configure: The contents of configured PTHREAD_CC: arm-linux-gnueabihf-gcc

paveljanik commented at 11:18 pm on December 22, 2014: contributor

@theuni Any hint now?

paveljanik commented at 11:37 pm on December 22, 2014: contributor

Other platforms are OK. On all of them, _LIBS is empty, e.g.:

0configure: The contents of configured PTHREAD_CFLAGS: -pthread
1configure: The contents of configured PTHREAD_LIBS: 
2configure: The contents of configured PTHREAD_CC: gcc -m64

theuni commented at 0:31 am on December 23, 2014: member

Well that’s annoying. I had assumed that the m4 would treat LIBS as LDFLAGS, sticking linker options (-pthread) in LIBS as needed. But upon closer inspection, it doesn’t even try to compile without linking! It just assumes you’ll forward CFLAGS to the linker, which explains why I had it setup that way to begin with.

After trying a few things and checking deep in some docs, I now agree with @jgarzik. Your original suggestion was probably the correct thing to do, but it’s just not worth doing for us. I’d rather leave a harmless warning around than risk trouble in the future.

paveljanik commented at 9:45 am on December 23, 2014: contributor

See https://savannah.gnu.org/patch/index.php?8585 for a bugreport to GNU Autoconf Archive.

Is https://github.com/paveljanik/bitcoin/commit/e7f114cd8787cd0c107a7519b5ae143748e93af9 worth cherrypicking so we have auto-detected values saved in the Travis logs/configure log?

Otherwise we can’t do more here. Thank you all. Closing.

paveljanik closed this on Dec 23, 2014

paveljanik commented at 9:10 pm on February 10, 2016: contributor

JFYI: ax_pthreads.m4 is getting fixed upstream:

https://savannah.gnu.org/patch/?8585

MarcoFalke locked this on Sep 8, 2021

Try also linking without -pthread and do not use it when not needed (clang on OS X) #5516