Undefined Symbols on MacOS 10.9 Mavericks from db.o and util.o #3228

I'm running MacOS 10.9 (13A598) with the latest Boost, BerkeleyDB, and libqrencode installed via MacPorts. Compilation goes fine. But linking seems to fail due to undefined symbols in util.o and db.o.

See full output here: https://gist.github.com/gthiruva/10d8b1e46e4598ca1328

Error output: Undefined symbols for architecture x86_64: "Db::verify(char const_, char const_, std::ostream_, unsigned int)", referenced from: CDBEnv::Verify(std::string, bool (_)(CDBEnv&, std::string)) in db.o CDBEnv::Salvage(std::string, bool, std::vector<std::pair<std::vector<unsigned char, std::allocator<unsigned char> >, std::vector<unsigned char, std::allocator<unsigned char> > >, std::allocator<std::pair<std::vector<unsigned char, std::allocator<unsigned char> >, std::vector<unsigned char, std::allocator<unsigned char> > > > >&) in db.o "boost::program_options::to_internal(std::string const&)", referenced from: boost::program_options::detail::basic_config_file_iterator<char>::getline(std::string&) in util.o "boost::program_options::detail::common_config_file_iterator::common_config_file_iterator(std::set<std::string, std::lessstd::string, std::allocatorstd::string > const&, bool)", referenced from: boost::program_options::detail::basic_config_file_iterator<char>::basic_config_file_iterator(std::istream&, std::set<std::string, std::lessstd::string, std::allocatorstd::string > const&, bool) in util.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt] Error 1

@gthiruva Which version of Boost are you using, 1.54.0 ?

Did you do a make clean / and or make sure all the object files were gone before building?

@fanquake Yep, 1.50.0 from macports:

The following ports are currently installed:
  boost [@1](/bitcoin-bitcoin/contributor/1/).54.0_0+no_single+no_static+python27 (active)

@laanwj And, yes. make clean and even make distclean.

this seems to be an issue with boost and clang on os x:

• http://stackoverflow.com/questions/18888327/using-boost-on-xcode-5-apple-llvm-5-0
• http://stackoverflow.com/questions/11081818/linking-troubles-with-boostprogram-options-on-osx-using-llvm
• https://svn.boost.org/trac/boost/ticket/7473
• #3228

you can probably fix it by changing the compiler to g++, i.e. adding these to qmake when generating the makefile: QMAKE_CC=gcc QMAKE_CXX=g++

I don't see any of our new interim libs being linked in. Am I to assume this is a pre-autotools build?

If so, it's not really relevant since we don't build this way anymore.

I found out how to solve that problem. OSX 10.9 change the default stdlib to libc++. You may need to compile with libstdc++ to prevent some errors. At the same time, boost should be compiled use a cxxflag '-stdlib=libstdc++'( eg: ./b2 cxxflags="-stdlib=libstdc++" ...blablabla )

This indeed seems to be an issue with boost and/or clang and/or macports' coping with the combination. I'll need to look a bit closer to verify. It's possible that we can mitigate if it isn't properly fixed soonish.

Got the same problem. Recompiling boost with libstdc++ didn't help. I also added CFLAGS += -stdlib=libstdc++ into makefile.osx. Still getting the ld: symbol(s) not found for architecture x86_64 error on make -f makefile.osx

Any suggestions?

I fixed the issue in the makefile.osx by commenting out that line CFLAGS += -stdlib=libstdc++ - now I can build properly on Mavericks. Still having issues compiling the QT client on Mavericks though, trying to update the .pro file so it builds as it should...

having a similar issue here. Made sure my homebrew dependencies were all up to date, started of a clean make several times. Running on OSX 10.9.1.

I don't have a 'makefile.osx' (I remember seeing one when building Litecoin though), after autogen.sh and ./configure, I end up with a regular 'Makefile' file.

My errors seem to start around here during the build:

/usr/bin/ar -rs libleveldb.a db/builder.o db/c.o db/db_impl.o db/db_iter.o db/dbformat.o db/filename.o db/log_reader.o db/log_writer.o db/memtable.o db/repair.o db/table_cache.o db/version_edit.o db/version_set.o db/write_batch.o table/block.o table/block_builder.o table/filter_block.o table/format.o table/iterator.o table/merger.o table/table.o table/table_builder.o table/two_level_iterator.o util/arena.o util/bloom.o util/cache.o util/coding.o util/comparator.o util/crc32c.o util/env.o util/env_posix.o util/env_win.o util/filter_policy.o util/hash.o util/histogram.o util/logging.o util/options.o util/status.o port/port_posix.o
ar: creating archive libleveldb.a
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libleveldb.a(env_win.o) has no symbols
  CXXLD    bitcoind
clang: warning: argument unused during compilation: '-pthread'
clang: warning: argument unused during compilation: '-pie'
Undefined symbols for architecture x86_64:
  "boost::thread_detail::commit_once_region(boost::once_flag&)", referenced from:
      LogPrintStr(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in libbitcoin_common.a(util.o)
      LockedPageManager::Instance() in libbitcoin_server.a(init.o)
      boost::spirit::classic::impl::object_with_id_base<boost::spirit::classic::impl::grammar_tag, unsigned long>::acquire_object_id() in libbitcoin_cli.a(rpcclient.o)

Getting the same error

Fixed it by specifying the boost lib dir location, in my case I'm working with Homebrew and this did it. (Never in the process I had to deal with a file named "makefile.osx", I just had a Makefile)

$ ./configure --with-boost-libdir=/usr/local/Cellar/boost/1.55.0/lib

Also, to make sure boost was linked properly I unlinked it and linked it back before building bitcoin doing brew unlink boost && brew link boost (maybe this didn't help, but it was just a step to be sure I had no boost linking issues)

If you're using Homebrew, the easiest way to workaround this issue is to compile the Boost library from source using the brew command. This will create clang/libc++ compatible Boost binaries to link against and automatically add them to the appropriate directory so you don't have to manually add them.

First uninstall the gcc/libstdc++ compiled Boost binaries by running brew uninstall boost, then run brew install boost --build-from-source --HEAD to download the latest Boost source code, compile it, and add the binaries to your classpath.

You should now be able to successfully run make -f makefile.osx, just remember to clean the build if you've already failed a compilation.

@SimonKaluza didn't work for me for bitcoin-qt :(

To fix this I uninstalled db4 and boost with brew, and then updated and reinstalled.

Still no success on building QT-Client on Mavericks :(

Project MESSAGE: Building with UPNP supportUndefined symbols for architecture x86_64: "boost::program_options::to_internal(std::string const&)", referenced from: boost::program_options::detail::basic_config_file_iterator<char>::getline(std::string&) in util.o "boost::program_options::detail::common_config_file_iterator::common_config_file_iterator(std::set<std::string, std::lessstd::string, std::allocatorstd::string > const&, bool)", referenced from: boost::program_options::detail::basic_config_file_iterator<char>::basic_config_file_iterator(std::istream&, std::set<std::string, std::lessstd::string, std::allocatorstd::string > const&, bool) in util.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation)

@mikehearn are you saying you able to get QT working by simply by uninstall and reinstalling db4 and boost? Did you build both from the source or?

Reinstalling the dependencies via brew uninstall/install worked for me. The problem is every MacOS X or XCode upgrade breaks things at the moment.

Still no luck building (--without-gui) with macports. Is there a plan of action in effect to fix this?

@BugAndNewsReporter Could you please post (at pastebin or similar) your complete build output and config.log ?

RE: plan of action: the plan of action will be to drop support for MacPorts unless somebody using MacPorts steps forward and volunteers to fix things as they break. I used to use MacPorts, but have switched to brew.

@theuni, config.log, make.out.

All the reports in this ticket look like C++ runtime library mismatches. With the release of OS X Mavericks, Apple removed all GCC compilers (there's a g++ symlink, but it points to clang++) and switched clang++'s default C++ runtime to libc++.

You will get these problems when trying to link against a library built against a different C++ runtime library than your binary uses – and the link error is actually a good thing, because the alternative would be hard-to-debug crashes at runtime.

So, you need to decide which C++ runtime library to use: Apple's new libc++, or GCC's libstdc++. In my opinion the only sane choice is clear, because Apple will no longer support GCC/libstdc++, and this combination is stuck in a version that's multiple years outdated. Especially if bitcoin wants to use C++11 at some point, the use of libc++ cannot be avoided.

Long story short: Make sure your boost lib uses the same C++ runtime as bitcoin.

MacPorts uses the same defaults as Apple's compilers, i.e. libstdc++ on systems below Mavericks, and libc++ on Mavericks and beyond. If you're compiling on Mavericks against MacPorts libraries you don't have to do anything with the autotools-based build system, because your compiler will already default to libc++. Note that this isn't the case if you have the MACOSX_DEPLOYMENT_TARGET environment variable set to anything below 10.9 (because that implicitly switches the compiler's default).

So, in conclusion:

• if you're on 10.9 and compile against MacPorts libs, ensure MACOSX_DEPLOYMENT_TARGET is unset. You cannot use the libs from MacPorts to build bitcoin for older systems by setting it.
• if you're on 10.8 and below and compile against MacPorts libs, libstdc++ will be used automatically and you shouldn't have any problems.
• if you use boost or any other dependency from a different source, check which C++ runtime it uses using otool -L. If it's libc++ and you're on <= 10.8, you need to compile with clang++ -stdlib=libc++. If it's libstdc++ and you're on 10.9 or later, you need to compile with clang++ -stdlib=libstdc++.
• if the C++ runtimes used by your dependencies differ, you will have problems.

From a MacPorts POV, there doesn't seem to by anything to fix here?

@neverpanic, can you clarify if the devs have to change something in the bitcoin build system, or that it should just work for users as long they have no external libs and everything comes from macports?

I'm on 10.9 with a fresh macports install and MACOSX_DEPLOYMENT_TARGET is unset. All dependencies come from macports, yet I still experience the linking errors described above.

@BugAndNewsReporter Your config.log shows that configure picks up /opt/local/bin/g++ as C++ compiler. It seems to be a symlink to MacPorts' GCC 4.8. You cannot use this compiler to compile C++ code on Mavericks and beyond, because it does not compile against libc++[1]. Please use clang++ as compiler instead.

[1] The truth is a little more complicated than that, but unless you're ready to dive into the gory details, this rule of thumb will serve you well and you can stop reading here. For those in the know: If you're sure you won't have problems with runtime incompatibilities you can still use g++. You can also make g++ use libc++ using http://stackoverflow.com/questions/22228208/using-g-with-libc. Note that g++ is not able to use AVX instructions on OS X, only clang++ can do that due to the outdated GNU assembler shipped by Apple, so that's another good reason to avoid GCC.

Another cause for this error can be if you're switching between old and new Xcode's and forgot which one you have. Use xcode-select to ensure you're using the latest Xcode and then if still encountering such errors (with brew) you can just uninstall and reinstall the dependencies that are failing to link.

ok, after what @neverpanic said and some googling I figured out that I had to do port select --set gcc none to reset clang++ as the default compiler. the linking errors disappeared after that.

@BugAndNewsReporter i tried it, but i receive this: -bash: port: command not found What should i do now?

Got here from google, thanks for the links @taoeffect

Um, as for @steaerox , you dont have Macports installed.... And why are you here on a building issue if you don't know the simplest bash errors? ;P

What I did to make it work.

1. Like Mike's recommendations. Uninstall/install db48/boost with brew.
2. I deleted db48 from macports (just to be sure).
3. relinked boost and db48 (brew unlink/link). Don't think it's entirely needed.
4. make clean. <-- Absentmindedly, left this out.

I think this ticket should be closed. There's nothing wrong here from a bitcoin POV, since this only happens when somebody tries to use the wrong compiler or an incompatible boost.

If anything, documentation could be improved to suggest using CXX=clang++ as argument to ./configure on OS X. The configure script could also be modified to default to that.

Of course, somebody could modify configure.ac to link a test C++ binary, use otool -L $binary | grep -E lib(std)?c++ to figure out which C++ runtime is being used and check the boost library to be used for the same runtime – I'm just not sure that would be a good use of your time.

I had the same problem of getting the message "Undefined symbols for architecture x86_64" but with building different project, and got here from google, then finally solved my problem.

From my past troubles with homebrew, I had got rid of it and just have been stuck only with macports. So none of things that are related to homebrew in this thread were helpful, but I got some hints that I haven't tried. And actually it is very simple way to try, so better to share it with hoping that someone would solve their problem with an easy action.

In the past, some of my libraries were not familiar with clang++ and had to build other gcc ports. And my finding is that what I only need to do is switching back to Apple from GNU. I have three versions available for gcc, which are "apple-gcc42", "llvm-gcc42", and "mp-gcc48" which was in active. By running "port select --set gcc "apple-gcc42", I succeed linking with boost 1.55.0.

FBSA,

For OSX 10.9 environment using Homebrew, the latest "brew install boost" bottle fixed the pasted issue below. The boost brew a few months back is definitely sour...

/usr/bin/ar -rs libleveldb.a db/builder.o db/c.o db/db_impl.o db/db_iter.o db/dbformat.o db/filename.o db/log_reader.o db/log_writer.o db/memtable.o db/repair.o db/table_cache.o db/version_edit.o db/version_set.o db/write_batch.o table/block.o table/block_builder.o table/filter_block.o table/format.o table/iterator.o table/merger.o table/table.o table/table_builder.o table/two_level_iterator.o util/arena.o util/bloom.o util/cache.o util/coding.o util/comparator.o util/crc32c.o util/env.o util/env_posix.o util/env_win.o util/filter_policy.o util/hash.o util/histogram.o util/logging.o util/options.o util/status.o port/port_posix.o ar: creating archive libleveldb.a /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libleveldb.a(env_win.o) has no symbols

I get the exact same issue as in the first post on OSX, up to date, using brew.

I think the problem come from some disagreement between gcc and clang, and I'm not sure this is actually fixable at this level.

im getting the same issue on 10.11.4 and for a while i had a working compiler, but someone asked if i could figure out how to compile for older versions of os x and so i did some research and uninstalled and reinstalled everything but now i cant get anything to compile due to the error in the op, does anyone have any advice? i am using macports

whats solution

Still have this problem

Is this fixed by building from depends?

https://github.com/bitcoin/bitcoin/tree/master/depends

@MarcoFalke Yes, I had the same issue and building from depends works. Thanks for the solution.

In addition, I found the cause. Homebrew's and Macports's bottles are created using Clang. Since it's not interoperable with GCC, compiling Bitcoin Core with (either new or old) Clang should work.