RFC: switching to C++ 11

laanwj commented at 11:04 am on June 1, 2015: member

Now that 0.11 has been branched off it may be a good time to resume discussion about switching to C++11.

When would be a good time? 0.12 (planned the end of this year)? or 0.13 (halfway 2016)?

In my view, the only drawback of switching to C++11 is that the source will no longer be compilable on older Linux distributions, as well as on embedded toolchains with outdated gccs. E.g. the gcc in Ubuntu 12.04 is 4.6, which does not support the full set of C++11 though it can be done with some trickery.

The compiled executables in releases will still work on the same platforms, since the build has been switched to static libstdc++ since #5819.

(see also: compile with C++11 standard and update to modern constructs #4025)

laanwj added the label Build system on Jun 1, 2015

jonasschnelli commented at 11:15 am on June 1, 2015: contributor

+1 for C++11 and slowly reduce the usage of boost. The problems with toolchains on embedded systems could be a issue (especially for the user-case where people using part of bitcoin-cores sources in other projects). I think 0.13 would be realistic.

gavinandresen commented at 1:42 pm on June 1, 2015: contributor

I’d vote for allowing whatever c++11 features gcc 4.6 supports (listed here https://gcc.gnu.org/projects/cxx0x.html ) in 0.12. Even just supporting ‘auto’ would make me more productive, and it would be lovely to get rid of BOOST_FOREACH.

Full c++11 support in 0.13.

jonasschnelli commented at 1:49 pm on June 1, 2015: contributor

side note: clang llvm also supports c++11 fully since version 3.3 which is from the mid of 2013.

Diapolo commented at 2:45 pm on June 1, 2015: none

I also think It’d be nice to reduce the need for Boost in exchange for C++11 features. But we would also need a newer pulltester/Travis compiler I guess?

laanwj commented at 4:59 pm on June 1, 2015: member

Thanks for the comments.

I think using partial c++11, although seemingly a no-brainer, was rejected last time by @theuni, as it is hard to delimit the exact feature set that should be used and which not. @diapolo I’m assuming Travis supports a whole range of compilers. As for gitian we can just go to an 14.04 image.

theuni commented at 5:31 pm on June 1, 2015: member

+1 for c++11, though my stance on a partial implementation has not changed. With the current compiler environment, I don’t see any need for it. I maintain that the c++11 constructs (auto, uniform initialization, move semantics and rvalue references, variadic templates, lambdas, range-based-for, etc) bleed into eachother, so trying to set boundaries would get nasty and very confusing.

However, I think that the move is important, so I’m willing to concede that point if necessary.

I think it’s also worth considering making libbitcoinconsensus able to compile without c++11, but I could go either way on that.

ghost commented at 4:43 pm on June 2, 2015: none

++1 for c++11. otherwise, I echo theuni’s thoughts.

laanwj commented at 9:36 am on June 6, 2015: member

It’d be worth switching over just for the initialization improvements and and range-based-for, it comes up too much lately, as people for some reason seem to be very eager to get rid of boost usage (at least #6227 #6242 and that’s only the first page).

luke-jr commented at 6:52 pm on June 15, 2015: member

IMO we should support the lowest common denominator dependencies provided by major stable/LTS distros. I think last time I looked at that, RedHat was the limiting factor, but that may have changed…

0RHEL 7: GCC 4.8
1Debian 8: GCC 4.8, 4.9
2Ubuntu 14.04: GCC 4.8
3Gentoo stable: GCC 4.8
4Fedora 22: GCC 5.1

Any others I’m forgetting?

luke-jr commented at 6:55 pm on June 15, 2015: member

Oh, another relevant factor I’d forgotten: ABI compatibility. Last I checked, C++11 in GCC is not ABI-compatible with libraries compiled without C++11 enabled, or with C++11 support between different minor compiler versions. This could be a real problem in terms of supporting builds on distros not using C++11 yet.

laanwj commented at 7:25 pm on June 15, 2015: member

@luke-jr You mean for the compiled executables? They are built statically against libstdc++, so the binary ABI for c++ shouldn’t make a difference.

Thanks for listing the compiler versions. If the lowest common denomicator is gcc 4.8 that means we can use quite a large part of c++11.

luke-jr commented at 7:41 pm on June 15, 2015: member

@laanwj I mean for users who build from source themselves. They may link to the right libstdc++ (I hope), but for example, if Qt was compiled without C++11, it might have subtle ABI incompatibilities.

laanwj commented at 11:55 am on June 16, 2015: member

@luke-jr Right. But you could have the same problem the other way around, if the dependency libraries have been built with c++11 but not bitcoind. If there is an ABI compatibility within one system there is no escape from that issue.

luke-jr commented at 9:26 pm on June 16, 2015: member

@laanwj If code is C++98, then it will compile in either mode, so compatible with systems built with C++11 or not…

pstratem commented at 3:44 pm on June 29, 2015: contributor

It should be noted that gcc lists c++11 support as experimental.

laanwj commented at 6:58 am on July 1, 2015: member

@pstratem Some C++11 features have been finalized for some time, some are still experimental even in recent versions. What we should do is make a list of C++11 features we agree to use and add that to the coding standard, then set the minimum gcc and clang version based on that.

luke-jr commented at 7:03 am on July 1, 2015: member

https://gcc.gnu.org/wiki/Cxx11AbiCompatibility suggests switching to C++11 before all supported platforms reach GCC 5.2 may be annoyingly difficult. :(

sipa commented at 11:32 pm on July 9, 2015: member

I think just move semantics, rvalue references, auto variables, and range loops are already very much worth it.

randy-waterhouse commented at 1:01 am on July 11, 2015: contributor

@luke-jr @Diapolo regarding “lowest common denominator”, worth being aware that the Travis linux builds upgrade to support ubuntu 14.04 LTS https://github.com/travis-ci/travis-ci/issues/2046 is looking a bit stagnant.

laanwj commented at 10:21 am on August 25, 2015: member

From @luke-jr’s list it looks like 4.8 is the lowest common denominator of current stable gccs. This lists the status of C++11 support in 4.8: https://gcc.gnu.org/gcc-4.8/cxx0x_status.html The only no is Minimal support for garbage collection and reachability-based leak detection. Rvalue references for *this and decltype and call expressions were only introduced in 4.8.1.

Everything @sipa mentions is supported. It’s unfortunate that Travis doesn’t support Ubuntu 14.04 yet. Don’t know what version of gcc it is running but this may effectively undermine our automatic testing after using some C++11 features.

droark commented at 10:48 pm on November 20, 2015: contributor

Hello. When I was working on Armory, C++11 code was added in 0.93. We did this before realizing that, as mentioned here, Ubuntu 12.04 can’t compile C++11 code by default. We also had trouble figuring out how to compile a version of Armory that would run in 12.04. This commit shows what we settled on. The solution we developed was to tell people that, unfortunately, they couldn’t build Armory on 12.04 without switching to clang or a non-standard GCC 4.7.3 or 4.8+ package floating around out there. (Fortunately, I know of nobody compiling Armory on 12.04. I know Core probably doesn’t have that luxury.) The best we were able to do was set things up such that static compiling and a SWIG workaround allowed 12.10 and beyond to create a binary that could run on 12.04.

I’m hard pressed to give a firm opinion on when C++11 should be introduced to Core. My initial thought is to go with what Gavin said, although I know 0.12 is getting close to the release date. So, 0.13 would likely be a better starting point at this point. However, Luke probably brings up a good point regarding ABI issues, which I’ve heard about here and there. I don’t know enough to say whether or not he’s right. If anybody has a test that I can run by compiling Armory in Ubuntu 15.10 (I believe it uses GCC 5.2 by default), let me know and I’ll be happy to do it. I’ve got VMs out the wazoo.

Thanks for making it this far. :)

jeaye commented at 4:28 am on November 30, 2015: none

+1 for C++11. Given that we’re approaching 2016, +1 for C++14 (which GCC 5 fully supports).

A number of benefits have been listed, primarily around auto/rvalue refs/range-based for, but I think the real driving factor is the potential safety improvements. C++11 and C++14 are moving toward a more functional paradigm, with first-class functions (std::function) and lambdas (which can be closures), and a more generic paradigm with auto, variadic templates, static_assert, and new/improved type traits.

By upgrading, we’re rewarded with added compile-time type safety (with improved generics, constexpr functions, and static_assert), memory safety (std::unique_ptr, std::shared_ptr, std::weak_ptr, std::array), and thread safety (std::thread, std::future, std::mutex, std::atomic, and a new thread-aware memory model).

I’d be happy to help with this migration and I’m hoping that this ball can get rolling ASAP.

dcousens commented at 4:43 am on November 30, 2015: contributor

@jeaye I don’t think there will be a migration as such, for the same reasons as a code formatting. Its likely we’ll just be fixing any warnings that pop up upon enabling the flags, then just ensuring future PR’s take advantage of these features properly.

That is my impression anyway.

luke-jr commented at 4:44 am on November 30, 2015: member

@jeaye If you’d like to help, please read over the issues in this thread and try to figure out a way that we can migrate without breaking compatibility with existing systems. As far as I can tell, C++11 is completely unusable in practice today. :(

droark commented at 4:49 am on November 30, 2015: contributor

@jeaye - I suspect C++14 support won’t be coming along for awhile. My understanding is that the devs are pretty conservative when it comes to supporting development environments, hence Wladimir’s opening statement regarding Ubuntu 12.04. My understanding is also that Ubuntu 14.04 doesn’t officially support GCC 4.9 (mostly complete C++14 support), although there are some unofficial (?) packages. There’s also the matter of the Windows build, which uses MinGW to create a Windows build. The Gitian setup that generates the official Windows build uses 14.04 too, and GCC-MinGW has the same version issue (4.8.2) as regular GCC. Ubuntu 15.10 is the first version to properly (AFAIK) support C++14. Once 14.04 has faded enough from the spotlight and been replaced by 16.04, C++14 may be reasonable. Until then, it’s probably best to avoid it for at least a couple of years. :(

Of course, if any devs want to correct me, please do. :) Also, C++14 is technically off-topic here. A new issue should probably be opened if C++14 support is considered critical enough.

jeaye commented at 5:08 am on November 30, 2015: none

I suspect C++14 support won’t be coming along for awhile. … Also, C++14 is technically off-topic here.

Yeah, I don’t mean to sidetrack; we can drop the C++14 talk.

I don’t think there will be a migration as such, for the same reasons as a code formatting.

That would be a shame. Also, comparing C++03 -> C++11 upgrade to code formatting is unfair; you rarely get safety and performance improvements from code formatting. :)

As far as I can tell, C++11 is completely unusable in practice today.

As multiple people have shown above, C++11 is supported (GCC 4.8+) on every major Linux distro, on multiple BSDs, on OS X (clang had it long before GCC), and on Windows (cygwin supports GCC 4.9.x, MSVC 13 supports C++11). Many of them have supported it for quite some time and are moving on to newer standards.

It’s unfortunate that Travis doesn’t support Ubuntu 14.04 yet. Don’t know what version of gcc it is running but this may effectively undermine our automatic testing after using some C++11 features.

Plenty of projects, including some of mine, are using C++11 (and 14) with travis. Here’s a quick example and a related SO answer.

luke-jr commented at 5:20 am on November 30, 2015: member

@jeaye So what of the ABI problems? Software compiled by GCC in C++11 mode are ABI-incompatible with libraries compiled in C++98 mode (ie, most libraries now in any modern system, since a lot of software won’t build in C++11 mode).

dcousens commented at 5:36 am on November 30, 2015: contributor

@luke-jr more importantly, what libraries that bitcoin-core uses does it affect? AFAIK of the external dependencies, each are modern enough to allow compilation to C++11? The rest we maintain in-house such that they can easily be upgraded.

jeaye commented at 6:17 am on November 30, 2015: none

@luke-jr Following what @dcousens said, simply speculating that there may be issues shouldn’t prevent us from trying this out. There is absolutely the possibility, depending on bitcoin’s dependencies (with which I’m unfamiliar), that bitcoin relies on a C++ library, which is statically linked, that doesn’t provide a C++11 build. However, that very well may not be the case; I can’t speak for every platform on this one.

OS X, for example, allows boost with C++11 through homebrew with a simple additional flag during install. Further investigation would be needed on various platforms (admittedly, most of my systems are rather bleeding edge, so, again, I can’t speak for others).

If we end up with some dependencies which need manual recompilation on older distros, like Ubuntu 12, we can consider packaging them along with bitcoin and building them separately. It’s common to do this with boost (I know of multiple projects which do).

luke-jr commented at 7:27 am on November 30, 2015: member

@dcousens AFAIK there is no simple way to determine what libraries are affected. It is unacceptable to require users to recompile external dependencies which should be installed and managed by the distro. The distro cannot upgrade those libraries to be built with C++11 ABI without breaking applications using them (those which need C++98). Basically GCC has set things up so either the entire OS must be C++98 or the entire OS must be C++11. :( @jeaye It is not merely speculation that there are ABI differences, and I am unaware of any guarantee that such ABI conflicts will be cause at compile time. It is possible that the conflicts might even result in a subtle consensus break which we cannot reasonably detect with the current incomplete state of unit tests.

dcousens commented at 7:45 am on November 30, 2015: contributor

Basically GCC has set things up so either the entire OS must be C++98 or the entire OS must be C++11. :(

:(. Thanks for the continued explanation.

I guess we’re just going to wait on some percentage of distributions to switch to gcc>5.0.0 then?

RHEL 7: GCC 4.8 Debian 8: GCC 4.8, 4.9 Ubuntu 14.04: GCC 4.8 Gentoo stable: GCC 4.8 Fedora 22: GCC 5.1

Any others I’m forgetting?

Arch, which is at 5.2.0

jeaye commented at 8:01 am on November 30, 2015: none

I am unaware of any guarantee that such ABI conflicts will be cause at compile time. It is possible that the conflicts might even result in a subtle consensus break which we cannot reasonably detect with the current incomplete state of unit tests.

I can’t disagree with any of that. The speculation, however, isn’t that the ABI has changed, it’s whether or not that’s actually going to mean anything on these platforms.

As you said, the issues may not arise at compile-time at all, however, we have a fixed set of dependencies and distributions have standardized package versions. Fortunately, everything is not unpredictable if we rely on standardized setups (if someone has recompiled half of their system with a cross-compiler, it’s not reasonable for us to provide support, for example). Furthermore, for compiled C++ programs, tracing the run-time dependencies (using ldd, for example) is much easier than something like Python.

Alas, how much investigation and assurance is required is currently unknown. I don’t see this as insurmountable, though it seems I’m the minority.

jeaye commented at 8:05 am on November 30, 2015: none

To update the list:

0RHEL 7: GCC 4.8
1Debian 8: GCC 4.8, 4.9
2Ubuntu 14.04: GCC 4.8
3Gentoo stable: GCC 4.8
4Fedora 22: GCC 5.1
5Arch: GCC 5.2
6Slackware 14.1: GCC 4.8.2
7Slackware -current: GCC 5.2
8OS X: GCC 5.2, Clang 3.6.2 (homebrew)

luke-jr commented at 9:15 am on November 30, 2015: member

@jeaye We absolutely should support a user having compiled their entire OS themselves.

Gentoo stable is now at GCC 4.9 FWIW.

laanwj commented at 9:24 am on November 30, 2015: member

@luke-jr @dcousens I disagree that gcc 4.x versus 5.x ABI issues are a blocker for this, or even our problem.

There is plenty of software using c++11 in the wild successfully. It is not some experimental thing anymore.

If distributions and OSes have gcc ABI issues it’s up to them to solve that. e.g. OpenBSD has a particularly nasty one between different versions of 4.x. Using (a) our binaries, if available, or (b) use of depends system to build all dependencies, should be enough to work around that.

Of course, if any devs want to correct me, please do. :) Also, C++14 is technically off-topic here. A new issue should probably be opened if C++14 support is considered critical enough.

Yes, c++14 is off-topic.

jeaye commented at 9:29 am on November 30, 2015: none

We absolutely should support a user having compiled their entire OS themselves.

Perhaps you misunderstood; that’s not what I was implying. “if someone has recompiled half of their system with a cross-compiler…” The emphasis on half shows the introduction of uncontrollable ABI incompatibilities. We should support a stable, consistent system, either with the new ABI or old. My example failed, apparently, at providing something in between stable states.

There is plenty of software using c++11 in the wild successfully. It is not some experimental thing anymore.

This is absolutely my point.

dcousens commented at 9:40 am on November 30, 2015: contributor

@laanwj it is only a blocker if we intend to target the platform versions with which we could have [theoretically] ABI issues with? As mentioned, if the incompatibility is subtle, it could affect consensus.

I’m all for making this happen, as I use C++14 in production everywhere myself.

luke-jr commented at 10:03 am on November 30, 2015: member

@laanwj It’s not 4.x vs 5.x; it’s same-version [-std=c++98] vs same-version -std=c++11 for all versions of GCC. What C++11 software do you know of in the wild that we can learn from?

laanwj commented at 11:03 am on November 30, 2015: member

@laanwj It’s not 4.x vs 5.x; it’s same-version [-std=c++98] vs same-version -std=c++11 for all versions of GCC.

That would mean we can never use c++11 because there may be some library, somewhere, that is still compiled with `-std=c++98’, or some system on which this ABI conflict exists.

Other software that is compiled using c++11 and uses any libraries written in c++ would also suffer from this. I’ve never even heard of this issue apart from here. In the cases where this could be an issue, we should add a warning and start suggesting building boost and Qt from scratch, I guess. But I’m not going to hold off the use of c++11 in Bitcoin Core on this.

laanwj added this to the milestone 0.13.0 on Dec 3, 2015

laanwj referenced this in commit a0fe544117 on Dec 3, 2015

laanwj added the label Refactoring on Feb 16, 2016

theuni referenced this in commit 270a122478 on Apr 21, 2016

theuni referenced this in commit 58621cac45 on Apr 21, 2016

sipa referenced this in commit 8710f826e4 on Apr 26, 2016

laanwj referenced this in commit 67969af09f on Apr 26, 2016

theuni referenced this in commit 7e48c5c136 on Apr 26, 2016

laanwj commented at 11:18 am on April 28, 2016: member

Closing, this was done.

laanwj closed this on Apr 28, 2016

rebroad referenced this in commit dce9d65c0d on Dec 7, 2016

deadalnix referenced this in commit 07a947ed9b on Dec 13, 2016

deadalnix referenced this in commit b856580b68 on Dec 13, 2016

deadalnix referenced this in commit c268afe85e on Jan 1, 2017

deadalnix referenced this in commit 2869473eec on Jan 6, 2017

deadalnix referenced this in commit 8cc97f26a1 on Jan 8, 2017

deadalnix referenced this in commit cc2bc947be on Jan 9, 2017

deadalnix referenced this in commit c54998b44e on Jan 10, 2017

deadalnix referenced this in commit b513c416fc on Jan 15, 2017

deadalnix referenced this in commit df9722e546 on Jan 16, 2017

deadalnix referenced this in commit 7ffd5ab5d1 on Jan 17, 2017

Infernoman referenced this in commit 4ba59bbc9c on Jan 31, 2017

sickpig referenced this in commit ef0695f31f on Feb 27, 2017

protonn referenced this in commit 5da27115b6 on Apr 11, 2017

OlegGirko referenced this in commit 41663e4f3d on May 3, 2017

OlegGirko referenced this in commit dcf4f06fa2 on May 3, 2017

UdjinM6 referenced this in commit 39750439bf on May 5, 2017

MarcoFalke locked this on Sep 8, 2021

RFC: switching to C++ 11 #6211