Autotools RFC: Round 1

theuni commented at 8:55 am on June 7, 2013: member

Here is the first go at an Autotools build system replacement. It is meant to be a drop-in replacement for the current system(s), providing the same features with no net changes. It can also live side-by-side with the old system while sharing the same build-related variables in order to facilitate a smooth transition.

I hope the benefits are obvious enough: A single/shared build procedure, portability, ease of packaging, ease for downstreams, ease for repository maintainers, cross-compilation, etc. I don’t vouch for Autotools in any way, in fact, this configure.am is downright ugly (mainly just because of mingw though), but it’s portable and well-established.

I’ve opted not to write the documentation yet, because I would like to see what comments/concerns come out of the first round of review before committing to anything.

This does away with the need for qmake, as the Makefile is capable of generating everything it needs in a portable way.

Qt-creator can be used with the Autotools plugin, and is working nicely. For those who wish to use it this way, install the Autotools plugin from the ‘about’ menu, then open Makefile.am as a project. It handles the build procedure, so there is no need to mess with the command-line procedures listed below.

Building from CLI: For Linux, assuming the dependencies have been met, the build procedure looks like this:

0./autogen.sh
1./configure
2make

Same for OS X, but the pkg-config path needs to be hooked up from macports first:

0echo "/opt/local/share/aclocal" | sudo tee -a /usr/share/aclocal/dirlist

For mingw it’s the same, but you will need to provide lots of paths in the form of:

0./autogen.sh
1CPPFLAGS="-I/path/to/include -I/path/to/other/include" LDFLAGS="-L/path/to/libs -L/path/to/other/libs" ./configure
2make

In addition, there are helpers for qt and boost to help with finding some locations. Use ./configure –help to see the available options.

Native windows built is untested, as I don’t have a windows environment at my disposal.

I have verified that linux builds are deterministic insofar as a fresh git clone will build bit-exact binaries to match another clean checkout on the same machine. To my knowledge, that means that gitian shouldn’t have any trouble making the switch (after the descriptors are updated to use autotools).

‘make check’ will run the unit tests and print the results.

I’ve done my best to avoid adding any new behavior or features, and I would much prefer to aim for feature-parity before making any improvements.

TODO:

Update gitian descriptors.
Add documentation to reflect comments here
Cosmetics

autotools: cleanup usage of MSG_NOSIGNAL

This is Part of an autotools buildsystem port.

Previously MSG_NOSIGNAL was defined in a few places, and -DMSG_NOSIGNAL=0 was
passed by the osx build to signify that it was unavailable there. For Win32, it
was assumed to be unavailable.

This commit flips the logic so that the unix build declares the feature. The
HAVE_MSG_NOSIGNAL define will be (automatically detected and) set by autotools
as well, so that both build methods will give the same results.

There should be no observable changes in functionality here.

eeb8569d71

autotools: include bitcoin-config.h for autotools builds e982815666

autotools: add gitignore 3602b34c8c

autotools: add autotools files fa5805fadc

jgarzik commented at 2:51 pm on June 7, 2013: contributor

Looks mostly OK, well done. Comments:

MSG_NOSIGNAL cleanup should be a separate commit, but the rest should be collapsed into a single commit. e.g. “add gitignore” logically belongs with the “add autotools files” commit.
use SUBDIRS, multiple Makefile.am files, and avoid relative paths, particularly for the test data.

laanwj commented at 3:30 pm on June 7, 2013: member

Looks good.

One question: why is the configure script and autogen.sh in src/, and not in the root of the project like most projects do?

sipa commented at 4:21 pm on June 7, 2013: member

One (potential) nit: several of the M4 files are GPL licensed (with a special exception); I don’t know enough about licensing issues, but perhaps this impacts our source distribution? @gmaxwell @luke-Jr comments?

theuni commented at 10:06 pm on June 7, 2013: member

@jgarzik

Sure, I’ll squash down if/when reviewers sign-off. It was pushed this way so that the code-changes are easy to review separate from the buildsystem ones.
I used a single makefile in order to make life easier for qt-creator users. How firm are you on the split? Agreed about the test-data. Will change it.

theuni commented at 10:09 pm on June 7, 2013: member

@laanwj I suppose there’s no real reason, it just made more sense to me this way. One current issue with moving to the top would mean that the .pro file would have to move (if it’s kept around after autotools merge), since it produces a Makefile that would conflict.

theuni commented at 10:17 pm on June 7, 2013: member

@sipa the gpl in those files is effectively meaningless. Since they’re always distributed in source-form, and resulting scripts are exempt from infection, the only way to violate would be to strip the attribution (which would violate MIT as well). So the net effect here is effectively nil.

laanwj commented at 11:33 pm on June 7, 2013: member

@theuni just assume that either the .pro will be used or the autotools build system, not both in parallel, so the makefiles won’t get in each others way. I’d prefer the configure script to be at the top level, but others may disagree…

sipa commented at 5:45 pm on June 8, 2013: member

I have a slight preference for configure in the top level, but I doń’t feel strongly about it.

BitcoinPullTester commented at 6:03 pm on June 8, 2013: none

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/fa5805fadc369ab8fd695dbd2fa0cc7a0bee1514 for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

sipa commented at 2:45 pm on June 15, 2013: member

Just a generic comment: I’m strongly in favor of moving towards a real build system, but under the condition that we switch entirely. I wouldn’t want to retain both the old and the new, and needing to maintain them both. So, as soon as this can be used to build on linux/linux-mingw/osx/win32, and the gitian builds work, I’m fine with merging. Perhaps @laanwj wants to retain the .pro file, but assuming the autotools plugin to Qt Creator is powerful enough, I’d rather get rid of it.

theuni commented at 6:48 pm on June 15, 2013: member

Ok. I’m working on the reorg concerns above, I didn’t have much time to get to it this week. In the meantime, though, @warren has helped me to get the gitian descriptors up and running for win32, and linux should be easy after that.

I’ve gotten the same vibe from the others, so I’ll finish up with the impression that back-compat is not needed. Also, @laanwj said he is ok with figuring out some solution for himself for the .pro file, so I’ll not concern myself with that either.

sipa commented at 3:14 pm on June 29, 2013: member

@theuni Hi, any updates about this?

There have been a few refactors lately, which may impact this work (core.cpp/h and chainparams.cpp/h were added), but I think the largest changes are over now.

I’d really like to see this merged.

theuni commented at 7:51 am on June 30, 2013: member

Hi @sipa

I’ve been very occupied for the last few weeks, sorry for the lack of updates here. I have a branch that splits out the makefiles as requested. Personally I think it’s a sizable regression, but I’m ok with that if it’s what it takes to get it merged.

I’ll try find some time tomorrow to push up a rebased branch for review round2.

sipa commented at 3:51 pm on June 30, 2013: member

@theuni Great, no actual hurry - I just wouldn’t like this work to get lost.

theuni commented at 5:20 am on July 1, 2013: member

Closing in favor of round 2: #2805

theuni closed this on Jul 1, 2013

Bushstar referenced this in commit e47af29d41 on Apr 5, 2019

DrahtBot locked this on Sep 8, 2021

Autotools RFC: Round 1 #2748