Run CI against ubuntu 14.04

Empact commented at 11:18 pm on December 18, 2018: member

In light of #14979, I realized that only qt 5.5+ was being tested under CI, while compatibility lists 5.2+.

Ubuntu 14.04 carries QT 5.2.1, so running it on CI is a good candidate for maintaining compatibility in an ongoing basis. Adding the change revealed some incompatibilities that have crept in.

Alternatively, we can increase the dependencies to match those carried by Ubuntu 16.04. I don’t think we should claim support for dependencies we aren’t testing on an ongoing basis.

Putting this forward for comment. Currently the CI run times out after build and some tests running.

Fixes #14983

Empact force-pushed on Dec 18, 2018

fanquake added the label Tests on Dec 18, 2018

DrahtBot commented at 11:57 pm on December 18, 2018: member

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#15262 (build: Enable C++14 in build, require C++14 compiler. by practicalswift)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

promag commented at 0:05 am on December 19, 2018: member

I was under the impression that the minimum would bump?

Empact commented at 0:28 am on December 19, 2018: member

Yep I’m down for either, just putting this forward as an option.

Empact force-pushed on Dec 19, 2018

in .travis.yml:114 in 86b7190495 outdated

109+        DOCKER_NAME_TAG=ubuntu:14.04
110+        PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libssl-dev libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libprotobuf-dev protobuf-compiler libqrencode-dev"
111+        NO_DEPENDS=1
112+        RUN_FUNCTIONAL_TESTS=false # Disabled for now. TODO identify suppressions or exclude specific tests
113+        GOAL="install"
114+        BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=qt5 CC=clang CXX=clang++"

MarcoFalke commented at 3:24 pm on December 22, 2018:

gcc should be sufficient so we save ourself from another round of clang workarounds?

Empact force-pushed on Dec 25, 2018

Empact force-pushed on Dec 27, 2018

in .travis.yml:110 in 7ec0352e2b outdated

105+    - stage: test
106+      env: >-
107+        HOST=x86_64-unknown-linux-gnu
108+        DOCKER_NAME_TAG=ubuntu:14.04
109+        PACKAGES="libglib2.0-dev libicu-dev libpng-dev python3-zmq qtbase5-dev qttools5-dev-tools protobuf-compiler libdbus-1-dev libharfbuzz-dev libprotobuf-dev"
110+        DEP_OPTS="NO_QT=1 NO_UPNP=1 DEBUG=1 ALLOW_HOST_PACKAGES=1"

MarcoFalke commented at 2:59 pm on December 28, 2018:

This should say NO_DEPENDS=1 and we should use the boost libs from ubuntu (E.g. https://packages.ubuntu.com/trusty/libboost-thread-dev)?

MarcoFalke commented at 2:33 pm on December 29, 2018:

See #15056

in .travis.yml:112 in 7ec0352e2b outdated

107+        HOST=x86_64-unknown-linux-gnu
108+        DOCKER_NAME_TAG=ubuntu:14.04
109+        PACKAGES="libglib2.0-dev libicu-dev libpng-dev python3-zmq qtbase5-dev qttools5-dev-tools protobuf-compiler libdbus-1-dev libharfbuzz-dev libprotobuf-dev"
110+        DEP_OPTS="NO_QT=1 NO_UPNP=1 DEBUG=1 ALLOW_HOST_PACKAGES=1"
111+        GOAL="install"
112+        BITCOIN_CONFIG="--enable-zmq --with-gui=qt5 --enable-glibc-back-compat --enable-reduce-exports --enable-debug CXXFLAGS=\"-g0 -O2\""

MarcoFalke commented at 3:01 pm on December 28, 2018:

Could use --with-gui=no for now and only fix the code (in a separate pull) when we decide to support 5.2 (as opposed to 5.4/5.5)?

Empact commented at 7:00 pm on January 3, 2019:

5.2 is listed as the minimum required version here: https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md

Empact force-pushed on Dec 28, 2018

Empact force-pushed on Dec 29, 2018

Empact force-pushed on Dec 30, 2018

Empact force-pushed on Dec 31, 2018

Empact commented at 7:33 pm on December 31, 2018: member

Not clear why the build is failing - the reported test seems to pass:

 02/113 - feature_maxuploadtarget.py failed, Duration: 59 s
 1stdout:
 22018-12-31T10:43:50.480000Z TestFramework (INFO): Initializing test directory /tmp/test_runner_₿_🏃_20181231_104336/feature_maxuploadtarget_108
 32018-12-31T10:44:06.536000Z TestFramework (INFO): Peer 0 disconnected after downloading old block too many times
 42018-12-31T10:44:47.537000Z TestFramework (INFO): Peer 1 able to repeatedly download new block
 52018-12-31T10:44:47.640000Z TestFramework (INFO): Peer 1 disconnected after trying to download old block
 62018-12-31T10:44:47.640000Z TestFramework (INFO): Advancing system time on node to clear counters...
 72018-12-31T10:44:47.742000Z TestFramework (INFO): Peer 2 able to download old block
 82018-12-31T10:44:47.743000Z TestFramework (INFO): Restarting nodes with -whitelist=127.0.0.1
 92018-12-31T10:44:49.319000Z TestFramework (INFO): Peer still connected after trying to download old block (whitelisted)
102018-12-31T10:44:49.373000Z TestFramework (INFO): Stopping nodes
112018-12-31T10:44:49.525000Z TestFramework (INFO): Cleaning up /tmp/test_runner_₿_🏃_20181231_104336/feature_maxuploadtarget_108 on exit
122018-12-31T10:44:49.525000Z TestFramework (INFO): Tests successful
13stderr:
14Exception ignored in: 
15TEST                                  | STATUS    | DURATION
16wallet_hd.py                          | ✓ Passed  | 7 s
17feature_maxuploadtarget.py            | ✖ Failed  | 59 s

https://travis-ci.org/bitcoin/bitcoin/jobs/473717379#L2731

Empact force-pushed on Dec 31, 2018

Empact force-pushed on Jan 1, 2019

MarcoFalke commented at 12:03 pm on January 1, 2019: member

It fails because the stderr is not empty

Empact force-pushed on Jan 1, 2019

Empact renamed this:
~~[WIP] Run CI against ubuntu 14.04~~
Run CI against ubuntu 14.04
on Jan 1, 2019

Empact commented at 9:42 pm on January 1, 2019: member

It fails because the stderr is not empty

~Thanks, that prompted this: #15076~

Thanks, overlooked that output.

Empact force-pushed on Jan 1, 2019

Empact force-pushed on Jan 2, 2019

DrahtBot added the label Needs rebase on Jan 3, 2019

Empact force-pushed on Jan 3, 2019

in test/functional/test_runner.py:503 in 5cf7b700c5 outdated

498@@ -499,7 +499,10 @@ def get_next(self):
499                     log_out.seek(0), log_err.seek(0)
500                     [stdout, stderr] = [log_file.read().decode('utf-8') for log_file in (log_out, log_err)]
501                     log_out.close(), log_err.close()
502-                    if proc.returncode == TEST_EXIT_PASSED and stderr == "":
503+                    # Ignore "Exception ignored in: " stderr messages
504+                    # produce by python <3.6 cleanup (ubuntu < 16.04)

MarcoFalke commented at 11:09 pm on January 3, 2019:

Nit: this only exists in python 3.4

MarcoFalke commented at 11:18 am on January 4, 2019:

What I meant to say is to mention this only exists in python 3.4, so that we can remove it when python 3.4 support is dropped.

Empact commented at 11:41 pm on January 3, 2019: member

~Travis has not been running against this PR as of today. Pinged Github to raise the issue.~

in .travis.yml:113 in 5cf7b700c5 outdated

108@@ -109,6 +109,16 @@ jobs:
109         GOAL="install"
110         BITCOIN_CONFIG="--enable-zmq --disable-wallet --with-gui=qt5 CPPFLAGS=-DDEBUG_LOCKORDER --with-sanitizers=thread --disable-hardening --disable-asm CC=clang CXX=clang++"
111 
112+    - stage: test
113+      name: ''x86_64 Linux  [GOAL: install]  [trusty]  [no depends, only system libs]''

fanquake commented at 11:42 pm on January 3, 2019:

Looks like extra quotes around the name.

Empact commented at 9:16 pm on January 4, 2019:

Ah yeah, thanks.

MarcoFalke commented at 11:18 am on January 4, 2019: member

You can get the travis status on https://travis-ci.org/bitcoin/bitcoin/requests. It says

Could not parse .travis.yml

fanquake removed the label Needs rebase on Jan 4, 2019

Empact force-pushed on Jan 4, 2019

Empact force-pushed on Jan 6, 2019

Empact force-pushed on Jan 7, 2019

DrahtBot added the label Needs rebase on Jan 9, 2019

Empact force-pushed on Jan 10, 2019

DrahtBot removed the label Needs rebase on Jan 10, 2019

in src/validation.cpp:3836 in 4f2eb22e84 outdated

3833@@ -3834,7 +3834,7 @@ CBlockIndex * CChainState::InsertBlockIndex(const uint256& hash)
3834 
3835 bool CChainState::LoadBlockIndex(const Consensus::Params& consensus_params, CBlockTreeDB& blocktree)
3836 {
3837-    if (!blocktree.LoadBlockIndexGuts(consensus_params, [this](const uint256& hash) EXCLUSIVE_LOCKS_REQUIRED(cs_main) { return this->InsertBlockIndex(hash); }))
3838+    if (!blocktree.LoadBlockIndexGuts(consensus_params, [this](const uint256& hash) { AssertLockHeld(cs_main); return this->InsertBlockIndex(hash); }))

MarcoFalke commented at 4:57 pm on January 15, 2019:

I think even if it compiles, the compiler couldn’t enforce the lock annotations for lambdas anyway, since they could be used across compilation units?

Empact commented at 10:18 pm on January 31, 2019:

I’m not sure the implication, but I think this is good - EXCLUSIVE_LOCKS_REQUIRED(cs_main) is also enforced via the CChainState::LoadBlockIndex declaration, and AssertLockHeld looks into g_lockstack, which is built up in the usual way. We could perhaps do without AssertLockHeld given the LoadBlockIndex declaration. Thoughts?

Empact commented at 10:22 pm on January 31, 2019:

Removing as it’s also asserted in InsertBlockIndex

Empact commented at 10:35 pm on January 31, 2019:

Scratch that, a thread-safety-warning is issued absent this assert, thanks to the EXCLUSIVE_LOCKS_REQUIRED on InsertBlockIndex.

Empact force-pushed on Jan 16, 2019

Empact force-pushed on Jan 17, 2019

Empact force-pushed on Jan 18, 2019

MarcoFalke commented at 8:15 pm on January 30, 2019: member

Needs rebase

MarcoFalke added the label Needs rebase on Jan 30, 2019

DrahtBot removed the label Needs rebase on Jan 30, 2019

Empact force-pushed on Jan 30, 2019

Empact force-pushed on Jan 31, 2019

Do not apply attributes to lambda expressions

This was added to clang after clang 3.4. Current minimum supported version
of clang is 3.3.
https://github.com/Microsoft/clang/commit/4bfb063cae5d2dbe59778bbcd7fea4ca143c7743

Example failure:
    validation.cpp:3820:85: error: expected body of lambda expression
        if (!blocktree.LoadBlockIndexGuts(consensus_params, [this](const uint256& hash) __attribute__((exclusive_locks_required(cs_main))) { return this->InsertBlockIndex(hash); }))
                                                                                        ^
    1 error generated.
https://travis-ci.org/Empact/bitcoin/jobs/468855360

19c8e09139

Piecewise construct to avoid invalid construction

In CMainSignals::RegisterWithMempoolSignals running under Ubuntu 14.04
(QT 5.2), absent piecewise construction this fails to create the pair
because the argument is a connection, which is converted into a
non-copyable scoped_connection.

    validationinterface.cpp:80:186:   required from here
    /usr/include/boost/signals2/connection.hpp:234:7: error: ‘boost::signals2::scoped_connection::scoped_connection(const boost::signals2::scoped_connection&)’ is private
           scoped_connection(const scoped_connection &other);
           ^
    In file included from /usr/include/c++/4.8/utility:70:0,
                     from /usr/include/c++/4.8/algorithm:60,
                     from ./prevector.h:13,
                     from ./script/script.h:10,
                     from ./primitives/transaction.h:11,
                     from ./validationinterface.h:9,
                     from validationinterface.cpp:6:
    /usr/include/c++/4.8/bits/stl_pair.h:134:45: error: within this context
      : first(std::forward<_U1>(__x)), second(__y) { }
https://travis-ci.org/bitcoin/bitcoin/jobs/473689141#L2172

a0afd6158d

qt: Use the SLOT calling convention for QTimer::singleShot

Until Qt 5.4, singleShot did not support the new funtion-pointer-based call signature.
http://doc.qt.io/archives/qt-5.5/qtimer.html#singleShot-2

    qt/bitcoin.cpp:508:9: error: no matching function for call to 'singleShot'
            QTimer::singleShot(100, paymentServer, &PaymentServer::uiReady);
            ^~~~~~~~~~~~~~~~~~
    /usr/include/qt5/QtCore/qtimer.h:81:17: note: candidate function not viable: no known conversion from 'void (PaymentServer::*)()' to 'const char *' for 3rd argument
        static void singleShot(int msec, const QObject *receiver, const char *member);
                    ^
    /usr/include/qt5/QtCore/qtimer.h:82:17: note: candidate function not viable: requires 4 arguments, but 3 were provided
        static void singleShot(int msec, Qt::TimerType timerType, const QObject *receiver, const char *member);
                    ^
    1 error generated.
https://travis-ci.org/Empact/bitcoin/jobs/469269171

352f36e01c

bench: Implement IS_TRIVIALLY_CONSTRUCTIBLE via __has_trivial_constructor on clang

libcxx, which carries the implementation is apparently unversioned, so
this is an alternative to tracking down the specific version of clang
which switched from has_trivial_default_constructor to
is_trivially_default_constructible.
https://github.com/llvm-mirror/libcxx/commit/2fd6d25bf1758218aa71938ab343dcaefff4ffeb
https://libcxx.llvm.org/

    bench/prevector.cpp:27:21: error: no template named 'is_trivially_default_constructible' in namespace 'std'; did you mean 'has_trivial_default_constructor'?
    static_assert(!std::is_trivially_default_constructible<nontrivial_t>::value,
                   ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                        has_trivial_default_constructor
    /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/type_traits:1211:12: note: 'has_trivial_default_constructor' declared here
        struct has_trivial_default_constructor
               ^
    bench/prevector.cpp:31:20: error: no template named 'is_trivially_default_constructible' in namespace 'std'; did you mean 'has_trivial_default_constructor'?
    static_assert(std::is_trivially_default_constructible<trivial_t>::value,
                  ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                       has_trivial_default_constructor
    /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/type_traits:1211:12: note: 'has_trivial_default_constructor' declared here
        struct has_trivial_default_constructor
               ^
    2 errors generated.
https://travis-ci.org/Empact/bitcoin/jobs/468866070

313ca3c23c

test: Provide test_runner TestResult functionality when status is None

Avoid e.g. sort failures by reliably returning a value for every result:
https://travis-ci.org/bitcoin/bitcoin/jobs/473943177#L2728

230f834157

test: Don't consider 'Exception ignored in: ' a failure

https://travis-ci.org/bitcoin/bitcoin/jobs/474288596#L2733

a267fc8425

Prefer boost::optional#get_value_or over #value_or

The latter is not defined in the earliest supported version of boost,
1.47.
https://www.boost.org/doc/libs/1_47_0/libs/optional/doc/html/boost_optional/detailed_semantics.html
https://travis-ci.org/bitcoin/bitcoin/jobs/486674823

43a42293f0

Revert "travis: Compile trusty with depends for now"

Thus building against system rather than depends libs

This reverts commit fa5ce3f10ea98e5cd6bdd829b5b7b8fd2736d286.

c8aa08e153

Empact force-pushed on Jan 31, 2019

Empact commented at 10:36 pm on January 31, 2019: member

Closing because we’re finally green and a clean PR would be nice.

Empact closed this on Jan 31, 2019

MarcoFalke referenced this in commit 2c0867a181 on Feb 1, 2019

DrahtBot locked this on Dec 16, 2021

Run CI against ubuntu 14.04 #14998

Conflicts