[qa] 0.15.1 Backports #11445

MarcoFalke commented at 6:16 PM on October 3, 2017: member

This includes test related backports for 0.15.1. The motivation is twofold:

Make backporting new tests written for current master easier
Fix the most common test issues that happen(ed) frequently on travis

Even though this includes the new TestNode class, which comes with a lot of refactoring, I believe that the issues caused by refactoring are found and fixed by now.

[tests] Introduce TestNode

TestNode is a class responsible for all state related to a bitcoind node
under test. It stores local state, is responsible for tracking the
bitcoind process and delegates unrecognised messages to the RPC
connection.

This commit changes start_nodes and stop_nodes to start and stop the
bitcoind nodes in parallel, making test setup and teardown much faster.

Github-Pull: #10711
Rebased-From: 7897338918dac072e788b8ab2919d4559f311bef

fc2aa09cf3

test: Increase initial RPC timeout to 60 seconds

When running the tests locally with a parallelism of 4 on an otherwise
busy system, RPC can take quite a wait to come up.

Change the timeout to 60 seconds just to be safe.

Github-Pull: #11091
Rebased-From: c1470a058f21bf98d83b9dc345c61626b87035cc

c276c1e1d1

[tests] fix timeout issues from TestNode

Fixes a couple of bugs from the introduction of TestNode:

- test scripts were no longer able to specify a custom timeout for
starting a node. Therefore tests with nodes that take a long time to
start up (eg pruning.py) would fail.
- the test for whether a node has failed on start up was broken
by changing 'assert x is None' to 'assert not x'. Since
subprocess.poll() can return None (indicating the node is still running)
or 0 (indicating the node exited with return code 0), this was a
regression.

Github-Pull: #11077
Rebased-From: 2b4ea520b717b3ca894adbac17796786667764d3

1d80d1e319

qa: Move wait_until to util

Github-Pull: #11068
Rebased-From: 08ce33f8e95efa81b37ddc6b3350462c61bbfd51

5398f205e3

[tests] add TestNodeCLI class for calling bitcoin-cli for a node

Github-Pull: #10798
Rebased-From: b23549f6e677a8e22953568704eac7ea0c2c1289

016b9ada21

[tests] Add bitcoin_cli.py test script

Github-Pull: #10798
Rebased-From: c6ec4358a797b7a11283238a0cf0b4531def9e92

716066dbea

RPC: gettxout: Slightly improve doc and tests

Github-Pull: #10859
Rebased-From: 6d2d2eb49389d70a5db327e133c9c90748f82d5f

2a5d099e47

Add getmininginfo functional test

Github-Pull: #11150
Rebased-From: 4f2905b76b502a9de235dbe739e5fe504ef5bc22

847c75ec64

[tests] fix - use rpc_timeout as rpc timeout

Github-Pull: #11121
Rebased-From: be2a2ab6a67beef97e3c3cf42bd5eeea6c4e55cf

11a5992c90

[tests] TestNode: separate add_node from start_node

Separates the act of creating a TestNode object from starting the node.
The test_framework now keeps track of its list of TestNodes, and test
writers can call start_node() and stop_node() without having to update
the self.nodes list.

Github-Pull: #11121
Rebased-From: 36b626867087e9fae6d85f926248997ebff327b7

4d3ba18386

[tests] Avoid passing around member variables in test_framework

Github-Pull: #11121
Rebased-From: 6cf094a0229d051ab8a15189c8a0bc6011919e72

bb5e7cb308

[tests] don't override __init__() in individual tests

Almost all test scripts currently need to override the __init__()
method. When they do that they need to call into super().__init__() as
the base class does some generic initialization.

This commit makes the base class __init__() call into set_test_params()
method. Individual test cases can override set_test_params() to setup
their test parameters.

Github-Pull: #11121
Rebased-From: 5448a1471d6fc638a2220ea5a2f3782172efe14c

801d2ae924

[tests] Functional tests must explicitly set num_nodes

Github-Pull: #11121
Rebased-From: 7148b74dc39110f53c665b94fa9d994c6ad6dc1c

82bf6fc6d4

[tests] fixups from set_test_params()

Github-Pull: #11215
Rebased-From: 3918d93f3cd2305dc12cb964aebba0fc1f8720b4

cef03198dc

Improve signmessages functional test

This patch improves branch coverage of the test, making sure a
message can not be verified with the wrong address or signature.

Github-Pull: #11241
Rebased-From: b3d6fc654770e3b4d2f82e8d77e531df9e522982

44241765e3

Remove redundant testutil files

Github-Pull: #11234
Rebased-From: d1138e36208dac87a27f415f39c70377636e93f1

f0b67954f0

Stop test_bitcoin-qt touching ~/.bitcoin

Github-Pull: #11210
Rebased-From: dea086f498097d19a2c9acbfc753c9c2d68dbb03

f3f7891776

[qa] TestNode: Add wait_until_stopped helper method

Github-Pull: #11067
Rebased-From: faa8d9581affde35a8242c63fca8a52004d4f943

eeb24a3a01

[test] Improve assert_raises_jsonrpc docstring

Github-Pull: #11125
Rebased-From: e1274947d4574bb83a020d0e178f0e5db7fc6282

812c870043

[test] Add support for custom arguments to TestNodeCLI

Github-Pull: #11125
Rebased-From: 5c18a84b9a49e3f9dacf0502dbf7d5d755f38da6

e0bfd28de2

[test] Add assert_raises_process_error to assert process errors

Github-Pull: #11125
Rebased-From: 232e3e8471edb346c09f906c996b2f350cabc72a

e38211f5e8

[test] Replace check_output with low level version

Github-Pull: #11125
Rebased-From: ce379b47b92d6c04250c21719bb1abfb41c586ad

2b97b36e5d

MarcoFalke force-pushed on Oct 3, 2017

[tests] fixup dbcrash interaction with add_nodes()

Github-Pull: #11230
Rebased-From: 8fdb6f9126fe1e13b60de11fd34c204247679ef4

d8dd8e7356

Add listwallets RPC test to multiwallet.py

Github-Pull: #11310
Rebased-From: fb1f3258225a7052f4fc9e3f2accc3811c53e221

b6468d3c42

[qa] zapwallettxes: Wait up to 3s for mempool reload

Github-Pull: #11308
Rebased-From: fadd0c16b6bd62e9d663d906755320ae089e02d3

2e1ac70df9

[tests] Make p2p-leaktests.py more robust

Github-Pull: #11078
Rebased-From: 0063d2c3dce9d9c1678197d2c65ee878793d1ef9

f1ced0d776

[tests] Check connectivity before sending in assumevalid.py

assumevalid.py would try to send over a closed P2P connection in a loop,
hitting the following failure many times:

TestFramework.mininode (ERROR): Cannot send message. No connection to node!

The test still passes, but this is a lot of noise in the test log.

Just check that the connection is open before trying to send.

Github-Pull: #11345
Rebased-From: e9e9391083721ca9733cc00a1907384f83b6435e

f9cf7b589f

[script] Unit tests for script/standard functions

Github-Pull: #11116
Rebased-From: d7afe2d1577eb3f6775301c81b94a191e0db99d9

794a80eee3

[script] Unit tests for IsMine

Does not test watch-only addresses.

Github-Pull: #11116
Rebased-From: 7a1e873b27b790c965d9927ecd465710dc103136

2c4ff35a8f

Tests for zmqpubrawtx and zmqpubrawblock

Github-Pull: #10552
Rebased-From: d3677ab757ea57b21d8aed24d5bc3c4bfe9bb68c

e4605d9dd4

qa: Treat mininode p2p exceptions as fatal

Github-Pull: #11319
Rebased-From: a7820422e0b182db6f1df8904242e5d76d6c73fa

2f0b30a58a

qa: Fix bug introduced in p2p-segwit.py

Changing __init__() -> set_test_params() in the tests should not have
applied to NodeConnCB-derived objects.

Github-Pull: #11319
Rebased-From: f97ab35fa9687fd5c110ad6cca5be5b4a5c2142d

8d2e51d862

Verify DBWrapper iterators are taking snapshots

The LevelDB docs seem to indicate that an iterator will not take
snapshots (even providing instructions on how to do so yourself).
In several of the places we use them, we assume snapshots to have
been taken.

In order to make sure LevelDB doesn't change out from under us
(and to prevent the next person who reads the docs from having the
same fright I did), verify that snapshots are taken in our tests.

Github-Pull: #11422
Rebased-From: bb8376bbc3d96af727444f90e5f60d47105609dc

a36f3320a9

Fix bip68-sequence rpc test

Github-Pull: #11399
Rebased-From: 49f869fe91716785b3276925d64bf8955feff69f

a825d4af5e

add functional test for mempoolreplacement command line arg

Github-Pull: #11407
Rebased-From: 1088b5322d0e7a8366a285e2baa49c766a9ba5bd

806c78f014

MarcoFalke force-pushed on Oct 3, 2017

fanquake added the label Tests on Oct 3, 2017

fanquake added this to the milestone 0.15.1 on Oct 3, 2017

fanquake added the label Backport on Oct 3, 2017

qa: Restore bitcoin-util-test py2 compatibility

Github-Pull: #11433
Rebased-From: fafff1220cf798e25f02cdd8affb70506dd366cc

e169349f8c

laanwj commented at 12:21 PM on October 4, 2017: member

utACK

qa: Fix lcov for out-of-tree builds

Github-Pull: #11443
Rebased-From: fae60e338639b3a09d2f26d8eb5feb3bb576b1fd

019c492490

fanquake commented at 2:38 AM on October 6, 2017: member

utACK 019c492

laanwj merged this on Oct 11, 2017

laanwj closed this on Oct 11, 2017

laanwj referenced this in commit 51bad9195e on Oct 11, 2017

UdjinM6 referenced this in commit ed82988445 on Sep 27, 2019

DrahtBot locked this on Sep 8, 2021