0.17: qa backports #14566

pull MarcoFalke wants to merge 14 commits into bitcoin:0.17 from MarcoFalke:Mf1810-qaBackports changing 91 files +773 −304

MarcoFalke commented at 2:04 AM on October 25, 2018: member

Just the test and doc changes from #14328 to prevent that one from piling up.

qa: Add TestNode::assert_debug_log

Github-Pull: #14024
Rebased-From: fa3e9f7627784ee00980590e5bf044a0e1249999

06544faff0

[qa] Use correct python index slices in example test

Github-Pull: #14215
Rebased-From: 9dcb6763fb6851df6572e75278a41029fbfe8809

b72fbabe17

qa: Add some actual witness in rpc_rawtransaction

Github-Pull: #14052
Rebased-From: fae040010deda9404b15b214cec2a099fb831253

7ff32a6b98

scripted-diff: Pass node into p2p_segwit acceptance tests

-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended -e 's/(test_witness_block|test_transaction_acceptance)\((self\.nodes\[.\])\.rpc/\1(\2/g' test/functional/p2p_segwit.py
-END VERIFY SCRIPT-

Github-Pull: #14101
Rebased-From: 749ba35e7c9fbc21dbea27fd1be102b91313d132

96dc936862

scripted-diff: Use named arguments in feature_block

-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended -e "s/sync_blocks\((.*?), (True|False), (1.?), /sync_blocks(\1, success=\2, reject_code=\3, reject_reason=/g" ./test/functional/feature_block.py
-END VERIFY SCRIPT-

Github-Pull: #14101
Rebased-From: b4d33096734d787b0e1d754064039cbb64ce8d61

73e538cf6a

qa: Use named args in some tests

Github-Pull: #14101
Rebased-From: fa782a308dbe7bc579c122f63c1c65666fc85e91

168efeaca6

test: Add tests for RPC help

Github-Pull: #14020
Rebased-From: 6af6d9b23dae4edd20c3dad49c835caabf3909b5

24d796a6cc

Test rpc_help.py failed: Check whether ZMQ is enabled or not.

Github-Pull: #14122
Rebased-From: 8dfc2f30dea6bde0f74d23691377f248966011ab

8bc1badada

qa: Premine to deterministic address with -disablewallet

Github-Pull: #14180
Rebased-From: faa669cbcd1fc799517b523b0f850e01b11bf40a

86fadee990

qa: Run all tests even if wallet is not compiled

Github-Pull: #14180
Rebased-From: fac95398366f644911b58f1605e6bc37fb76782d

f7adb32e38

Import CInv from correct module

Github-Pull: #13965
Rebased-From: a9cf5c9623ad547d9aeebea2b51c2afcfc0f3f4f

29899ecd36

doc/descriptors.md tweaks

Add some implementation details, and tweak phrasing in examples section to be
more explicit about how script expressions are used for matching.

Github-Pull: #14161
Rebased-From: eb494125624eb3d311c6b2e45ce403c61faddd62

5f51fd6d59

descriptors.md: Refer to descriptors as describing instead of matching
Github-Pull: #14161 Rebased-From: eeeaa29214ab99e7f491b21d21bf362a86a48305
703a24418c

Disable wallet and address book Qt tests on macOS minimal platform

macOS Qt minimal platform is frequently broken, and these are currently failing
with Qt 5.11.1.

The tests do pass when run on the full cocoa platform
(with `test_bitcoin-qt -platform cocoa`).

Github-Pull: #14011
Rebased-From: a3197c5294df4711bab0ff6dcdd061ceab115c7d

9461f98c53

MarcoFalke added the label Backport on Oct 25, 2018
MarcoFalke added this to the milestone 0.17.1 on Oct 25, 2018
MarcoFalke commented at 2:12 AM on October 25, 2018: member

This fixes the issue reported by @instagibbs https://travis-ci.org/bitcoin/bitcoin/jobs/443960300, and the issue to run the extended tests (example_test.py fails)
jnewbery commented at 5:36 PM on October 25, 2018: member

Can we pull commit 90b1d057141aa118f45bbcea7b88187ee86b5fdc (travis: Pin flake8 version) into a separate PR. That would fix the 0.17 Travis build.
MarcoFalke force-pushed on Oct 26, 2018
fanquake commented at 2:15 AM on October 26, 2018: member

@jnewbery done in #14579.

fanquake commented at 3:29 AM on October 26, 2018: member

tACK 9461f98

Travis & the linters are ok. make check is fixed on macOS (9461f98) Ran the --extended tests and all passed except feature_pruning.py:

TEST                                  | STATUS    | DURATION

example_test.py                       | ✓ Passed  | 2 s
feature_assumevalid.py                | ✓ Passed  | 15 s
feature_bip68_sequence.py             | ✓ Passed  | 36 s
feature_block.py                      | ✓ Passed  | 68 s
feature_blocksdir.py                  | ✓ Passed  | 2 s
feature_cltv.py                       | ✓ Passed  | 5 s
feature_config_args.py                | ✓ Passed  | 2 s
feature_csv_activation.py             | ✓ Passed  | 10 s
feature_dbcrash.py                    | ✓ Passed  | 1109 s
feature_dersig.py                     | ✓ Passed  | 5 s
feature_fee_estimation.py             | ✓ Passed  | 307 s
feature_help.py                       | ✓ Passed  | 0 s
feature_includeconf.py                | ✓ Passed  | 3 s
feature_logging.py                    | ✓ Passed  | 5 s
feature_maxuploadtarget.py            | ✓ Passed  | 67 s
feature_minchainwork.py               | ✓ Passed  | 6 s
feature_notifications.py              | ✓ Passed  | 2 s
feature_nulldummy.py                  | ✓ Passed  | 2 s
feature_proxy.py                      | ✓ Passed  | 5 s
feature_rbf.py                        | ✓ Passed  | 8 s
feature_reindex.py                    | ✓ Passed  | 4 s
feature_segwit.py                     | ✓ Passed  | 23 s
feature_uacomment.py                  | ✓ Passed  | 3 s
feature_versionbits_warning.py        | ✓ Passed  | 7 s
interface_bitcoin_cli.py              | ✓ Passed  | 2 s
interface_http.py                     | ✓ Passed  | 3 s
interface_rest.py                     | ✓ Passed  | 8 s
interface_zmq.py                      | ✓ Passed  | 5 s
mempool_accept.py                     | ✓ Passed  | 4 s
mempool_limit.py                      | ✓ Passed  | 4 s
mempool_packages.py                   | ✓ Passed  | 36 s
mempool_persist.py                    | ✓ Passed  | 11 s
mempool_reorg.py                      | ✓ Passed  | 4 s
mempool_resurrect.py                  | ✓ Passed  | 2 s
mempool_spend_coinbase.py             | ✓ Passed  | 1 s
mining_basic.py                       | ✓ Passed  | 3 s
mining_getblocktemplate_longpoll.py   | ✓ Passed  | 67 s
mining_prioritisetransaction.py       | ✓ Passed  | 5 s
p2p_compactblocks.py                  | ✓ Passed  | 42 s
p2p_disconnect_ban.py                 | ✓ Passed  | 4 s
p2p_feefilter.py                      | ✓ Passed  | 21 s
p2p_fingerprint.py                    | ✓ Passed  | 8 s
p2p_invalid_block.py                  | ✓ Passed  | 1 s
p2p_invalid_locator.py                | ✓ Passed  | 2 s
p2p_invalid_tx.py                     | ✓ Passed  | 2 s
p2p_leak.py                           | ✓ Passed  | 7 s
p2p_mempool.py                        | ✓ Passed  | 1 s
p2p_node_network_limited.py           | ✓ Passed  | 9 s
p2p_segwit.py                         | ✓ Passed  | 69 s
p2p_sendheaders.py                    | ✓ Passed  | 14 s
p2p_timeouts.py                       | ✓ Passed  | 63 s
p2p_unrequested_blocks.py             | ✓ Passed  | 5 s
rpc_blockchain.py                     | ✓ Passed  | 8 s
rpc_createmultisig.py                 | ✓ Passed  | 13 s
rpc_decodescript.py                   | ✓ Passed  | 1 s
rpc_deprecated.py                     | ✓ Passed  | 3 s
rpc_fundrawtransaction.py             | ✓ Passed  | 41 s
rpc_getblockstats.py                  | ✓ Passed  | 1 s
rpc_getchaintips.py                   | ✓ Passed  | 3 s
rpc_help.py                           | ✓ Passed  | 1 s
rpc_invalidateblock.py                | ✓ Passed  | 7 s
rpc_named_arguments.py                | ✓ Passed  | 1 s
rpc_net.py                            | ✓ Passed  | 2 s
rpc_preciousblock.py                  | ✓ Passed  | 3 s
rpc_psbt.py                           | ✓ Passed  | 14 s
rpc_rawtransaction.py                 | ✓ Passed  | 20 s
rpc_scantxoutset.py                   | ✓ Passed  | 7 s
rpc_signmessage.py                    | ✓ Passed  | 1 s
rpc_signrawtransaction.py             | ✓ Passed  | 2 s
rpc_txoutproof.py                     | ✓ Passed  | 3 s
rpc_uptime.py                         | ✓ Passed  | 1 s
rpc_users.py                          | ✓ Passed  | 3 s
rpc_zmq.py                            | ✓ Passed  | 3 s
wallet_abandonconflict.py             | ✓ Passed  | 7 s
wallet_address_types.py               | ✓ Passed  | 56 s
wallet_backup.py                      | ✓ Passed  | 109 s
wallet_basic.py                       | ✓ Passed  | 35 s
wallet_bumpfee.py                     | ✓ Passed  | 19 s
wallet_disable.py                     | ✓ Passed  | 1 s
wallet_disableprivatekeys.py          | ✓ Passed  | 3 s
wallet_disableprivatekeys.py --usecli | ✓ Passed  | 3 s
wallet_dump.py                        | ✓ Passed  | 10 s
wallet_encryption.py                  | ✓ Passed  | 5 s
wallet_fallbackfee.py                 | ✓ Passed  | 2 s
wallet_groups.py                      | ✓ Passed  | 14 s
wallet_hd.py                          | ✓ Passed  | 13 s
wallet_import_rescan.py               | ✓ Passed  | 19 s
wallet_importmulti.py                 | ✓ Passed  | 6 s
wallet_importprunedfunds.py           | ✓ Passed  | 4 s
wallet_keypool.py                     | ✓ Passed  | 5 s
wallet_keypool_topup.py               | ✓ Passed  | 18 s
wallet_labels.py                      | ✓ Passed  | 24 s
wallet_listreceivedby.py              | ✓ Passed  | 8 s
wallet_listsinceblock.py              | ✓ Passed  | 8 s
wallet_listtransactions.py            | ✓ Passed  | 18 s
wallet_multiwallet.py                 | ✓ Passed  | 23 s
wallet_multiwallet.py --usecli        | ✓ Passed  | 28 s
wallet_resendwallettransactions.py    | ✓ Passed  | 2 s
wallet_txn_clone.py                   | ✓ Passed  | 6 s
wallet_txn_clone.py --mineblock       | ✓ Passed  | 6 s
wallet_txn_clone.py --segwit          | ✓ Passed  | 5 s
wallet_txn_doublespend.py             | ✓ Passed  | 5 s
wallet_txn_doublespend.py --mineblock | ✓ Passed  | 6 s
wallet_zapwallettxes.py               | ✓ Passed  | 3 s
rpc_bind.py --ipv4                    | ○ Skipped | 0 s
rpc_bind.py --ipv6                    | ○ Skipped | 0 s
rpc_bind.py --nonloopback             | ○ Skipped | 0 s
feature_pruning.py                    | ✖ Failed  | 1199 s

ALL                                   | ✖ Failed  | 3907 s (accumulated) 
Runtime: 1199 s

which failed with:

<details><summary>Details</summary><p>


feature_pruning.py failed, Duration: 1199 s

stdout:
2018-10-26T02:25:27.022000Z TestFramework (INFO): Initializing test directory /var/folders/z2/cn877pxd3czdfh47mfkmbwgm0000gn/T/test_runner_₿_🏃_20181026_102522/feature_pruning_107
2018-10-26T02:25:29.028000Z TestFramework (INFO): Warning! This test requires 4GB of disk space and takes over 30 mins (up to 2 hours)
2018-10-26T02:25:29.029000Z TestFramework (INFO): Mining a big blockchain of 995 blocks
2018-10-26T02:31:29.406000Z TestFramework (INFO): Check that we haven't started pruning yet because we're below PruneAfterHeight
2018-10-26T02:31:29.407000Z TestFramework (INFO): Success
2018-10-26T02:31:29.407000Z TestFramework (INFO): Though we're already using more than 550MiB, current usage: 591
2018-10-26T02:31:29.407000Z TestFramework (INFO): Mining 25 more blocks should cause the first block file to be pruned
2018-10-26T02:31:44.934000Z TestFramework (INFO): Success
2018-10-26T02:31:44.936000Z TestFramework (INFO): Usage should be below target: 479
2018-10-26T02:31:44.937000Z TestFramework (INFO): Check that we'll exceed disk space target if we have a very high stale block rate
2018-10-26T02:31:44.937000Z TestFramework (INFO): Mine 24 (stale) blocks on Node 1, followed by 25 (main chain) block reorg from Node 0, for 12 rounds
2018-10-26T02:43:24.306000Z TestFramework (INFO): Usage can be over target because of high stale rate: 638
2018-10-26T02:43:24.308000Z TestFramework (INFO): Check that we can survive a 288 block reorg still
2018-10-26T02:43:25.699000Z TestFramework (INFO): Current block height: 1320
2018-10-26T02:43:25.699000Z TestFramework (INFO): Invalidating block 750bef50699b748116059dab8acbf280db503522c14da7592009f38a84f94521 at height 1033
2018-10-26T02:43:30.544000Z TestFramework (INFO): New best height: 1032
2018-10-26T02:43:32.032000Z TestFramework (INFO): Generating new longer chain of 300 more blocks
2018-10-26T02:43:36.339000Z TestFramework (INFO): Reconnect nodes
2018-10-26T02:43:40.797000Z TestFramework (INFO): Verify height on node 2: 1332
2018-10-26T02:43:40.799000Z TestFramework (INFO): Usage possibly still high bc of stale blocks in block files: 638
2018-10-26T02:43:40.799000Z TestFramework (INFO): Mine 220 more blocks so we have requisite history (some blocks will be big and cause pruning of previous chain)
2018-10-26T02:44:18.949000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
  File "/Users/michael/github/bitcoin/test/functional/test_framework/test_framework.py", line 170, in main
    self.run_test()
  File "/Users/michael/github/bitcoin/test/functional/feature_pruning.py", line 397, in run_test
    (self.forkheight,self.forkhash) = self.reorg_test() #(1033, )
  File "/Users/michael/github/bitcoin/test/functional/feature_pruning.py", line 180, in reorg_test
    sync_blocks(self.nodes[0:3], timeout=300)
  File "/Users/michael/github/bitcoin/test/functional/test_framework/util.py", line 388, in sync_blocks
    best_hash = [x.getbestblockhash() for x in rpc_connections]
  File "/Users/michael/github/bitcoin/test/functional/test_framework/util.py", line 388, in <listcomp>
    best_hash = [x.getbestblockhash() for x in rpc_connections]
  File "/Users/michael/github/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
  File "/Users/michael/github/bitcoin/test/functional/test_framework/authproxy.py", line 136, in __call__
    response = self._request('POST', self.__url.path, postdata.encode('utf-8'))
  File "/Users/michael/github/bitcoin/test/functional/test_framework/authproxy.py", line 106, in _request
    self.__conn.request(method, path, postdata, headers)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1055, in _send_output
    self.send(chunk)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 977, in send
    self.sock.sendall(data)
OSError: [Errno 41] Protocol wrong type for socket
2018-10-26T02:44:19.007000Z TestFramework (INFO): Stopping nodes
2018-10-26T02:44:19.008000Z TestFramework.node1 (ERROR): Unable to stop node.
Traceback (most recent call last):
  File "/Users/michael/github/bitcoin/test/functional/test_framework/test_node.py", line 210, in stop_node
    self.stop()
  File "/Users/michael/github/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
  File "/Users/michael/github/bitcoin/test/functional/test_framework/authproxy.py", line 136, in __call__
    response = self._request('POST', self.__url.path, postdata.encode('utf-8'))
  File "/Users/michael/github/bitcoin/test/functional/test_framework/authproxy.py", line 106, in _request
    self.__conn.request(method, path, postdata, headers)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1240, in _send_request
    self.putrequest(method, url, **skips)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1098, in putrequest
    raise CannotSendRequest(self.__state)
http.client.CannotSendRequest: Request-sent
2018-10-26T02:45:25.723000Z TestFramework.utils (ERROR): wait_until() failed. Predicate: (['    def is_node_stopped(self):\n', '        """Checks whether the node has stopped.\n', '\n', '        Returns True if the node has stopped. False otherwise.\n', '        This method is responsible for freeing resources (self.process)."""\n', '        if not self.running:\n', '            return True\n', '        return_code = self.process.poll()\n', '        if return_code is None:\n', '            return False\n', '\n', "        # process has stopped. Assert that it didn't return an error code.\n", '        assert return_code == 0, self._node_msg(\n', '            "Node returned non-zero exit code (%d) when stopping" % return_code)\n', '        self.running = False\n', '        self.process = None\n', '        self.rpc_connected = False\n', '        self.rpc = None\n', '        self.log.debug("Node stopped")\n', '        return True\n'], 225)
[node 5] Cleaning up leftover process
[node 2] Cleaning up leftover process
[node 1] Cleaning up leftover process


stderr:
Traceback (most recent call last):
  File "/Users/michael/github/bitcoin/test/functional/feature_pruning.py", line 461, in <module>
    PruneTest().main()
  File "/Users/michael/github/bitcoin/test/functional/test_framework/test_framework.py", line 195, in main
    self.stop_nodes()
  File "/Users/michael/github/bitcoin/test/functional/test_framework/test_framework.py", line 342, in stop_nodes
    node.wait_until_stopped()
  File "/Users/michael/github/bitcoin/test/functional/test_framework/test_node.py", line 247, in wait_until_stopped
    wait_until(self.is_node_stopped, timeout=timeout)
  File "/Users/michael/github/bitcoin/test/functional/test_framework/util.py", line 227, in wait_until
    raise AssertionError("Predicate {} not true after {} seconds".format(predicate_source, timeout))
AssertionError: Predicate (['    def is_node_stopped(self):\n', '        """Checks whether the node has stopped.\n', '\n', '        Returns True if the node has stopped. False otherwise.\n', '        This method is responsible for freeing resources (self.process)."""\n', '        if not self.running:\n', '            return True\n', '        return_code = self.process.poll()\n', '        if return_code is None:\n', '            return False\n', '\n', "        # process has stopped. Assert that it didn't return an error code.\n", '        assert return_code == 0, self._node_msg(\n', '            "Node returned non-zero exit code (%d) when stopping" % return_code)\n', '        self.running = False\n', '        self.process = None\n', '        self.rpc_connected = False\n', '        self.rpc = None\n', '        self.log.debug("Node stopped")\n', '        return True\n'], 225) not true after 60 seconds

</p></details><br>

However in subsequent runs of that test it passes successfully:

bash-3.2$ test/functional/test_runner.py feature_pruning.py
Temporary test directory at /var/folders/z2/cn877pxd3czdfh47mfkmbwgm0000gn/T/test_runner_₿_🏃_20181026_104940
.....
feature_pruning.py passed, Duration: 881 s
TEST               | STATUS    | DURATION
feature_pruning.py | ✓ Passed  | 881 s
ALL                | ✓ Passed  | 881 s (accumulated) 
Runtime: 881 s

MarcoFalke merged this on Oct 28, 2018
MarcoFalke closed this on Oct 28, 2018
MarcoFalke referenced this in commit 76dd5257f9 on Oct 28, 2018
MarcoFalke deleted the branch on Oct 28, 2018
DrahtBot locked this on Sep 8, 2021

Contributors

Labels

Milestone
0.17.1

Linked (view graph)

#14328 [0.17] Backports