[qa] 0.13.2 Backports #9191

MarcoFalke commented at 10:23 PM on November 19, 2016: member

No description provided.

[qa] add assert_raises_message to check specific error message

Github-Pull: #9168
Rebased-From: 307acdd3df03082295ac0f7fe9eba7dd35973bc4

3107280e14

[qa] Increase wallet-dump RPC timeout

Increase wallet-dump RPC timeout from 30 seconds to 1 minute. This avoids a
timeout error that seemed to happen regularly (around 50% of builds) on a
particular jenkins server during the first getnewaddress RPC call made by the
test.

The failing stack trace looked like:

    Unexpected exception caught during testing: timeout('timed out',)

    File ".../bitcoin/qa/rpc-tests/test_framework/test_framework.py", line 146, in main
      self.run_test()
    File ".../bitcoin/qa/rpc-tests/wallet-dump.py", line 73, in run_test
      addr = self.nodes[0].getnewaddress()
    File ".../bitcoin/qa/rpc-tests/test_framework/coverage.py", line 49, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File ".../bitcoin/qa/rpc-tests/test_framework/authproxy.py", line 145, in __call__
      response = self._request('POST', self.__url.path, postdata.encode('utf-8'))
    File ".../bitcoin/qa/rpc-tests/test_framework/authproxy.py", line 121, in _request
      return self._get_response()
    File ".../bitcoin/qa/rpc-tests/test_framework/authproxy.py", line 160, in _get_response
      http_response = self.__conn.getresponse()
    File "/usr/lib/python3.4/http/client.py", line 1171, in getresponse
      response.begin()
    File "/usr/lib/python3.4/http/client.py", line 351, in begin
      version, status, reason = self._read_status()
    File "/usr/lib/python3.4/http/client.py", line 313, in _read_status
      line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
    File "/usr/lib/python3.4/socket.py", line 374, in readinto
      return self._sock.recv_into(b)

Github-Pull: #9077
Rebased-From: 8463aaa63c5ac76421c4d2754ea9e17a31584c93

1d4c884cd3

[qa] Add more helpful RPC timeout message

Replace previous timeout('timed out',) exception with more detailed error.

Github-Pull: #9077
Rebased-From: e89614b6abf28d7fe201c3db44a0df6e4db6de03

da4926b1d2

MarcoFalke added the label Backport on Nov 19, 2016

MarcoFalke added the label Tests on Nov 19, 2016

MarcoFalke added this to the milestone 0.13.2 on Nov 19, 2016

test: Fix use-after-free in scheduler tests

Make a copy of the boost time-point to wait for, otherwise the head of
the queue may be deleted by another thread while this one is waiting,
while the boost function still has a reference to it.

Although this problem is in non-test code, this is not an actual problem
outside of the tests because we use the thread scheduler with only one
service thread, so there will never be threads fighting at the head of
the queue.

The old boost fallback escapes this problem because it passes a scalar
value to wait_until instead of a const object reference.

Found by running the tests in LLVM-4.0-master asan.

Github-Pull: #9186
Rebased-From: 12519bf62b8c49b1c1744eca6ea5b3162a61f962

dccdc3aa34

[qa] Wait for specific block announcement in p2p-compactblocks

Change check_announcement_of_new_block() to wait specifically for the
announcement of the newly created block, instead of waiting for any
announcement at all. A difficult to reproduce failure in
check_announcement_of_new_block() that happened in a travis build
(https://travis-ci.org/bitcoin/bitcoin/jobs/175198367) might have happened
because an older announcement was mistaken for the expected one. The error
looked like:

  Assertion failed: Failed

  File ".../bitcoin/qa/rpc-tests/test_framework/test_framework.py", line 145, in main
    self.run_test()
  File ".../bitcoin/build/../qa/rpc-tests/p2p-compactblocks.py", line 787, in run_test
    self.test_sendcmpct(self.nodes[1], self.segwit_node, 2, old_node=self.old_node)
  File ".../bitcoin/build/../qa/rpc-tests/p2p-compactblocks.py", line 201, in test_sendcmpct
    check_announcement_of_new_block(node, test_node, lambda p: p.last_cmpctblock is None and p.last_inv is not None)
  File ".../bitcoin/build/../qa/rpc-tests/p2p-compactblocks.py", line 194, in check_announcement_of_new_block
    assert(predicate(peer))

This commit also changes the assertion failed message above to include more
detailed information for debug.

Github-Pull: #9159
Rebased-From: dfa44d1b07a6d1022005dba63dd6372739eee8a0

eca9b46537

[qa] Fix bug in compactblocks v2 merge

Bug caused the wait_for_block_announcement to be called on the wrong node,
leading to nondeterminism and occasional test failures. Bug was introduced in
merge commit:

d075479 Merge #8882: [qa] Fix race conditions in p2p-compactblocks.py and sendheaders.py

Underlying commits which conflicted were:

27acfc1 [qa] Update p2p-compactblocks.py for compactblocks v2
6976db2 [qa] Another attempt to fix race condition in p2p-compactblocks.py

The first commit changed the test_compactblock_construction function signature
and second commit added code which wasn't updated during the merge to use the
new arguments.

Suhas Daftuar <sdaftuar@chaincode.com> noticed the bug and suggested the fix.

Github-Pull: #9058
Rebased-From: 47e9659ecfbe07077a4564591095bd5510e0f917

2ba5d78427

[qa] Fix stale data bug in test_compactblocks_not_at_tip

Clear test_node.last_block before requesting blocks in the
compactblocks_not_at_tip test so comparisons won't fail if a blocks were received
before the test started.

The bug doesn't currently cause any problems due to the order tests run, but
this will change in the next commit.

Github-Pull: #9058
Rebased-From: 55bfddcabbf9e8a3743f77167ba4a43aaba9f948

286e548d87

Dont remove a "preferred" cmpctblock peer if they provide a block

Github-Pull: #8637
Rebased-From: 02a337defdd854efc78ecba6d1fb19cb1c075f16

36e3b95103

More agressively filter compact block requests

Unit test adaptations by Pieter Wuille.

Github-Pull: #8637
Rebased-From: fe998e962dc015978f104b782afb7daec3c4d4df

76ba1c9739

Add cmpctblock to debug help list

Github-Pull: #8637
Rebased-From: b2e93a343ec2dc7d255b970e6ee45e9c390f7ed0

3d23a0eaa3

Align constant names for maximum compact block / blocktxn depth

Github-Pull: #8637
Rebased-From: 3ac6de0a045cc9b2047ceb19af970e7ffbf905fa

2cad5db6f7

MarcoFalke added the label Refactoring on Nov 20, 2016

MarcoFalke commented at 2:15 PM on November 20, 2016: member

This also includes #9058 and thus, also the cleanup pull #8637.

MarcoFalke force-pushed on Nov 20, 2016

Modify getblocktxn handler not to drop requests for old blocks

The current getblocktxn implementation drops and ignores requests for old
blocks, which causes occasional sync_block timeouts during the
p2p-compactblocks.py test as reported in
https://github.com/bitcoin/bitcoin/issues/8842.

The p2p-compactblocks.py test setup creates many new blocks in a short
period of time, which can lead to getblocktxn requests for blocks below the
hardcoded depth limit of 10 blocks. This commit changes the getblocktxn
handler not to ignore these requests, so the peer nodes in the test setup
will reliably be able to sync.

The protocol change is documented in BIP-152 update "Allow block responses
to getblocktxn requests" at https://github.com/bitcoin/bips/pull/469.

The protocol change is not expected to affect nodes running outside the test
environment, because there shouldn't normally be lots of new blocks being
rapidly added that need to be synced.

Github-Pull: #9058
Rebased-From: dac53b58b555183ccc0d5e64c428528267cd98b3
Github-Pull: #9160
Rebased-From: ec34648766c4052816e4072cc61ad429430bcfd9

e8461666ec

MarcoFalke force-pushed on Nov 20, 2016

laanwj commented at 7:48 AM on November 21, 2016: member

Commit-list-ACK

sdaftuar commented at 1:17 AM on November 23, 2016: member

ACK

fanquake commented at 6:43 AM on November 23, 2016: member

ACK, I think #9169 should be included as well?

MarcoFalke force-pushed on Nov 23, 2016

MarcoFalke commented at 12:17 PM on November 23, 2016: member

I tried to focus on test fixes and necessary refactoring only in this pull and left the other backports ( https://github.com/bitcoin/bitcoin/pulls?utf8=%E2%9C%93&q=is%3Apr%20label%3A%22Needs%20backport%22%20 ) for later.

laanwj merged this on Dec 2, 2016

laanwj closed this on Dec 2, 2016

laanwj referenced this in commit 29435db6a4 on Dec 2, 2016

laanwj commented at 7:18 AM on December 2, 2016: member

New backports pull: #9264

MarcoFalke deleted the branch on Dec 2, 2016

MarcoFalke locked this on Sep 8, 2021