Utils and libraries: Leveldb warning fixes #12550

A C-style cast is equivalent to try casting in the following order:

1. const_cast(...)
2. static_cast(...)
3. const_cast(static_cast(...))
4. reinterpret_cast(...)
5. const_cast(reinterpret_cast(...))

By using static_cast<T>(...) explicitly we avoid the possibility
of an unintentional and dangerous reinterpret_cast. Furthermore
static_cast<T>(...) allows for easier grepping of casts.

* SetAddressBook(...) is locking cs_wallet internally
* DelAddressBook(...) is locking cs_wallet internally

Irrespective of the failure reason, un-marking fInMempool
out-of-order is incorrect - it should be unmarked when
TransactionRemovedFromMempool fires.

git keeps changing the number of digits in abbreviated hashes, resulting in the GitHub archive hash changing because we include it here.
To workaround this and avoid hashes that become increasingly ambiguous later on, just include the full commit hash when building from git.
This has no effect on tagged releases.

Implements a virtual destructor on the BaseRequestHandler class to protect against undefined behavior in
the event that a derived BaseRequestHandler class has a destructor and an object of such derived class
is destroyed through a pointer to its base class.

This moves the error messages for misbehavior (when available) into the
line that reports the misbehavior, as well as moves the logging to the
`net` category.

This is a continuation of #11583 and avoids serious-looking errors due
to misbehaving peers.

To do this, Misbehaving() gains an optional `message` argument.

E.g. change:

    2018-01-18 16:02:27 Misbehaving: x.x.x.x:62174 peer=164603 (80 -> 100) BAN THRESHOLD EXCEEDED
    2018-01-18 16:02:27 ERROR: non-continuous headers sequence

to

    2018-01-18 16:02:27 Misbehaving: x.x.x.x:62174 peer=164603 (80 -> 100) BAN THRESHOLD EXCEEDED: non-continuous headers sequence

* Pass `sys.executable` when calling a python script via the subprocess
  module
* Don't remove the log file while it is still open and written to
* Properly use os.pathsep and os.path.sep when modifying the PATH
  environment variable
* util-tests: Use os.path.join for Windows compatibility

You can now build the doxygen documentation with `make docs` and clean it with `make clean-docs`.

Fixes: #11949

125f4a4909 [tests] Require all tests to follow naming convention (Anthony Towns)

Pull request description:

  Based on top of #11774

Tree-SHA512: 1eb156b5a97b30c203b7b0ad3d2055b391ef825e2e57805c7745b5259a9b1caaa115768ec225452f12f354e550f83e071f9c6fee2c36698b4679191895aab8de

Also clean out release notes.

Tree-SHA512: c4d5b52c089e14438be37381e1b0dab3711cc72aa8d345d1024169fff0055f3d021c8ca9d46fb794110694ebcbf7cbca0a12619f650873c9d381530adea7100e

Master was bumped to 0.16.99, so update the man pages too to avoid
confusion.

Tree-SHA512: 63622d6ebea2fb052ffe05fb80fe08bd627c34310a7ca22b2bc1af74003b20ab1a1fde51746ee69d401379d65232981b68541a9fc7f329e04b854507f836b19e

P2PDataStore subclasses NodeConnCB. This class keeps a store
of txs and blocks and responds to getdata requests from
the node-under-test.

The specific length of the uacomment is one shorter on `0.16.0` than on
`0.15.99` causing the (stupid) test to fail.
Just match the latter part of the message only.

aac6bce112 test: Make ua_comment test pass on 0.16.0 (Wladimir J. van der Laan)

Pull request description:

  The specific length of the uacomment is one shorter on `0.16.0` than on `0.15.99` causing the (stupid) test to fail.

  This change makes `assert_start_raises_init_error` optionally take a regexp, so that the error message can be checked without being specific about the reported length.

Tree-SHA512: 1c5e9f1d0b36f004f8113c7e211e8281194ce8057869ac3702172d8b021df3c3aa3b8f79b9434678ed0fb7146f848802410647d434fc884aa200b6a5e26afe8b

ee5e896 Organise Linux build instructions to be categorised by distro (Alex Vear)
4c85517 Add NetBSD build instruction links (Alex Vear)

Pull request description:

  * Added references to the newly created [`doc/build-netbsd.md`] (#12294) instructions in the [`doc/README.md`] and the [`doc/build-unix.md`] files.
  * Organise [`doc/build-unix.md`] dependency build instructions by Linux distribution. This will help discoverability of dependency build instructions for specific distros. Future instructions will also be able to be added easier.

  I am not quite sure about the FreeBSD instructions being in the [`doc/build-unix.md`], while both the OpenBSD and NetBSD instructions are contained within separate files ([`doc/build-openbsd.md`] and [`doc/build-netbsd.md`] respectively).

  Feedback is greatly appreciated. 😄

  [`doc/build-netbsd.md`]:https://github.com/bitcoin/bitcoin/blob/master/doc/build-netbsd.md
  [`doc/build-unix.md`]:https://github.com/bitcoin/bitcoin/blob/master/doc/build-unix.md
  [`doc/build-openbsd.md`]:https://github.com/bitcoin/bitcoin/blob/master/doc/build-openbsd.md
  [`doc/README.md`]:https://github.com/bitcoin/bitcoin/blob/master/doc/README.md

Tree-SHA512: ebe2604d1802795851bbfce2335f159b53ea696bc9afb309be7825c697b992cc3963270fe945ca3e449b18522046e227fde3fae1b9c01bd49c3a7a513b5bd40c

This adds support for setting the environment variable `BUILDDIR`
to point to executables that are outside the source directory.

E.g. to invoke the tool when the build is in $PWD/build:

```bash
BUILDDIR=$PWD/build contrib/devtools/gen-manpages.sh
```

Adds a short explanation how to update chainTxData to the release
process. Mention where to get the data, and link to an example.

7f968ae107 doc: Explain how to update chainTxData in release process (Wladimir J. van der Laan)

Pull request description:

  Adds a short explanation how to update chainTxData to the release process. Mention where to get the data, and link to an example.

Tree-SHA512: 66b0eb12a25afb7b1da0788c8f9f9701d566cb7ce55899402440a57bef0e177b55e926442d3c8aa482299abe7c48ab94735d501c37f0a11fefb86e2fc2e33d9b

96dbd38 net: initialize socket to avoid closing random fd's (Cory Fields)

Pull request description:

  An excellent spot by @david60.

  Even if it isn't causing the fd issue we're looking for, this should be fixed.

Tree-SHA512: 062a8f2cdd39d895213e1263dbd7b8391473ddaea2f93c82c211a9bb6ea6744d48a6c84c8ff804b16b865d14145492635c500a9fd138d0988fee5e4f719ebb91

Defer the GUI coin control instancing so that argument processing
is taken into account for the default coin control values.

6558f8acc [gui] Defer coin control instancing (João Barbosa)

Pull request description:

  Defer the GUI coin control instancing so that argument processing
  is taken into account for the default coin control values.

  Fixes #12312

Tree-SHA512: ecda28b94f4709319e9484b01afe763c7c3569097d2afb89db79da8a195c46d20ea77166df7edce0c8ab77627b295def01c072148714503436d27675d5e75d99

660f5f1 net: don't retry failed oneshot connections forever (Cory Fields)

Pull request description:

  As introduced by (my suggestion, sorry, in) #11512, failed dns resolves end up as oneshots. But failed oneshots are re-added as oneshots, so we need to make sure that we're not queuing these up forever after failed resolves.

  Rather than trying to differentiate, I think we should just not re-add failed oneshots and be done with it.

  Maybe @sipa can shed a light on what the original intention was.

Tree-SHA512: 2dfe35dabfb6354c315cf6f8ae42971765d36575e685662caae7ed8f9dea9472c6fb1fd5e62ec35301550b74b6613a54265e90fca2a6618544f78dacaac4d4fd

1340eda3b7 Fix typos (practicalswift)

Pull request description:

  Fix typos.

Tree-SHA512: 533a136831387ef26e9a74ba078437496bee38cc026da73fa9e6f6e7f4d5665eccac24cf3ef05e6d3af1329a1214f5ce71b039ddb8378b074e6d4408b8701f95

9bb59cf7ba QA: segwit.py: s/find_unspent/find_spendable_utxo/ (Jorge Timón)

Pull request description:

  Separated from #8994
  It was found out testing that PR but I think this fix should be done even without #8994 the fix is not necessary by luck. Unless I'm missing something.

Tree-SHA512: ba1a970e674878ae2f27ee8bba372fa3b42dafff682374f50e5ddc9896fd8808fef2b2b70c7028f2a7b0dbea9f3e29a757e2cde98970f5761bf414455ba02c09

This commit adds the final block hash of the window to getchaintxstats
and documents how reviewers can verify changes to chainTxData.

7444149 Document method for reviewers to verify chainTxData (John Newbery)

Pull request description:

  This commit adds the final block hash of the window to getchaintxstats
  and documents how reviewers can verify changes to chainTxData.

Tree-SHA512: d16abb5f47d058e52660f4d495f1e453205b1b83716d7c810ff62a70338db721386c1808ec1fc8468f514e4d80cc58e3c96eeb3184cbbcb1d07830fa5e53f342

37% of the log default log entries for a node that has been up for ~24hrs was "version handshake timeout..."

d3e467520f Properly alphabetize output of CLI --help option. (murrayn)

Pull request description:

  The --help output of bitcoind, bitcoin-cli, bitcoin-tx, qt/bitcoin-qt, et al. is only about 90% alphabetized by option, which is kind of sloppy and occasionally misleading. This change (mostly) organizes the output alphabetically.

Tree-SHA512: 3029900dbe99f03397c1cbdb5e4ac09a13bc99bafe73c6855097206e4cdd9ad70d0b5cedb5e1e520005c3e9ef1c4cd32bb9d8c98ce6918d8434fec8bf06e56c8

c409b1adac [rpc] Reduce scope of cs_main and cs_wallet locks in listtransactions (João Barbosa)

Pull request description:

  Trivial change, no behaviour change.

  Benchmark done as follow:
   - run with `-regtest`
   - wallet with 5000 transactions
   - measured the time spent with the lock and the total time
   - times are an average of 100 `listtransactions --count=...` calls

  | `--count` | lock (ms) | total (ms) | saving |
  |--:|--:|--:|--:|
  | 10 | 0.2230 | 0.2510 | 11% |
  | 100 | 2.5150 | 2.8690 | 12% |
  | 1000 | 20.0320 | 23.3490 | 14% |
  | 10000 | 105.2070 | 125.5310 | 16% |

Tree-SHA512: ebedfeeb4c8ad75c89128e53cae976a82967dbb5ffd129da0f7204ccf9c3c15070b3d509f3767bebd745512e410200cc546147c836e82409f95fc9b8d14fc3ed

Change `-conf`'s and others' help messages to indicate that relative path
values will be prefixed by the datadir path. This behavior is confusing when
attempting to specify a configuration file in the current directory with
`-conf=bitcoin.conf`, but loading the `bitcoin.conf` file in ~/.bitcoin
datadir.

Most commandline/config args are interpreted as relative to datadir if
not passed absolute. Consolidate the logic for this normalization.

c887f87 Extend #11583 to include the most common message generated by non-contributing peers (port scanners?) 37% of the log default log entries for a node that has been up for ~24hrs was "version handshake timeout..." (Clem Taylor)

Pull request description:

  37% of the default log entries for a node that has been up for ~24hrs was "version handshake timeout..."

Tree-SHA512: dceeee5d55a9ff7570174aeb63faac9beda239087220522adefef7ed11e0eeffa008ca28726011247c8834c1a222d37817baf895635ab874a95ebc435959070e

d3a185a net: Move misbehaving logging to net logging category (Wladimir J. van der Laan)

Pull request description:

  This moves the error messages for misbehavior (when available) into the line that reports the misbehavior, as well as moves the logging to the `net` category.

  This is a continuation of #11583 and avoids serious-looking errors due to misbehaving peers. As it is impossible to correlate the `peer=X` numbers to specific incoming connections now without enabling the `net` category, it doesn't really help to see these messages by default.

  To do this, Misbehaving() gains an optional `message` argument.

  E.g. change:

      2018-01-18 16:02:27 Misbehaving: x.x.x.x:62174 peer=164603 (80 -> 100) BAN THRESHOLD EXCEEDED
      2018-01-18 16:02:27 ERROR: non-continuous headers sequence

  to

      2018-01-18 16:02:27 Misbehaving: x.x.x.x:62174 peer=164603 (80 -> 100) BAN THRESHOLD EXCEEDED: non-continuous headers sequence

  When there is a category for "important" net messages (see #12219 ), we should move it there.

Tree-SHA512: 51c97e9a649bf5409f2fd4625fa1243a036e9c9de6037bb064244207408c2e0eb025e3af80866df673cdc006b8f35dc4078d074033f0d4c6a73bbb03949a269f

eeeb416 Remove suggestion to make cloned repository world-writable for Windows build. (murrayn)

Pull request description:

  Current documentation for Windows build on Ubuntu suggests cloning the repository into /usr/src, as root, and making the tree world-writable(!). I can see no problem this solves, and it introduces obvious security issues.

Tree-SHA512: 05429a64319c046f5506f7d27c64c94f94cfe6d14ec5f01dccf843fc417e954fe96e1abc43126b9204a1178f101e4a8da9eece32b5de4b348c7c9358615c7e0f

fabb72b contrib: Remove xpired 522739F6 key (MarcoFalke)
faeab66 contrib: Replace developer keys with list of pgp fingerprints (MarcoFalke)

Pull request description:

  Having to host a copy of the keys in this repo was a common source of discussion and distraction, caused by problems such as:

  * Outdated keys. Unclear whether and when to replace by fresh copies.
  * Unclear when to add a key of a new developer or Gitian builder.

  The problems are solved by
  * Having no keys but only the fingerprints
  * Adding a rule of thumb, when to add a new key

  <strike>Moving the keys to a different repo solves none of these issues, but since the keys are not bound to releases or git branches of Bitcoin Core, they should live somewhere else.

  Obviously, all keys are hosted and distributed on key servers, but were added to the repo solely for convenience and redundancy.

  Moving the mirror of those keys to a different repo makes it less distracting to update them -- let's say -- prior to every major release.

  I updated our `doc/release-process.md` to reflect the new location.

  DEPENDS_ON https://github.com/bitcoin-core/gitian.sigs/pull/621
  </strike>

Tree-SHA512: c00795a07603190e26dc4526f6ce11e492fb048dc7ef54b38f859b77dcde25f58ec4449f5cf3f85a5e9c2dd2743bde53f7ff03c8eccf0d75d51784a6b164e47d

5460460 Add AbsPathForConfigVal to consolidate datadir prefixing for path args (James O'Beirne)
a1e1305 Clarify help messages for path args to mention datadir prefix (James O'Beirne)

Pull request description:

  Change `-conf`'s help message to indicate that relative path values will be prefixed by the datadir path. This behavior probably merits clarification; it's kind of confusing when attempting to specify a configuration file in the current directory with `-conf=bitcoin.conf`, but instead loading the `bitcoin.conf` file in ~/.bitcoin datadir.

  ### Edit

  This PR has been modified to document all cases where relative path configurations are modified to be under datadir. A small refactoring has also been added which consolidates this normalization.

Tree-SHA512: be4fc0595fbeba33d17af08f59898af45e76a44f00719ea0282403b155ac6755584604fab765250a3aa14ed6991882c4d1ccbe601184362c5ba97c886bdda344

8a6c62b [tests] Update README after filename change (Conor Scott)

Pull request description:

  Update test README after filename changes from #11774

Tree-SHA512: 2dd2e4d12e9e8bef4e76996f610aea758d221f8da31e14163168a6a0c635d32fc547542112d43c37fa165c289572b12798caf467fd933082f8eb129f8e5d6ca8

bdb3231 Implements a virtual destructor on the BaseRequestHandler class. (251)

Pull request description:

  Granted that there is no undefined behavior in the current implementation, this PR implements a virtual destructor on the BaseRequestHandler class to protect against undefined behavior in the event that an object of a potential future derived BaseRequestHandler class with a destructor is destroyed through a pointer to this base class.

  This PR also fixes "_warning: delete called on 'BaseRequestHandler' that is abstract but has non-virtual destructor [-Wdelete-non-virtual-dtor]_" warnings in environments where the project is built with the `-Wsystem-headers` flag; or environments where the `-Wdelete-non-virtual-dtor` diagnostics flag fires from system headers.

Tree-SHA512: 3c3b0797a8dbce8d8c5b244709e8bca41c4e28d5ba554a974bf7fc9128413e1098c457a00e51b21154ce6c11ce5da3071626e71d593b2550d0020bc589406eed

This resolves an issue where getrawmempool() can race mempool
notification signals. Intuitively we use mempool.cs as a "read
lock" on the mempool with cs_main being the write lock, so holding
the read lock intermittently while doing write operations is
somewhat strange.
This also avoids the introduction of cs_main in getrawmempool()
which reviewers objected to in the previous fix in #12273

This prevents a potential race condition if control flow ends up in
`ShutdownHTTPServer` before the thread gets to `queue->Run()`,
deleting the work queue while workers are still going to use it.

Meant to fix #12362.

Signed-off-by: Wladimir J. van der Laan <laanwj@gmail.com>

This function, which waits for all threads to exit, is no longer needed
now that threads are joined instead.

Signed-off-by: Wladimir J. van der Laan <laanwj@gmail.com>

If the user somehow manages to get into ShutdownRequested before
ThreadImport gets to ActivateBestChain() we may hang waiting on
condvar_GenesisWait forever. A simple wait_for and
ShutdownRequested resolves this case.

If the ShutdownRequested() check at the top of ActivateBestChain()
returns false during initial genesis block load we will fail an
assertion in UTXO DB flush as the best block hash IsNull(). To work
around this, we move the check until after one round of
ActivateBestChainStep(), ensuring the genesis block gets connected.

Output bech32 addresses in dumpwallet if address type is not as legacy

The HTTP worker thread counter, as well as the RAII object that was used
to maintain it, is unused now, so can be removed.

Signed-off-by: Wladimir J. van der Laan <laanwj@gmail.com>

The `splashFinished` event was never sent if AppInitMain fails,
causing the splash screen to stick around, causing problems
later.

This bug has existed for a while but is now trigging potential crashed
because the splash screen subscribes to wallet events.

Meant to fix #12372.

Signed-off-by: Wladimir J. van der Laan <laanwj@gmail.com>

Signed-off-by: Wladimir J. van der Laan <laanwj@gmail.com>

9ad6746ccd Use static_cast instead of C-style casts for non-fundamental types (practicalswift)

Pull request description:

  A C-style cast is equivalent to try casting in the following order:

  1. `const_cast(...)`
  2. `static_cast(...)`
  3. `const_cast(static_cast(...))`
  4. `reinterpret_cast(...)`
  5. `const_cast(reinterpret_cast(...))`

  By using `static_cast<T>(...)` explicitly we avoid the possibility of an unintentional and dangerous `reinterpret_cast`. Furthermore `static_cast<T>(...)` allows for easier grepping of casts.

  For a more thorough discussion, see ["ES.49: If you must use a cast, use a named cast"](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#es49-if-you-must-use-a-cast-use-a-named-cast) in the C++ Core Guidelines (Stroustrup & Sutter).

Tree-SHA512: bd6349b7ea157da93a47b8cf238932af5dff84731374ccfd69b9f732fabdad1f9b1cdfca67497040f14eaa85346391404f4c0495e22c467f26ca883cd2de4d3c

dd2de47 Fix fast-shutdown crash if genesis block was not loaded (Matt Corallo)
1c9394a Fix fast-shutdown hang on ThreadImport+GenesisWait (Matt Corallo)

Pull request description:

  The second commit is a much simpler alternative fix for the issue fixed in #12349. To test I made ShutdownRequested() always StartShutdown() after a certain number of calls, which turned up one other hang, fixed in the first commit.

Tree-SHA512: 86bde6ac4b8b4e2cb99fff87dafeed02c0d9514acee6d94455637fb2da9ffc274b5ad31b0a6b9f5bd7b700ae35395f28ddb14ffc65ddda3619aa28df28a5607d

2222bf0 qt: Poll ShutdownTimer after init is done (MarcoFalke)

Pull request description:

  The shutdown process has started in `requestShutdown`, but initialize will happily continue with `initializeResult` and start threads late in the shutdown progess. Deleting this running thread will crash the application according to the qt docs:
  https://github.com/qt/qtbase/blob/e5033a5c9b769815112e922d0b224af860afd219/src/corelib/thread/qthread.cpp#L412-L415

  Potential fix for https://github.com/bitcoin/bitcoin/issues/12372#issuecomment-363642332

  This reverts #11831 for now and hopefully restores the previous behaviour.

Tree-SHA512: 8e1706afe90ddf2d972aca12c12d4cb2a9a4f38646c59c5466fe5a1a67361896b93c43917d5ac283841ee2bcc62e6bb8dc2bc81dea9129c899b354e9a4ef241b

1e5d14b qt: Clarify some comments (Wladimir J. van der Laan)
f5a4c3d qt: Make sure splash screen is freed on AppInitMain fail (Wladimir J. van der Laan)

Pull request description:

  The `splashFinished` event was never sent if AppInitMain fails, causing the splash screen to stick around, causing problems later.

  This bug has existed for a while but is now trigging potential crashed because the splash screen subscribes to wallet events.

  Meant to fix #12372.

Tree-SHA512: 192a7e3a528015e771d7860dd95fd7b772292fd8064abf2a3cf3a8ea0d375cd43a6e8ed37ca1a38962fe1410c934599e557adf6a8ef9d87ec7f61b6e5fd8db7e

11e0151 http: Remove numThreads and ThreadCounter (Wladimir J. van der Laan)
f946654 http: Remove WaitExit from WorkQueue (Wladimir J. van der Laan)
b1c2370 http: Join worker threads before deleting work queue (Wladimir J. van der Laan)

Pull request description:

  This prevents a potential race condition if control flow ends up in
  `ShutdownHTTPServer` before the thread gets to `queue->Run()`,
  deleting the work queue while workers are still going to use it.

  Meant to fix #12362.

Tree-SHA512: 8108514aeee5b2067a3736ed028014b580d1cbf8530ac7682b8a23070133dfa1ca21db4358c9158ea57e8811e0551395b6cb769887876b9cfce067ee968d0642

02fc886 Add braces to meet code style on line-after-the-one-changed. (Matt Corallo)
85aa839 Hold mempool.cs for the duration of ATMP. (Matt Corallo)

Pull request description:

  This resolves an issue where getrawmempool() can race mempool
  notification signals. Intuitively we use mempool.cs as a "read
  lock" on the mempool with cs_main being the write lock, so holding
  the read lock intermittently while doing write operations is
  somewhat strange.

  This also avoids the introduction of cs_main in getrawmempool()
  which reviewers objected to in the previous fix in #12273

Tree-SHA512: 29464b9ca3890010ae13b7dc1c53487cc2bc9c3cf3d32a14cb09c8aa33848f57959d8991ea096beebcfb72f062e4e1962f104aefe4252c7db87633bbfe4ab317

45eea40 Bech32 addresses in dumpwallet (fivepiece)

Pull request description:

  Output bech32 addresses in dumpwallet if address type is not as legacy

Tree-SHA512: f6b6f788293779fe6339b94d9b792180e1d1dcb9c8e826caef8693557e1710213ba57891981c17505ace8d67b407eeca6fd9a8825757dd292cca2aa12575d15c

b947d38 Add gitian PGP key: akx20000 (Akira Takizawa)

Pull request description:

Tree-SHA512: 14f8baf23120fece260ea2929c431f398a5715ef047bef9d3f6811abddf0d223defdbc30bc0be95f30550ed0cb8a81bab8ecbd21464a39b1860a60f593388250

5bdbbdc Refactor HaveKeys to early return on false result (João Barbosa)

Pull request description:

  This consists in a trivial change where the return type of `HaveKeys()` is now `bool` meaning that it returns whether all keys are in the keystore, and early returns when one isn't.

Tree-SHA512: 03e35ea8486404b84884b49f6905c9f4fc161a3eeef080b06482d77985d5242a2bdd57a34b8d16abe19ee8c6cfa3e6fbcb935c73197d53f4cd468a2c7c0b889b

27736f2 add fivepiece key fingerprint (fivepiece)

Pull request description:

Tree-SHA512: 6b2b7ca22eb02338ac2e41e8ac577bd9401f771571531d3d4c473aacc544bd4304318e311cc50b7e84236bebd7a2fda9d4c16232fefe0de4291bbbc6959b4f4b

db1cbcc [RPC] Remove deprecated addmultisigaddress return format (John Newbery)
cb28a0b [RPC] Remove deprecated createmultisig object (John Newbery)
ed45c82 [tests] Remove test for deprecated createmultsig option (John Newbery)
d066a1c [rpc] Remove deprecated getmininginfo RPC option (John Newbery)
c6f09c2 [rpc] remove deprecated estimatefee RPC (John Newbery)
a8e437a [tests] Remove estimatefee from rpc_deprecated.py test (John Newbery)
a5623b1 [tests] Remove tests for deprecated estimatefee RPC (John Newbery)
d119f2e [tests] Fix style warnings in feature_fee_estimation.py (John Newbery)

Pull request description:

  There were some RPC/RPC options deprecated in v0.16. Those can now be removed from master since v0.16 has been branched.

  - `estimatefee` RPC has been removed. The `feature_fee_estimation.py` test has been updated to remove the RPC, but doesn't yet have good coverage of the replacement RPC `estimatesmartfee`. Improving the test coverage should be done in a new PR. (#11031)
  - the `errors` field returned by `getmininginfo` has been deprecated and replaced by a `warning` field. (#10858)
  - providing addresses as inputs to `createmultisig` has been deprecated. Users should use `addmultisigaddress` instead (#11415)
  - The return format from `addmultisigaddress` has changed (#11415)

  `getwitnessaddress` was also deprecated in v0.16 and can be removed, but many tests are using that RPC, so it's a larger job to remove. It should be removed in a separate PR (possibly after #11739 and #11398 have been merged and the segwit test code tidied up)

Tree-SHA512: 8ffaa5f6094131339b9e9e468e8b141de4b144697d2271efa2992b80b12eb97849ade3da8df5c1c9400ed4c04e6a029926550a3e5846d2029b644f9e84ac7124

a9d0ebc262 Enable flake8 warnings for all currently non-violated rules (practicalswift)
4cbab15e75 tests: Fix accidental redefinition of previously defined variable via list comprehension (practicalswift)
0b9207efbe Enable flake8 warning for "list comprehension redefines 'foo' from line N" (F812) (practicalswift)

Pull request description:

  * Enable `flake8` warnings for all currently non-violated rules
  * Fix accidental redefinition via list comprehension

Tree-SHA512: 738b87789e99d02abb2c6b8ff58f65c0cbfeb93e3bf320763e033e510ebd0a4f72861bc8faaf42c14a056a5d4659c33dc70a63730a32cc15159559427bf21193

fa795cf wallet: Disallow abandon of conflicted txes (MarcoFalke)

Pull request description:

  Abandon transactions that are already conflicted is a noop, so don't try and return false/throw instead.

Tree-SHA512: fd2af4149bd2323f7f31fe18685c763790b8589319b4e467b464ab456d5e8971501ab16d124e57a22693666b06ae433ac3e59f0fd6dfbd2be2c6cae8be5bcbd8

Output the value that is tested, rather than the unmodified fee value.

This will include the error code and debug output as well as the reason string.

See #11955 for the motivation.

2f960b5 [wallet] Indent only change of CWallet::AvailableCoins (João Barbosa)
1beea7a [wallet] Make CWallet::ListCoins atomic (João Barbosa)

Pull request description:

  Fix a potencial race in `CWallet::ListCoins`.

  Replaces `cs_main` and `cs_wallet` locks by assertions in `CWallet::AvailableCoins`.

Tree-SHA512: 09109f44a08b4b53f7605d950ab506d3f748490ab9aed474aa200e93f7b0b9f96f9bf60abe1c5f658240fd13d9e3267c0dd43fd3c1695d82384198ce1da8109f

669c943 Avoid leaking prioritization information when relaying transactions (Suhas Daftuar)
e868b22 fee estimator: avoid sorting mempool on shutdown (Suhas Daftuar)
0975406 Correct mempool mapTx comment (Suhas Daftuar)

Pull request description:

  Following up on #12127 and #12118, this cleans up a comment that was left incorrect in txmempool.h, and addresses a couple of the observations @TheBlueMatt made about an unnecessary use of `queryHashes()` and a small information leak when prioritizing transactions.

  Left undone is nuking queryHashes altogether; that would require changing the behavior of the `getrawmempool` rpc call, which I think I might be in favor of doing, but wanted to save for its own PR.

Tree-SHA512: c97d10b96dcd6520459287a4a2eda92774173757695100fcfe61e526aef86f394507c331d17f9e0c14b496c33ec46198a0f165a847762ca50f7c6780b993f162

This resolves a bug introduced in
66aa1d58a158991a8014a91335b5bc9c00062f56 where, if when responding
to a series of transaction requests in a getdata we hit the send
buffer limit and set fPauseSend, we will skip one transaction per
call to ProcessGetData.

Bug found by Cory Fields (@theuni).

c8edc2c [docs] initial QT documentation, move Qt Creator instructions (Sjors Provoost)

Pull request description:

  I'll update this as I figure out how everything is tied together, but I think it's a useful enough start.

Tree-SHA512: d96e5c9ba8ccc3a1b92a0894a8a8449317100eebb14e5d390b51793534458f50eac296cf2945fccf81b85aff23fa32d91d6015a0a76ada4f7091a400d7508ae5

1687cb4 Refactor: One CBaseChainParams should be enough (Jorge Timón)

Pull request description:

  There's no need for class hierarchy with CBaseChainParams, it is just a struct with 2 fields.
  This starts as a +10-43 diff

Tree-SHA512: 0a7dd64ab785416550b541787c6083540e4962d76b6cffa806bb3593aec2daf1752dfe65ac5cd51b34ad5c31dd8292c422b483fdd2d37d0b7e68725498ed4c2d

464015f6f1 [scripts] gitian-builder.sh: fix --setup doc, LXC is default (Sjors Provoost)

Pull request description:

Tree-SHA512: 9e4c029bc0179265b20a6cecf13619f0e9d932b6911832d1aa6fc488c62c6e60bc5a10eb6eff4c668ef9cbc66942ca5c13177d0bc3b9259398780ce12a53a645

51d3ab34ba Merge #10: Add pushKV(key, boolean) function (replaces #5)
129bad96d5 [tests] test pushKV for boolean values
b3c44c947f Pushing boolean value to univalue correctly

git-subtree-dir: src/univalue
git-subtree-split: 51d3ab34ba2857f0d03dc07250cb4a2b5e712e67

-BEGIN VERIFY SCRIPT-
git grep -l "push_back(Pair" | xargs sed -i "s/push_back(Pair(\(.*\)));/pushKV(\1);/g"
-END VERIFY SCRIPT-

a8b5d20 Reset pblocktree before deleting LevelDB file (Sjors Provoost)

Pull request description:

  #11043 repaced:

  ```
  delete pblocktree;
  pblocktree = new CBlockTreeDB(nBlockTreeDBCache, false, fReset);
  ```

  With:

  ```
  pblocktree.reset(new CBlockTreeDB(nBlockTreeDBCache, false, fReset));
  ```

  This is problematic because `new CBlockTreeDB` tries to delete the existing file, which will fail with `LOCK: already held by process` if it's still open. That's the case for QT.

  When QT finds a problem with the index it will ask the user if they want to reindex. At that point it has already opened `blocks/index`.  It then runs this [while loop](https://github.com/bitcoin/bitcoin/blob/v0.16.0rc3/src/init.cpp#L1415) again with `fReset = 1`, resulting in the above error.

  This change makes that error go away, presumably because `reset()` without an argument closes the file.

Tree-SHA512: fde8b546912f6773ac64da8476673cc270b125aa2d909212391d1a2001b35c8260a8772126b99dfd76b39faaa286feb7c43239185fe584bd4dc2bc04a64044ce

c4af738 Fix ignoring tx data requests when fPauseSend is set on a peer (Matt Corallo)

Pull request description:

  This resolves a bug introduced in
  66aa1d58a158991a8014a91335b5bc9c00062f56 where, if when responding
  to a series of transaction requests in a getdata we hit the send
  buffer limit and set fPauseSend, we will skip one transaction per
  call to ProcessGetData.

  Bug found by Cory Fields (@theuni).

  Probably worth slipping into 0.16 :/.

Tree-SHA512: a9313cef8ac6da31eb099c9925c8401a638220cf7bc9b7b7b83151ecae4b02630f2db45ef6668302b9bb0f38571afbd764993427f1ec9e4d74d9a3be6647d299

004f999 boost: drop boost threads for [alert|block|wallet]notify (Cory Fields)
0827267 boost: drop boost threads from torcontrol (Cory Fields)
ba91724 boost: remove useless threadGroup parameter from Discover (Cory Fields)
f26866b boost: drop boost threads for upnp (Cory Fields)

Pull request description:

  This doesn't completely get rid of boost::thread, but this batch should be easy to review, and leaves us with only threadGroup (scheduler + scriptcheck) remaining.

  Note to reviewers: The upnp diff changes a bunch of whitespace, it's much more clear with 'git diff -w'

Tree-SHA512: 5a356798d0785f93ed143d1f0afafe890bc82f0d470bc969473da2d2aa78bcb9b096f7ba11b92564d546fb447d4bd0d347e7842994ea0170aafd53fda7e0a66e

11376b5 Fix a-vs-an typos (practicalswift)

Pull request description:

  Fix a-vs-an typos.

Tree-SHA512: 2cf74c15656a20ec13d2da7d86a39d14e634db368833d92da06a78d1266950accfc4fcc89cfecdaadd46e6b48b17e6fad29080428e564871e78482c53f3e855c

faefd29 qa: Prepare functional tests for Windows (MarcoFalke)

Pull request description:

  * Pass `sys.executable` when calling a python script via the subprocess
    module
  * Don't remove the log file while it is still open and written to
  * Properly use os.pathsep and os.path.sep when modifying the PATH
    environment variable
  * util-tests: Use os.path.join for Windows compatibility

  Ref:  #8227

Tree-SHA512: c507a536af104b3bde4366b6634099db826532bd3e7c35d694b5883c550920643b3eab79c76703ca67e1044ed09979e855088f7324321c8d52112514e334d614

a25cb0f Use ptrdiff_t type to more precisely indicate usage and avoid compiler warnings. (murrayn)

Pull request description:

  ptrdiff_t is a more strictly correct type, and gets rid of compiler warnings.

Tree-SHA512: 39718a5cdc10e698f14185f4622a9b439728bce619bd8b3a86f2b99ed5b056cf5a8545a3e5c4bc8a6a01b845fb73510036cee5e6d2629c58df26be692a957fba

faca18dcf feebumper: Use PreconditionChecks to determine bump eligibility (MarcoFalke)
718f05cab move more bumpfee prechecks to feebumper::PreconditionChecks (Gregory Sanders)

Pull request description:

  This only affects the gui.

  Fee-bumping of transactions that are already confirmed or are already conflicted by other transactions should not be offered by the gui.

Tree-SHA512: 4acf8087c69fbe5bd67be0485cdb4055e985bbf84acc420aa786ad31e2dc6c2572baaac1d359af10a6907790f626edca690285d9a46ae5440900ea12624c634f

91986ed206 scripted-diff: Use UniValue.pushKV instead of push_back(Pair()) (Karel Bilek)
a570098021 Squashed 'src/univalue/' changes from 07947ff2da..51d3ab34ba (MarcoFalke)

Pull request description:

  Rebased version of  #11386 by karel-3d.

  Closes:  #11386

Tree-SHA512: f3a81447e573c17e75813f4d41ceb34b9980eac81efdd98ddb149d7c51f792be7e2b32239b6ea7e6da68af23897afa6b4ce3f4e8070f9c4adf5105bf6075f2a0

Fixes #12386

f40df29d96 Fix Windows build errors introduced in #10498 (practicalswift)

Pull request description:

  Fix Windows build errors introduced in #10498

  Fixes #12386

  cc @ken2812221, @Sjors, @MarcoFalke and @floreslorca

Tree-SHA512: a807521fbd4015971e646fa4bab5495a3aaa97337e7b7d80b9161f33778e84ad6725cf4fbd5a35b50bf3a2bd97747cd7a630b51493ff516c2e1ad74acce148be

Credit @eklitzke for reproducing.

95e2e9a [tests] Change invalidtxrequest to use BitcoinTestFramework (John Newbery)
359d067 [tests] Fix flake8 warnings in invalidtxrequest (John Newbery)
c32cf9f [tests] Add P2PDataStore class (John Newbery)
cc046f6 [tests] Reduce NodeConn connection logging from info to debug (John Newbery)

Pull request description:

  Next step in #10603

  - first commit changes log level for an internal log from INFO to DEBUG. (Not really related, but I started finding the INFO level logging annoying when debuging test failures)
  - second commit introduces a `P2PStub` class - a subclass of `NodeConnCB` which has its own block and tx store and responds appropriately to getdata requests. Not all the functionality is used in `invalidtxrequest.py`, but will be used in `invalidblockrequest.py` and `p2p-fullblocktest` when those are changed to use `BitcoinTestFramework`
  - third commit tidies up `invalidtxrequest.py`
  - fourth commit removes usage of `ComparisonTestFramework`

Tree-SHA512: f3085c73c15d6ce894e401490bce8a7fa7cf52b0c9d135ff7e351f1f6f517c99accab8588fcdc443f39ea8315329aaabd66b2baa32499df5a774737882030373

a71c56a clientversion: Use full commit hash for commit-based version descriptions (Luke Dashjr)

Pull request description:

  git keeps changing the number of digits in abbreviated hashes, resulting in the GitHub archive hash changing because we include it here.
  To workaround this and avoid hashes that become increasingly ambiguous later on, just include the full commit hash when building from git.
  This has no effect on tagged releases.

  (Cleanly mergable back to 0.10 without backport)

Tree-SHA512: b0be5391fadd16fbc9bbeffe1574a61c95931cbf6dea885d7e3cfcd3474b89e71767b1b55b4eeeeb66e4e119e78ff579cd9d206366d36928a209a31e1c1eed75

New global variables were introduced in #11403 and not setting them causes:

    test_bitcoin: wallet/wallet.cpp:4259: CTxDestination GetDestinationForKey(const CPubKey&, OutputType): Assertion `false' failed.
    unknown location(0): fatal error in "importwallet_rescan": signal: SIGABRT (application abort requested)

It's possible to reproduce the failure reliably by running:

    src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/importwallet_rescan

Failures happen nondeterministically because boost test framework doesn't run
tests in a specified order, and tests that run previously can set the global
variables and mask the bug.

The patch Bitcoin Core has been maintaining for mac_alias was pulled by the mac_alias maintainer in commit 4f31cb084c1c6a8626128b0b00842020b6db9037. Delete the patch and remove the patch from the depends system.

Note that this PR won't be complete until a new version of mac_alias containing the path has been released, and the depends system is updated to reflect the new version.

b7f6002ed5 Fix rescan test failure due to unset g_address_type, g_change_type (Russell Yanofsky)

Pull request description:

  New global variables were introduced in #11403 and not setting them causes:

  ```
  test_bitcoin: wallet/wallet.cpp:4259: CTxDestination GetDestinationForKey(const CPubKey&, OutputType): Assertion `false' failed.
  unknown location(0): fatal error in "importwallet_rescan": signal: SIGABRT (application abort requested)
  ```

  It's possible to reproduce the failure reliably by running:

  ```
  src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/importwallet_rescan
  ```

  Failures happen nondeterministically because boost test framework doesn't run tests in a specified order, and tests that run previously can set the global variables and mask the bug.

  This is similar to bug #12150. Example travis failure is https://travis-ci.org/bitcoin/bitcoin/jobs/340642010

Tree-SHA512: ab40662b3356892b726f1f552e22d58d86b5e982538741e52b37ee447a0c97c76c24ae543687edf2e25d9dd925722909d37abfae95d93bf09e23fa245a4c3351

9b6454c Improve "Turn Windows Features On or Off" step (Ernest Hemingway)

Pull request description:

  Originally, this readme suggests searching for 'turn' to open this dialog but this will not necessarily work on all windows 10 PCs. It's better to use the executable name instead, which is consistent across installations.

Tree-SHA512: e5b95dd69a9a186ea5cd9c7aac2283e77f1857ecf628f8ad6ac0411f362c8aeb52e3bcffb46b90e3bab52f45fa244f269b1777f83d3e0519ac8a95935f7fb5b4

fa5f518 rpc: Reject deprecated reserveChangeKey in fundrawtransaction (MarcoFalke)

Pull request description:

Tree-SHA512: 8506d1494b13c4582b1379e3b8c3906016f1980ebe847727a43a90e7bb9f71b896a1792bc97a8dc7320ccce0534050eb04f92a6f82f811d08efa74a98b3e43f0

6ef86c9 Do not un-mark fInMempool on wallet txn if ATMP fails. (Matt Corallo)

Pull request description:

  Irrespective of the failure reason, un-marking fInMempool
  out-of-order is incorrect - it should be unmarked when
  TransactionRemovedFromMempool fires.

  Clean up of #11839, which I think was the wrong fix.

Tree-SHA512: 580731297eeac4c4c99ec695e15b09febf62249237bc367fcd1830fc811d3166f9336e7aba7f2f6f8601960984ae22cebed781200db0f04e7cd2008db1a83f64

d6f3a73 Remove redundant locks (practicalswift)

Pull request description:

  Remove redundant locks:
  * ~~`FindNode(...)` is locking `cs_vNodes` internally~~
  * `SetAddressBook(...)` is locking `cs_wallet` internally
  * `DelAddressBook(...)` is locking `cs_wallet` internally

  **Note to reviewers:** From what I can tell these locks are redundantly held from a data integrity perspective (guarding specific variables), and they do not appear to be needed from a data consistency perspective (ensuring a consistent state at the right points). Review thoroughly and please let me know if I'm mistaken :-)

Tree-SHA512: 7e3ca2d52fecb16385dc65051b5b20d81b502c0025d70b0c489eb3881866bdd57947a9c96931f7b213f5a8a76b6d2c7b084dff0ef2028a1e9ca9ccfd83e5b91e

ada1af6d8f Fix typo in test_runner.py causing error (MeshCollider)

Pull request description:

  In the case that a test fails, the typo in run_tests() (introduced in #11858) will cause an error rather than printing out the combined logs, hiding the cause of the failure.

Tree-SHA512: 7d7aa406d92750320ed20610cc5f174cdc94086f630af8c0c4db2003497132e0c56d59b94312fb42ad4507904a2fa858226a4a9337450930bf206183fc35c0a0

fc1bfcf Update mac_alias to 2.0.7 (Douglas Roark)
deee216 Delete mac_alias patch (Douglas Roark)

Pull request description:

  The patch Bitcoin Core has been maintaining for mac_alias was pulled by the mac_alias maintainer in commit 4f31cb084c1c6a8626128b0b00842020b6db9037. Delete the patch and remove the patch from the depends system.

  Note that this PR won't be complete until a new version of mac_alias containing the patch has been released, and the depends system is updated to reflect the new version.

Tree-SHA512: e13f1b45c0a56e95645b1aff77036c8a24c29c3f18ea0d386fba8d6d0f5fd07c434afc09dcd644d46ca096d6a7a0d5097f1eca3be5b5a5475eb3d54407044fd9

2e9406c Interrupt loading thread after shutdown request (João Barbosa)

Pull request description:

  This change (currently) avoids loading the mempool if shutdown is requested.

Tree-SHA512: 3dca3a6ea5b09bd71db0974584d93dfe81819bc0bdbb4d9b6fa0474755306d1403f6c058ecb8211384493a8f7ca3a9134173db744b7344043cfc7d79286c8fd4

fa27623 qt: Initialize members in WalletModel (MarcoFalke)

Pull request description:

  This prevents segfaults (or errors when running qt in valgrind)

  ```
  Conditional jump or move depends on uninitialised value(s)
      WalletModel::checkBalanceChanged() (walletmodel.cpp:156)

Tree-SHA512: 38c8c03c7fa947edb3f1c13eab2ac7a62ef8f8141603c2329a7dc5821a887a349af8014dc739b762e046f410f44a9c6653b6930f08b53496cf66381cadc06246

This commit fixes problems with calling LockDirectory multiple times on
the same directory, or from multiple threads. It also fixes the build on
OpenBSD.

- Wrap the boost::interprocess::file_lock in a std::unique_ptr inside
  the map that keeps track of per-directory locks. This fixes a build
  issue with the clang 4.0.0+boost-1.58.0p8 version combo on OpenBSD
  6.2, and should have no observable effect otherwise.

- Protect the locks map using a mutex.

- Make sure that only locks that are successfully acquired are inserted
  in the map.

- Open the lock file for appending only if we know we don't have the
  lock yet - The `FILE* file = fsbridge::fopen(pathLockFile, "a");`
  wipes the 'we own this lock' administration, likely because it opens
  a new fd for the locked file then closes it.

Add a unit test for LockDirectory, introduced in #11281.

bb00c95 Consistently use FormatStateMessage in RPC error output (Ben Woosley)
8b8a1c4 Add test for 'mempool min fee not met' rpc error (Ben Woosley)
c04e0f6 Fix 'mempool min fee not met' debug output (Ben Woosley)

Pull request description:

  Output the value that is tested, rather than the unmodified fee value.

  Prompted by looking into: #11955

Tree-SHA512: fc0bad47d4af375d208f657a6ccbad6ef7f4e2989ae2ce1171226c22fa92847494a2c55cca687bd5a1548663ed3313569bcc31c00d53c0c193a1b865dd8a7657

65682da [tests] bind functional test nodes to 127.0.0.1 (Sjors Provoost)

Pull request description:

  Prevents OSX firewall allow-this-application-to-accept-inbound-connections permission popups and is generally safer.

  To test, make an arbitrary whitespace change to `src/bitcoind.cpp` and recompile. This normally resets the firewall's memory.

  Easiest way to reproduce a popup without running the test suite:

  ```sh
  src/bitcoind -regtest -bind=127.0.0.1 # No popup
  src/bitcoind -regtest # Popup
  ```

Tree-SHA512: ffa92f148a2ead2ceca978c285882979639be23eb31ad6a27aa81df9fdddba5d493719c92c09a351a81d638f6f739c351a721e42168d77ead60abe074df773d6

1d4cbd2 test: Add unit test for LockDirectory (Wladimir J. van der Laan)
fc888bf util: Fix multiple use of LockDirectory (Wladimir J. van der Laan)

Pull request description:

  Wrap the `boost::interprocess::file_lock` in a `std::unique_ptr` inside the map that keeps track of per-directory locks.

  This fixes a build issue with the clang 4.0.0+boost-1.58.0p8 version combo on OpenBSD 6.2, and should have no effect otherwise.

  Also add a unit test, make the function thread-safe, and fix Linux versus Windows behavior inconsistency.

  Meant to fix #12413.

Tree-SHA512: 1a94c714c932524a51212c46e8951c129337d57b00fd3da5a347c6bcf6a947706cd440f39df935591b2079995136917f71ca7435fb356f6e8a128c509a62ec32

ceaefdd fix possible shutdown assertion with -reindex-shutdown (Cory Fields)

Pull request description:

  Fixes the assertion error reported here: https://github.com/bitcoin/bitcoin/pull/12349#issuecomment-365095741

Tree-SHA512: db8e2a275f92a99df7f17852d00eba6df996e412aa3ed3853a9ea0a8cb9800760677532efd52f92abbf2cdcc4210957a87a5f919ac998d46c205365a7a7dffca

5f605e1 Make signrawtransaction accept P2SH-P2WSH redeemscripts (Pieter Wuille)

Pull request description:

  This is a quick fix for #12418, which is a regression in 0.16.

  It permits specifying just the inner redeemscript to let `signrawtransaction` succeed. This inner redeemscript is already reported by `addmultisigaddress` & co.

  #11708 uses a different approach, where `listunspent` reports both inner & outer redeemscript, but requires both to be provided to `signrawtransaction`. Part of #11708 is still needed even in combination with this PR however, as currently the inner redeemscript isn't reported by `listunspent`.

Tree-SHA512: a6fa2b2661ce04db25cf029dd31da39c0b4811d43692f816dfe0f77b4159b5e2952051664356a579f690ccd58a626e0975708afcd7ad5919366c490944e3a9a5

fafbf7f74e devtools: Exclude patches from lint-whitespace (MarcoFalke)

Pull request description:

  By default, unified patches have trailing whitespace in all context lines. Thus, exclude patches from linting.

Tree-SHA512: 8f89f1584581e94dd4e34bd522cba21602bafe7933b4631a3abc5da5a8f25568810862d696618fe63c15edf3e046869ad5077d09373f09792985503c6a415538

d7f438a gitian: bump descriptors for (0.)17 (Cory Fields)

Pull request description:

  Bumping before we forget again. If we end up calling the next release 17.0, we'll have to fixup the descriptors anyway, so there's no harm in just doing the trivial bump now.

Tree-SHA512: f3401ec9f813132765280c60d47ae5d1697af5e6966a3798cb907dad10b7d76398032bf2a642a5a4de1055facec07a76a3055e0e9050cdab1d50caf32940e738

be45a67 Add some script tests related to BOOL ops and odd values like negative 0. (Richard Kiss)

Pull request description:

  Add some script tests related to BOOL ops and odd values like negative 0.

Tree-SHA512: 8e633f7ea5eea39e31016994baf60f295fa1dc8cae27aa5fcfc741ea97136bfb3ddc57bb62b9c6bf9fe256fc09cdd184906ba8e611e297cf8d2d363da2bbf1d4

util_tests.cpp needs to include the signal.h header on FreeBSD.

Reported by denis2342 on IRC.

64052c76c5 Merge #15: Add filename to corruption errors
135ed0fb4e Add filename to corruption errors

git-subtree-dir: src/leveldb
git-subtree-split: 64052c76c567cff3dad32d1db0ef969d97b5882f

dd7e42cbb4 test: Add missing signal.h header (Wladimir J. van der Laan)

Pull request description:

  util_tests.cpp needs to include the signal.h header on FreeBSD.

  Reported by denis2342 on IRC.

Tree-SHA512: 0b946fc2770ca6043087f4fe22be96848d46efe44e3d4d45c5ee663e0f95b0e2e6145f4ade577a6257d0c0c4cfadf324111799e78b9e4e4be4da8a90f68e349f

25409b1 fixme: depends: Add D_DARWIN_C_SOURCE to miniupnpc CFLAGS (fanquake)
3335d45 [depends] latest config.guess and config.sub (fanquake)
41550d6 [depends] miniupnpc 2.0.20180203 (fanquake)
61647a4 [depends] ccache 3.4.1 (fanquake)
5a10859 [depends] expat 2.2.5 (fanquake)

Pull request description:

  miniupnpc changelog: http://miniupnp.free.fr/files/changelog.php?file=miniupnpc-2.0.20180203.tar.gz
  2.0.20180203 includes fixes for the recent buffer overflow and segfault issues, see https://github.com/miniupnp/miniupnp/issues/268.

  expat changelog: https://github.com/libexpat/libexpat/blob/R_2_2_5/expat/Changes
  2.2.2 & 2.2.3 included security fixes.

  ccache changelog: https://ccache.samba.org/releasenotes.html#_ccache_3_4_1

  Also includes latest config.guess and config.sub.

Tree-SHA512: 5115b6ccf2bc50c62fd16ab2350bdc752eef7db8b1e4fbe35998fe1aac3702baa6c7f5e471ec48f7c614278df20a68ee6a254dde7c3e2d5c6ce2d10257a5aa21

Moves the parts of validateaddress which require the wallet into getaddressinfo
which is part of the wallet RPCs. Mark those parts of validateaddress which
require the wallet as deprecated.

Validateaddress will  call getaddressinfo
for the data that both share for right now.

Moves IsMine functions to libbitcoin_common and then links libbitcoin_wallet
before libbitcoin_common in order to prevent linker errors since IsMine is no
longer used in libbitcoin_server.

Change all instances of validateaddress to getaddressinfo since it seems that
no test actually uses validateaddress for actually validating addresses.

-BEGIN VERIFY SCRIPT-
find ./test/functional -path '*py' -not -path ./test/functional/wallet_disable.py -not -path ./test/functional/rpc_deprecated.py -not -path ./test/functional/wallet_address_types.py -exec sed -i'' -e 's/validateaddress/getaddressinfo/g' {} \;
-END VERIFY SCRIPT-

The message changed in #12356

This reverts commit 65682da7e57af544abb46a7d8a7f4fdca0c73f56.

df47afb600 Revert "[tests] bind functional test nodes to 127.0.0.1" (John Newbery)

Pull request description:

  #12200 was merged without test, and breaks `rpc_bind.py`. (Tested locally by @MarcoFalke and me).

  This PR backs out the change.

  Fixes #12462

Tree-SHA512: e92901a0ce05af86486d4f74457953a5502689b03bb2ad9a94f6fd1664385fdda8aa351e601e27ee6580227ea9adb4d7d7fbdc381e69af6b998d9c6f6d358751

e710387ca9 test: Fix bip68 sequence test to reflect updated rpc error message (Ben Woosley)

Pull request description:

  The message changed in #12356, but this test is in the extended test suite, so it didn't fail on CI.

Tree-SHA512: ce800e2636ab7bbba7876aa533e1684e75c37a7d2a119f9c4602fd89ac9215e2e28710a1f27feb642b6737ed858da049ebc52fdd476ff4637e3ac3bb1d8399ce

a777244e4 Build: Add a makefile target for Doxygen documentation (Andrea Comand)

Pull request description:

  You can now build the doxygen documentation with `make docs` and clean it with `make clean-docs`.

  Fixes: #11949

Tree-SHA512: f2361ec7f771227367dd04bba1a444b44e59f13901463a678a5f2f579a10a56d67db2e29552e754e312a1c472a31593b6af189cbaac5cd351a428c57baf5ace7

8b661f971 fixes #12465 added missing terminating newline character in log message (Josh Hartshorn)

Pull request description:

  Adding missing new line as detailed in #12465

Tree-SHA512: 48f1e3f892f7c13b862d24b4e807cc42310001cab65d771ddf758155f727554bf13c59fa9a6c9fb93d1d0466f03f142289b5fcad9d7938fd55b1a268fc1307ed

b22cce014 scripted-diff: validateaddress to getaddressinfo in tests (Andrew Chow)
b98bfc5ed Create getaddressinfo RPC and deprecate parts of validateaddress (Andrew Chow)
1598f3230 [rpc] Move DescribeAddressVisitor to rpc/util (John Newbery)
39633ecd5 [rpc] split wallet and non-wallet parts of DescribeAddressVisitor (John Newbery)

Pull request description:

  This PR makes a new RPC command called `getaddressinfo` which relies on the wallet. It contains all of `validateaddress`'s address info stuff. Those parts in `validateaddress` have been marked as deprecated. The tests have been updated to use `getaddressinfo` except the `disablewallet` test which is the only test that actually uses `validateaddress` to validate an address.

Tree-SHA512: ce00ed0f2416200b8de1e0a75e8517c024be0b6153457d302c3879b3491cce28191e7c29aed08ec7d2eeeadc62918f5c43a7cb79cd2e4b6d9291bd83ec31c852

Splits signrwatransaction into a wallet version (signrawtransactionwithwallet) and
non-wallet version (signrawtransactionwithkey). signrawtransaction is marked as DEPRECATED
and will call the right signrawtransaction* command as per the parameters in order to
maintain compatibility.

Updated signrawtransactions test to use new RPCs

-BEGIN VERIFY SCRIPT-
sed -i 's/\<signrawtransaction\>/signrawtransactionwithwallet/g' test/functional/*.py
sed -i 's/\<signrawtransaction\>/signrawtransactionwithwallet/g' test/functional/test_framework/*.py
-END VERIFY SCRIPT-

Add a brief test for signrawtransaction to ensure that compatibility is maintained.

526e28220a contrib: Add support for out-of-tree builds in gen-manpages.sh (Wladimir J. van der Laan)

Pull request description:

  This adds support for setting the environment variable `BUILDDIR` to point to executables that are outside the source directory.

  E.g. to invoke the tool when the build is in $PWD/build:

  ```bash
  BUILDDIR=$PWD/build contrib/devtools/gen-manpages.sh
  ```

  This avoids having to manually copy the generated manpages after they end up in the build instead of source path, when setting TOPDIR instead.

Tree-SHA512: 8dc6dd7a47a0c014ae7d27f0ac9d86f69238ec6bac8a3007b975bb88c9f37014755c716c5e62604dd91baad2f8a41fd1544cdca3ba4b59bc76602e6593f4a4a7

d60234885b Add test for signrawtransaction (Andrew Chow)
eefff65a4b scripted-diff: change signrawtransaction to signrawtransactionwithwallet in tests (Andrew Chow)
1e79c055cd Split signrawtransaction into wallet and non-wallet (Andrew Chow)

Pull request description:

  This PR is part of #10570. It also builds on top of #10571.

  This PR splits `signrawtransaction` into two commands, `signrawtransactionwithkey` and `signrawtransactionwithwallet`. `signrawtransactionwithkey` requires private keys to be passed in and does not use the wallet for any signing. `signrawtransactionwithwallet` uses the wallet to sign a raw transaction and does not have any parameters to take private keys.

  The `signrawtransaction` RPC has been marked as deprecated and will call the appropriate RPC command based upon the parameters given. A test was added to check this behavior is still consistent with the original behavior.

  All tests that used `signrawtransaction` have been updated to use one of the two new RPCs. Most uses were changed to `signrawtransactionwithwallet`. These were changed via a scripted diff.

Tree-SHA512: d0adf5b4cd7077639c504ec07bee262a3b94658d34db0a5c86a263b6393f7aa62f45129eafe29a7c861aa58440dd19348ee0c8b685e8a62d6f4adae8ec8f8cb3

The output would produce arbitrary number of decimal points, sometimes resulting in 9 decimals:
AssertionError: Fee of 0.00000415 BTC too low! (Should be 0.000006175 BTC)
The above looks like the expected fee is 6175 sats when in reality it's 618.

Move `-blockmaxsize`, a deprecated option which is replaced by
`-blockmaxweight`, to debug options and remove the translation.

This message is absolutely terrible for translators (esp the `* 4`
part).

a6e6e39a8b Bugfix: respect user defined configuration file (-conf) when open conf. file from QT settings (Jonas Schnelli)

Pull request description:

  Fixes #12488.

  In master, opening the configuration file from the GUI settings will always open the file "bitcoin.conf" regardless of the `-conf=` settings.
  This PR makes the GUI settings open configuration file function respect the `-conf` option.

Tree-SHA512: fb54cc699b4d2a3947f749fdf5f1a51251ffd67d0f6c6a937a5b80f0ba5a5c1085d0eef190453bbc04696d4d76c2c266de0fe9712e65e4bb36116158b54263d4

Because it's a struct.

eacc5b24f8 Declare CMutableTransaction a struct in rawtransaction.h (Ben Woosley)

Pull request description:

  Because it's a struct.

  Fix for #10579 - this was called out in code review. https://github.com/bitcoin/bitcoin/pull/10579#discussion_r168936821

Tree-SHA512: 10758a667218481de6f50b5ed874e92eb350c621f7a6355fba7da6ab42b09e1764f827e89491c8663e554fcfd23f124b299f968237c6ad1ff7819e211bd7e521

e4c924c517 createmultisig no longer takes addresses (Gregory Sanders)

Pull request description:

Tree-SHA512: a6a752ef2282c5c893dd1a6ff5ccab42d3de1989847af627d82d41605ea19abc0aaf68567a62b6478933ba7eea09f911087b410ba7d3a815331ef15ec1ff9df0

d2ee6e3 init: Remove translation for `-blockmaxsize` option help (Wladimir J. van der Laan)

Pull request description:

  Move `-blockmaxsize`, a deprecated option which is replaced by `-blockmaxweight`, to debug options and remove the translation.

  This message is absolutely terrible for translators (esp the `* 4` part).

  (for 0.17 we should probably remove this option completely?)

  (reported by French Language Coordinator)

Tree-SHA512: 379150c9217672d2f2f93b4c02a3ac638e77ca56fb518e30c56c46d59f89eac422b4f540e70a9abd3c6ad653ac4b786d4734621b18f93804885d81e223f1a908

835a21b Squashed 'src/leveldb/' changes from c521b3ac65..64052c76c5 (MarcoFalke)

Pull request description:

  Pull in changes from https://github.com/bitcoin/bitcoin/pull/11674#issuecomment-348174674.

  Merges cleanly into master and 0.16 branch.

Tree-SHA512: 819c042c0dfac8dc3078fc182c1e22d4a85b343967475d3389be5b5b056361114d8c9892437cd1dc4b45808c27880c0e166e047afc2c2bd2bbc33e55336a8c33

42e1b5d [test] Round target fee to 8 decimals in assert_fee_amount (Karl-Johan Alm)

Pull request description:

  The output would produce arbitrary number of decimal points, sometimes resulting in 9 decimals:
  ```
  AssertionError: Fee of 0.00000415 BTC too low! (Should be 0.000006175 BTC)
  ```
  The above looks like the expected fee is 6175 sats when in reality it's 618.

Tree-SHA512: ddbff2926a88890d6e34a58db36f0b15a917a80064be6e40e9bcbec3f05ae6202d02adcd7873733945b043fa121d4a56dd930446ec368078fe1935cbfff898ce

ac91ea6 [depends] Allow depends system to support armv7l (Henrik Jonsson)

Pull request description:

  Credit for the actual patches goes to @theuni, I just tested them on an armv7l machine.

  This change allows the depends system to build our dependencies on 32-bit ARM / armv7l architecture. I.e. with this patch, the following steps now builds fine:
  ```
  $ uname -m
  armv7l
  $ ./autogen.sh
  $ cd depends && make NO_QT=1 && cd ..
  $ ./configure --prefix=$(pwd)/depends/armv7l-unknown-linux-gnueabihf
  $ make
  ```

  Without this patch, the `cd depends && make NO_QT=1` command fails on armv7l.

Tree-SHA512: 1a81be2f1346e0e7080e83a991d9851f0371438262a34580eda5f43ebd220a3ce1ed0d783c43a0a8aed18082bbb1afc91e5c9bf3f4a36eecc82b6a9b6eff5e4c

fadb39c test: Plug memory leaks and stack-use-after-scope (MarcoFalke)

Pull request description:

Tree-SHA512: 7bd6bbba43c7870bbd9732d73ecfc520f21701168e6fb4ad099a08ea5b21d9cd09215e70d22fb92a1af03993204ef89ad74b3e80d9fa5a10831c3e7cf2dd04cd

face7220b7 qa: Move common args to bitcoin.conf (MarcoFalke)

Pull request description:

  Beside removing duplicates of the same args in the code, this actually helps with debugging after a test failure.

  For example, `bitcoin-qt` has `server` turned off, so you'd have to turn it on every time, if you wanted to debug a temporary test datadir created by the test framework.
  Also, `keypool` would fill up if you forget to specify `-keypool=1`.

Tree-SHA512: 996bec8738dc0fce7297ab1fc5b4fbe3aa31b9b6241f8c4e685db728925363ccaca6145ad1edc6bee2f360e02ac4b9a53fcdff74eb79de90793393742d52b559

90ba2df11 Fix missing cs_main lock for GuessVerificationProgress() (Jonas Schnelli)

Pull request description:

  `GuessVerificationProgress()` needs `cs_main` due to accessing the `pindex->nChainTx`.
  This adds a `AssertLockHeld` in `GuessVerificationProgress()` and adds the missing locks in...
  * `LoadChainTip()`
  * `ScanForWalletTransactions()` (got missed in #11281)
  * GUI, `ClientModel::getVerificationProgress()` <--- **this may have GUI performance impacts**, but could be relaxed later with a cache or something more efficient.

Tree-SHA512: 13302946571422375f32af8e396b9d2c1180f2693ea363aeba9e98c8266ddec64fe7862bfdcbb5a93a4b12165a61eec1e51e4e7d7a8515fa50879095dc163412

Tree-SHA512: 4c33c9c85bf31aa7c73b09ba9f2295eec477adc90934e51323ed50228be0dde9341a2d99ccf7adb184060fe2293533436bccfdbdfe5a30efcd61127ebb5359b8

57e6786 qa: Improve getchaintxstats functional test (João Barbosa)
501b439 rpc: Refactor blockhash parse in getchaintxstats (João Barbosa)

Pull request description:

Tree-SHA512: 61dec5cb68122998df7ec7b5239830f3caf0fe7185c107a66f27653ab2531a800db19a09050671b6fa8dbb5b53181da861eb31199c79d8635f246ccfa0d10efd

57e6786 qa: Improve getchaintxstats functional test (João Barbosa)
501b439 rpc: Refactor blockhash parse in getchaintxstats (João Barbosa)

Pull request description:

Tree-SHA512: 61dec5cb68122998df7ec7b5239830f3caf0fe7185c107a66f27653ab2531a800db19a09050671b6fa8dbb5b53181da861eb31199c79d8635f246ccfa0d10efd