Update libsecp256k1 and switch to its builtin RFC6979 #5506

Now that signing is deterministic, we can require exact correspondence between the
automatically generated tests and the ones read from JSON. Do this, and update
the tests to those deterministic versions. Note that some flag changes weren't
correctly applied before.

Fix typos where appropriate
Update license/copyright

- this was reported on Transifex by pryds (Thank you!)

8a5c951 [REST] make selection of output-format mandatory, support dot url syntax (Jonas Schnelli)

Builds on #5316.

This avoids connecting to them again too soon in ThreadOpenConnections.

Make an exception for connection failures to the proxy as these
shouldn't affect the status of specific nodes.

b2d0162 Test resurrecting memory pool transactions during chain re-org (Gavin Andresen)
3dd8ed7 Delay writing block indexes in invalidate/reconsider (Pieter Wuille)
798faec Add 'invalidateblock' and 'reconsiderblock' RPC commands. (Pieter Wuille)

I dropped mention of libgmp that I had in my first draft because
it looks like we'll be able to get that out prior to release.

362001c Move CHECKMULTISIG order tests out of automatically generated block (Pieter Wuille)
f67a9ce Use deterministically generated script tests (Pieter Wuille)

Although script_valid.json and script_invalid.json are loaded correctly by the
JSON interpreter used by bitcoin core, these same files are often used by other
libraries and do not necessarily load correctly due to the fact that newlines
contained inside strings are not valid and must instead use the escape
character \n. The files tx_valid.json and tx_invalid.json handle this
correctly, so I've changed the formatting in script_valid.json and
script_invalid.json to mirror those files.

Immature coinbase spends are allowed in the memory pool if they can be mined in the next block.
They are not allowed in the memory pool if they cannot be mined in the next block.

This regression test tests those edge cases.

5fdbe67 Add 0.10 release notes on improvement to signing security. (Gregory Maxwell)

32b93a1 Extra explanation for getchaintips (Pieter Wuille)

c5b390b Make comments in main an init doxygen compatible (Michael Ford)

9e56532 Coinbases-in-mempool regression test (Gavin Andresen)

d227011 MOVEONLY: core/ -> primitives/ (Luke Dashjr)

libqt5core5 installs failed.
Package libqt5core5 has replaced with libqt5core5a in ubuntu and debian. 
The website is:
http://packages.ubuntu.com/trusty/libqt5core5a
https://packages.debian.org/sid/libqt5core5a

OP_CODESEPARATOR is an actual executed instruction, not a declarative
thing, so if it's wrapped in an OP_IF it can be turned off.

Using this to implement Rivest's Paywords is left as an exercise for the
reader.

be7b0bf Package libqt5core5 replaced with libqt5core5a (LongShao007)

307f7d4 Report script evaluation failures in log and reject messages (Pieter Wuille)

12a49ca Limit the number of new addressses to accumulate (Pieter Wuille)

199627c [REST] adding basic documentation (Jonas Schnelli)

60d1ecd change nSubsidy's type from int64_t to CAmount (HarryWu)

8487790 bitcoin-tx: Add the "-txid" option. Also add the hex-encoded transaction to the JSON output as the "hex" property. (mruddy)

87bddb7 Squashed 'src/secp256k1/' changes from ad2028f..b0210a9 (Pieter Wuille)
ffe3291 Remove gmp dependency and doc mentions (Pieter Wuille)

35e408f Regard connection failures as attempt for addrman (Wladimir J. van der Laan)

8a20cd3 prioritisetransaction RPC: Restore compatibility with existing implementations by using satoshis for fee offset rather than BTC (Luke Dashjr)

afd4b94 Move CMerkleBlock and CPartialMerkleTree to their own file (Matt Corallo)

No return at the end of void function.

db0916b Fix missing python2 in rpc-tests (Matt Corallo)

Signed-off-by: Gavin Andresen <gavinandresen@gmail.com>

TLS is subject to downgrade attacks when SSLv3 is available, and
 SSLv3 has vulnerabilities.

The popular solution is to disable SSLv3. On the web this breaks
 some tiny number of very old clients. While Bitcoin RPC shouldn't
 be exposed to the open Internet, it also shouldn't be exposed to
 really old SSL implementations, so it shouldn't be a major issue
 for us to disable SSLv3.

There is more information on the downgrade attacks and disabling
 SSLv3 at https://disablessl3.com/ .

- Add a space after the fixed string prepended to file names when input or
  output file changes
- Clarify the error message when the genesis block is not found in the
  hash list (...why do we have this at all?)

a26bf47 Add a new test for FormatParagraph (string longer than the default width). (Pavel Janík)
83b81f6 Format paragraphs properly - count the space between words. (Pavel Janík)

11d7a7d [RPC] add rpc-test for http keep-alive (persistent connections) (Jonas Schnelli)

683dc40 Disable SSLv3 (in favor of TLS) for the RPC client and server. (Gregory Maxwell)

- the function only returned true, so make it void
- add a comment about payment request network detection

- also rename current ones to match the new ones
- remove constant from guiconstant.h and add it to paymentserver.cpp

- current code only does this for payment request files, which are
  used on Mac
- also rename readPaymentRequest to readPaymentRequestFromFile, so it's
  obvious that function only handles payment request files and not URIs
- small logging changes in readPaymentRequestFromFile

- this test required to make readPaymentRequestFromFile() public in order
  to be able to is it in paymentservertests.cpp

This still leaves transactions in mempool that are potentially
invalid if the maturity period has been reorged out of, but at
least they're not missing inputs entirely.

65b0328 newlines in strings are invalid JSON (Ryan X. Charles)

5ec654b [Qt] update paymentserver license and cleanup ordering (Philip Kaufmann)
4333e26 [Qt] add BIP70 DoS protection test (Philip Kaufmann)
31f8494 [Qt] add BIP70 payment request size DoS protection for URIs (Philip Kaufmann)
2284ccb [Qt] remove dup lock that is done in SetAddressBook() (Philip Kaufmann)
1ec753f [Qt] ensure socket is set to NULL in PaymentServer::ipcSendCommandLine (Philip Kaufmann)
814429d [Qt] add BIP70/BIP71 constants for all messages and mime types (Philip Kaufmann)
b82695b [Qt] make PaymentServer::ipcParseCommandLine void (Philip Kaufmann)

1577df9 Port of walletbackup.sh to Python. (mrbandrews)

6e6a36c contrib: show pull # in prompt for github-merge script (Wladimir J. van der Laan)

- rest block request returns full unfolded tx details
- /rest/block/notxdetails/<HASH> returns block where transactions are only represented by its hash

4c69ebe Add /opt/local/include/db48 only if it exists. (Pavel Janík)

4be639e Use RPC_INVALID_PARAMETER instead of RPC_WALLET_ERROR for invalid amount. No return at the end of void function. (Pavel Janík)
b93173d Move SendMoney() to rpcwallet.cpp. (Pavel Janík)

932ef50 [REST] JSON output: remove block infos from tx details if it is nested in block (Jonas Schnelli)
cae5486 [REST] added /rest/block/notxdetails/<hash> into REST-interface.md documentation (Jonas Schnelli)
73351c3 [REST] /rest/block response with full tx details (Jonas Schnelli)

34318d7 RPC-test based on invalidateblock for mempool coinbase spends (Gavin Andresen)
7fd6219 Make CTxMemPool::remove more effecient by avoiding recursion (Matt Corallo)
b7b4318 Make CTxMemPool::check more thourough by using CheckInputs (Matt Corallo)
723d12c Remove txn which are invalidated by coinbase maturity during reorg (Matt Corallo)
868d041 Remove coinbase-dependant transactions during reorg. (Matt Corallo)

63d1ae5 Do all block index writes in a batch (Pieter Wuille)

ebdd6bf Remove genesis block hash from error (Micha)
8d174d6 Make several small string fixes in linearize-data.py (Micha)

This is a check that is mentioned in BIP 37, but never implemented in the
reference code. As Bitcoin Core so far never decodes partial merkle trees,
this is not a problem. But perhaps others use the code as a reference.

Since the last commit will force rebuilds of all depends, take the opportunity
to clean up a few other things that would trigger rebuilds as well.

- Move source stamps to the sources dir so that SOURCES_PATH is respected for
  "make download".
- Only print "fetching..." when actually downloading a file.
- Avoid using non-deterministic paths for the recipe hash (patch location).
  This should ensure that all builders get the same resulting build-ids.
- Use a per-package source paths. This will allow for removing old source files
  in the future.
- Use a host-agnostic path for downloads which gets cleaned up properly.

Also fixes a nasty bug that removes the downloaded comparisontool jar file.

Add instructions for manually fetching sources, as well as some misc. fixes.

830ee48 Update Bitcoin for libsecp256k1 API change (Pieter Wuille)
ecae2ac Squashed 'src/secp256k1/' changes from b0210a9..bccaf86 (Pieter Wuille)

e27d7cb docs: release process fixups (Cory Fields)
d7db4b6 depends: fix packages with hard-coded SOURCES_PATH (Cory Fields)
d546191 depends: Move source stamps to source dir and misc cleanups (Cory Fields)
422f873 depends: teach 'make download' to download sources for all hosts (Cory Fields)

aa768f1 signrawtransaction: validate private key. (Pavel Janík)

5ea3bc0 Move remaining CBlockIndex methods to chain.cpp (Pieter Wuille)
9dcd524 Make IsSuperMajority a standalone function (Pieter Wuille)

164a45f Fix download link for gitian builds (BtcDrak)

'Sane' was already defined by this code as:
 fee.GetFeePerK() > minRelayFee.GetFeePerK() * 10000
 But sanity was only enforced for data loaded from disk.

Note that this is a pretty expansive definition of 'sane': A 10 BTC
 fee is still passes the test if its on a 100kb transaction.

This prevents a single insane fee on the network from making us reject
 our stored fee data at start.  We still may reject valid saved fee
 state if minRelayFee is changed between executions.

This also reduces the risk and limits the damage from a cascading
 failure where one party pays a bunch of insane fees which cases
 others to pay insane fees.

87e8e1c minor fix on script test data docs (Manuel Araoz)

f676c80 Add /rest/headers (Pieter Wuille)

31aac02 Qt: new icons (Jonas Schnelli)

adds possibility to run ./qa/pull-tester/rpc-tests.sh <TESTNAME>

- remove splash screen images (reduce binary size)
- dynamicly draw splash screen with available icon.
- remove testnet icon
- dynamicly colorize testnet icon

ac23394 resize tray icon because a 1024x1024 icon won't show in ubuntu (bug) (Jonas Schnelli)
8e76ca0 Qt: Go back to using QIcon functionality for scaling (Wladimir J. van der Laan)
54f2571 Qt: HiDPI (retina) support for splash screen (Jonas Schnelli)

1340f00 [RPC-TESTS] ability to run single test manually (Jonas Schnelli)

Rebased-From: 67b2d819cdf6181e7f016e5366ce7479830893bd
Github-Pull: #5404

4a8fc15 [Qt] the RPC Console should be a QWidget to make window more independent (Jonas Schnelli)

270f42d [Qt] fix a <Qt4.7 compatibility issue raised in #5228 (Jonas Schnelli)

Changing the icons color for regtest in now lightweight.

Make some of the arguments in rest.cpp, that are passed by
reference but never modified, const to emphasise that.

ff63801 [Qt] light blue icon color for regtest (Jonas Schnelli)

- I saw this on http://en.cppreference.com/w/cpp/language/try_catch and
  thought it would be a good idea
- also unify used format to better be able to search for exception
  uses in our codebase

Besides giving a nicer error, this also prevents logging arbitrary data (which could have been used to exploit log readers) into debug.log

27df412 make all catch() arguments const (Philip Kaufmann)

7c8e4c5 Make pass-by-ref arguments const. (Daniel Kraft)

Moves coding guidelines and development tips/tricks into a single file.
Also adds a section explaining pull request terminology.

e36e619 Add mips, mipsel and aarch64 to depends platforms (Wladimir J. van der Laan)

The comment has been around since 0.1, but NTP inside the client
isn't deemed necessary so remove the comment to avoid confusion.

7a3b832 [QA] fix httpbasics RPC test typo (Jonas Schnelli)

e85855f Drop beta suffixes from readmes (Michael Ford)

cac15be Test unexecuted OP_CODESEPARATOR (Peter Todd)

88eb44d Revised translation process doc (Blake Jakopovic)

4709160 [Qt] fix a translation that occurs 2 times but was a little different (Philip Kaufmann)
50db7d9 [Qt] change some strings to reflect name change to Bitcoin Core (Philip Kaufmann)
2747f7c [Qt] fix missing plural form for a string in sendcoinsdialog (Philip Kaufmann)

eef747b libbitcoinconsensus: Add pkg-config support (Luke Dashjr)

Rebased-From: ccd056a30db0ba5e04b858a682907b378e464d34
Github-Pull: #5437

ba7fcc8 Discourage fee sniping with nLockTime (Peter Todd)

78253fc Remove references to X11 licence (Michael Ford)

Github-Pull: #5494
Rebased-From: 15de949bb9277e442302bdd8dee299a8d6deee60

Previously the minRelayTxFee was only enforced on user specified values.

It was possible for smartfee to produce a fee below minRelayTxFee which
 would just result in the transaction getting stuck because it can't be
 relayed.

This also introduces a maxtxfee option which sets an absolute maximum
 for any fee created by the wallet, with an intention of increasing
 user confidence that the automatic fees won't burn them. This was
 frequently a concern even before smartfees.

If the configured fee policy won't even allow the wallet to meet the relay
 fee the transaction creation may be aborted.

Rather than treating the .py as a regular built test, run it from check-local
so that we can better control the paths used.

- Exit when there is no signing key configured
- Exit when user does not want to sign off

7c041b3 Check against MANDATORY flags prior to accepting to mempool (Peter Todd)

4bc95c4 remove max orphan blocks config parameter since it is no longer functional (Jameson Lopp)

1e22d45 Add libbitcoinconsensus.pc to .gitignore (Michael Ford)

Don't allow immediate inv driven block downloads if
a peer already has MAX_BLOCKS_IN_TRANSIT_PER_PEER
active downloads.  Prevents bogus inv spam from
blowing up block transfer tracking data structures.

listsinceblock, listtransactions, listlockunspent, listaccounts
listaddressgroupings

39c8099 contrib: make github-merge require signing (Wladimir J. van der Laan)

3497022 Reject headers that build on an invalid parent (Pieter Wuille)

aa279d6 Enforce minRelayTxFee on wallet created tx and add a maxtxfee option. (Gregory Maxwell)

6484930 Apply AreSane() checks to the fees from the network. (Gregory Maxwell)

c907704 DOS: Respect max per-peer blocks in flight limit (Adam Weiss)

dd14a0e docs: update gitian building docs (Cory Fields)

Previously an empty script wouldn't be hashed, and CScriptID would be
assigned the incorrect value of 0 instead. This bug can be seen in the
RPC decodescript command:

    $ btc decodescript ""
    {
        "asm" : "",
        "type" : "nonstandard",
        "p2sh" : "31h1vYVSYuKP6AhS86fbRdMw9XHieotbST"
    }

Correct output:

    $ btc decodescript ""
    {
        "asm" : "",
        "type" : "nonstandard",
        "p2sh" : "3J98t1WpEZ73CNmQviecrnyiWrnqRhWNLy"
    }

d78f0da Fix CScriptID(const CScript& in) in empty script case (Peter Todd)

566c6cb gitian: attempt to fix tarball determinisim (Cory Fields)

a15dba5 en: Avoid ambiguous language regarding when transactions confirm (Luke Dashjr)

13f9031 init: minor parameter interaction updates (Philip Kaufmann)

I'm not comfortable with retaining this entry.

- Make sure it represents the current state

de779b3 [Qt] new icon for the debug window (Jonas Schnelli)

9b7d3fb Adopt style colour for button icons (Luke Dashjr)

The text talked about "Quit" while Bitcoin uses "Exit" in its menu.

99913f0 Fix reference. (Thomas Zander)

244fdc7 Fix typo (Julian Yap)

7c5dd60 Adding RPC tests for the following wallet related calls: getbalance, listsinceblock, listtransactions, listlockunspent, listaccounts listaddressgroupings (Everett Forth)

be43492 build: fix 'make distcheck' (Cory Fields)

The width of each unit is measured to set a fixed width for the widget, and the color is set to match the other status widgets.

469d564 Build winshutdownmonitor.cpp on Windows only. (Pavel Janík)

Changed 2014 to 2015 in preparation for new year

8db1760 rpcserver: attempt to fix uncaught exception. (Cory Fields)

It is easily confused with CCoinsViewCache(CCoinsView*), which creates a sub-cache, but instead of creating a sub-cache, the copy constructor would copy the original and use that original's base, defeating the intended isolation.

78a7cd3 Updated license date (Ben Holden-Crowther)

8f6860a Bugfix: RPCWallet: Docs: Booleans aren't quoted (Luke Dashjr)

1f67084 [Docs] clean up assets-attributen.md (Jonas Schnelli)

06206bb Correct tooltip on address book page (Michael Ford)

a094b3d Remove bitnodes.io from dnsseeds. (Gregory Maxwell)

7f71813 Bugfix: prioritisetransaction: Do some basic sanity checking on txid (Luke Dashjr)

branch 'core2' of git://github.com/jtimon/bitcoin into merge-5111

branch 'validateaddress-return-scriptpubkey' of git://github.com/petertodd/bitcoin into merge-4964

branch 'developer-notes_md' of git://github.com/fanquake/bitcoin into merge-5468

branch 'error_typos' of git://github.com/paveljanik/bitcoin into merge-5578

branch 'bitcoin-tx-copy-paste' of git://github.com/ers35/bitcoin into merge-5528

Broken hash logic caused all depends on some platforms (osx at least) to end up
with the same build-id. Without this fix, nothing will be rebuilt when recipes
or dependencies change.

dd3f697 Use real text rather than image-rendered text for unit selector (Luke Dashjr)

The normative place for these for the entire project is COPYING,
and the main README already has a MIT license section.

There is still a redundancy here between configure.ac and
clientversion.h.

d57b303 depends: fix major regression after d546191dc. (Cory Fields)

643415a [Qt] update a translation string and argument counts (Philip Kaufmann)

0d50c2f dmg: fix deterministic dmg creation and docs (Cory Fields)

Also make ProcessBlockFound static as it is not used outside
miner.cpp.

Alternative implementation of #5549.

1b178a7 Bugfix: ConnectBlock: In case the genesis block gets in with fJustCheck, behave correctly (Luke Dashjr)
228d238 Make CCoinsViewCache's copy constructor private (Luke Dashjr)

73caf47 Display time offset in the debug window's Peers tab (Pavel Janík)
26a6bae Add time offset to getpeerinfo output (Pavel Janík)

0125988 Implement test for merkle tree malleability in CPartialMerkleTree (Pieter Wuille)

tl;dr: Update to the newer stable toolchain and SDK for OSX without giving up
any backwards compatibility. We can move to clang 3.5 as a next step which
allows use to use libc++ and the 10.10 sdk, but we'll need to find a build that
works in gitian/travis first.

Switch to a new, better maintained fork of cctools:
https://github.com/tpoechtrager/cctools-port

I've forked this and will be working on it some as well:
https://github.com/theuni/cctools-port

This brings in:
cctools v862
ld64: v241.9

It also fixes 64bit builds, so there's no longer any need to use a 32bit clang.
Since clang is no longer tied to an old/crusty 32bit build, clang has been
upgraded to 3.3. Unfortunately, there's a bug in 3.4 that breaks builds. 3.5
works fine, but there are no binary builds compatible with precise, which is
currently used for gitian and travis. We could always build our own if
necessary.

After updating to stable clang/linker/cctools, it's possible to use a more
recent SDK. The current SDK (10.7) through the most recent 10.10 have all been
built/tested successfully, both with and without 10.6 compatibility. However,
10.10 requires clang 3.5.

SDKs >= 10.9 use libc++ rather than libstdc++. This is verified working as well.

This one no longer requires native libuuid or libcrypto

With the splashscreen being able to be closed it is possible to
shutdown during the lengthy verifyDB method. (Takes about a minute
on my machine). This change allows us to shutdown much sooner.

Github-Pull: #5557

4caad26 depends: bump cctools to a custom version with less deps (Cory Fields)
46f54bf build: osx builders no longer need 32bit compiler support (Cory Fields)
4fe6c3c depends: major upgrade to darwin toolchain (Cory Fields)
ec90c97 depends: osx: fix qt5 build against 10.10 sdk (Cory Fields)

146c0a7 Add RandAddSeedPerfmon to MakeNewKey (21E14)

12d927a RPC test for immature balance (Jonas Schnelli)
8024d67 Add immature balances to getwalletinfo. (Gregory Maxwell)
d44c545 Add unconfirmedbalance field to getwalletinfo (azeteki)

7eeeac0 Remove dead BitcoinUnits::id code and update assets-attribution for non-image unit selector (Luke Dashjr)

Also add a stub for arith_uint256 and its conversion functions,
for now completely based on uint256.

Eases step-by-step migration to blob.

Replace x=0 with .SetNull(),
x==0 with IsNull(), x!=0 with !IsNull().
Replace uses of uint256(0) with uint256().

SignatureHash and its test function SignatureHashOld
return uint256(1) as a special error signaling value.
Return a local static constant with the same value instead.

If uint256() constructor takes a string, uint256(0) will become
dangerous when uint256 does not take integers anymore (it will go
through std::string(const char*) making a NULL string, and the explicit
keyword is no help).

Also add conversion from/to uint256 where needed.

Introduce new opaque implementation of `uint256`, move old
"arithmetic" implementation to `arith_uint256.

We never do 160-bit arithmetic.

- Methods that access the guts of arith_uint256 are removed,
as these are incompatible between endians. Use uint256 instead

- Serialization is no longer needed as arith_uint256's are never
read or written

- GetHash is never used on arith_uint256

Remove initialization from vector (as this is only used in the tests).

Also implement SetHex and GetHex in terms of uint256, to avoid
duplicate code as well as avoid endianness issues (as they
work in term of bytes).

6bd0dc2 arith_uint256: remove initialization from byte vector (Wladimir J. van der Laan)
30007fd Remove now-unused methods from arith_uint256 and base_uint (Wladimir J. van der Laan)
edc7204 Remove arith_uint160 (Wladimir J. van der Laan)
dba2e91 Add tests for new uint256 (Wladimir J. van der Laan)
92cdb1a Add conversion functions arith_uint256<->uint_256 (Wladimir J. van der Laan)
bfc6070 uint256->arith_uint256 blob256->uint256 (Wladimir J. van der Laan)
734f85c Use arith_uint256 where necessary (Wladimir J. van der Laan)
34cdc41 String conversions uint256 -> uint256S (Wladimir J. van der Laan)
2eae315 Replace uint256(1) with static constant (Wladimir J. van der Laan)
8076585 Replace GetLow64 with GetCheapHash (Wladimir J. van der Laan)
4f15249 Replace direct use of 0 with SetNull and IsNull (Wladimir J. van der Laan)
5d3064b Temporarily add SetNull/IsNull/GetCheapHash to base_uint (Wladimir J. van der Laan)