Fix attempt Msvc wip #26987

Suppress mcl false positives msan

Introduce generic arith classes (on up-to-date master)

* doc: add info about status code 404 for some rest endpoints

* refactor: remove duplicate code from BlockAssembler

* build: Fix `capnp` package build for Android

* build: Specify native binaries explicitly when building `capnp` package

From `configure --help`:
  --with-external-capnp   use the system capnp binary (or the one specified
                          with $CAPNP) instead of compiling a new one (useful
                          for cross-compiling)

* net: remove useless call to IsReachable() from CConnman::Bind()

`CConnman::Bind()` is called without `BF_EXPLICIT` only when passed
either `0.0.0.0` or `::`. For those addresses `IsReachable()` is always
true (regardless of the `-onlynet=` setting!), meaning that the `if`
condition never evaluates to true.

`IsReachable()` is always true for the "any" IPv4 and IPv6 addresses
because `CNetAddr::GetNetwork()` returns `NET_UNROUTABLE` instead of
`NET_IPV4` or `NET_IPV6` and the network `NET_UNROUTABLE` is always
considered reachable.

It follows that `BF_EXPLICIT` is unnecessary, remove it too.

* sync: simplify MaybeCheckNotHeld() definitions by using a template

Reduce 4 of the `MaybeCheckNotHeld()` definitions to 2 by using a
template. This also makes the function usable for other
[BasicLockable](https://en.cppreference.com/w/cpp/named_req/BasicLockable)
types.

* sync: remove unused template parameter from ::UniqueLock

The template parameter `typename Base = typename Mutex::UniqueLock` is
not used, so remove it. Use internally defined type `Base` to avoid
repetitions of `Mutex::UniqueLock`.

* scripted-diff: Rename time symbols

-BEGIN VERIFY SCRIPT-

 ren() { sed -i "s:\<$1\>:$2:g" $(git grep -l "\<$1\>" ':(exclude)src/versionbits.cpp') ; }

 ren nStart                 time_start
 ren nTimeStart             time_start
 ren nTimeReadFromDiskTotal time_read_from_disk_total
 ren nTimeConnectTotal      time_connect_total
 ren nTimeFlush             time_flush
 ren nTimeChainState        time_chainstate
 ren nTimePostConnect       time_post_connect
 ren nTimeCheck             time_check
 ren nTimeForks             time_forks
 ren nTimeConnect           time_connect
 ren nTimeVerify            time_verify
 ren nTimeUndo              time_undo
 ren nTimeIndex             time_index
 ren nTimeTotal             time_total
 ren nTime1                 time_1
 ren nTime2                 time_2
 ren nTime3                 time_3
 ren nTime4                 time_4
 ren nTime5                 time_5
 ren nTime6                 time_6

 ren nBlocksTotal num_blocks_total

 # Newline after semicolon
 perl -0777 -pi -e 's/; time_connect_total/;\n        time_connect_total/g' src/validation.cpp
 perl -0777 -pi -e 's/; time_/;\n    time_/g'                               src/validation.cpp

-END VERIFY SCRIPT-

* Use steady clock for bench logging

* doc: add historical 0.20.2 release notes

* doc: add historical 0.21.2 release notes

* build: split ARM crc & crypto extension checks

We currently perform the same check twice, to put the same set of flags
in two different variables. Split the checks so we test for crc and crypto
extensions independently.

If we don't want to split, we should just delete the second AX_CHECK_COMPILE_FLAG
check, and set ARM_CRC_CXXFLAGS & ARM_CRC_CXXFLAGS at the same time.

* refactor: Do not discard `try_lock()` return value

Microsoft's C++ Standard Library uses the `[[nodiscard]]` attribute for
`try_lock()`.
See: https://github.com/microsoft/STL/blob/main/stl/inc/mutex

This change allows to drop the current suppression for the warning C4838
and helps to prevent the upcoming warning C4858.
See: https://github.com/microsoft/STL/commit/539c26c923b38cd0b5eba2bb11de4bea9d5c6e43

* signet/miner: reduce default interblock interval limit to 30min

Also allow the operator to change it, if desired, without having
to edit the code.

* Squashed 'src/leveldb/' changes from 22f1e4a02f..e2f10b4e47

e2f10b4e47 Merge bitcoin-core/leveldb-subtree#34: win32: fix -Wmissing-field-initializers warnings
12c52b392d win32: fix -Wmissing-field-initializers warnings

git-subtree-dir: src/leveldb
git-subtree-split: e2f10b4e47bc950a81bc96d1c6db3a8048216642

* ci: Bump vcpkg to the latest version `2022.09.27`

Dependency changes (2022.06.16.1 - 2022.09.27):
 - boost 1.79.0#0 -> 1.80.0#0
 - sqlite3 3.37.2#1 -> 3.39.2#0
 - zeromq 4.3.4#5 -> 4.3.4#6

* contrib: Fix capture_output in getcoins.py

Our required Python version 3.6.12 does not support `capture_output` as
a subprocess.run argument; this was added in python 3.7.

We can emulate it by setting stdout and stderr to subprocess.PIPE

* fuzz: Limit outpoints.size in txorphan target to avoid OOM

* refactor: move Boost datetime usage to wallet

This means we don't need datetime in a --disable-wallet build, and it
isn't included in the kernel.

* wallet: Use correct effective value when checking target

* test: Check external coin effective value is used in CoinSelection

* test: Use proper Boost macros instead of assertions

* ci: Run `bench_bitcoin.exe --sanity-check` in "Win64 native" task

Also a better name used for the script as it follows GNU's `make check`.

* doc: bump bips.md up-to-date version to v24.0

This is a trivial follow-up to #26124.

* refactor: Drop `owns_lock()` call

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>

* refactor: move DEFAULT_TXINDEX from validation to txindex

* refactor: move DEFAULT_COINSTATSINDEX from validation to coinstatsindex

* refactor: move DEFAULT_BLOCKFILTERINDEX from val to blockfilterindex

* kernel: remove util/bytevectorhash.cpp

* ci: Move `git config` commands into script where they are used

* ci: Use same `merge_script` implementation for Windows as for all

* ci: Remove unused package

Address feedback from https://github.com/bitcoin/bitcoin/pull/24561/files#r985719812

* ci: Allow PIP_PACKAGES on centos

This was added in 7fc5e865b93af59364e9c8bf75ec68b4decc7e5d but I can't
see a reason why this should be forbidden.

* test: Remove unused fCheckpointsEnabled from miner_tests

The earliest checkpoint is at height 11111, so this can't possibly have
any impact on this test.

* build, msvc: Enable C4834 warning

https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/c4834

* kernel: move RunCommandParseJSON to its own file

Because libbitcoinkernel does not include this new object, this has the
side-effect of eliminating the unnecessary boost::process dependency.

* ci: Workaround Windows filesystem executable bit loss

* fuzz: add util/mempool/h.cpp

Moving the mempool code (Boost) out of util.h, results in a ~10% speedup
(for me) when compiling the fuzz tests.

* fuzz: pass max fee into ConsumeTxMemPoolEntry

* refactor: Make 64-bit shift explicit

Also this change enables MSVC warning C4334 for all codebase.

https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-3-c4334

* refactor: move run_command from util to common

Quoting ryanofsky: "util can be the library for things included in the kernel
which the kernel can depend on, and common can be the library for other code
that needs to be shared internally, but should not be part of the kernel or
shared externally."

* Remove clang-format from lint task

clang-format could be used in scripted diffs, but remained largely
unused.

* test: Pass mempool reference to AssemblerForTest

* test: Use dedicated mempool in TestPrioritisedMining

No need for a shared mempool. Also remove unused chainparams parameter.

* test: Use dedicated mempool in TestPackageSelection

No need for a shared mempool. Also remove unused chainparams parameter.

* test: Use dedicated mempool in TestBasicMining

No need for a shared mempool. Also remove unused chainparams parameter.

Can be reviewed with --ignore-all-space

* refactor: mempool: add MemPoolLimits::NoLimits()

There are quite a few places in the codebase that require us to
construct a CTxMemPool without limits on ancestors and descendants.
This helper function allows us to get rid of all that duplication.

* refactor: mempool: use CTxMempool::Limits

Simplifies function signatures by removing repetition of all the
ancestor/descendant limits,  and increases readability by being
more verbose by naming the limits, while still reducing the LoC.

* test: use NoLimits() in MempoolIndexingTest

The (100, 1000000, 1000, 1000000) limits are arbitrarily high and
don't restrict anything, they are just meant to calculate ancestors
properly. Using NoLimits() makes this intent more clear and simplifies
the code.

* docs: improve docs where MemPoolLimits is used

* Remove unused CDataStream::rdbuf method

It is unused and seems unlikely to be ever used.

* index: Improve BaseIndex::BlockUntilSyncedToCurrentChain reliability

Since commit f08c9fb0c6a799e3cb75ca5f763a746471625beb from PR
https://github.com/bitcoin/bitcoin/pull/21726, index
`BlockUntilSyncedToCurrentChain` behavior has been less reliable, and there has
also been a race condition in the `coinstatsindex_initial_sync` unit test.

It seems better for `BlockUntilSyncedToCurrentChain` to actually wait for the
last connected block to be fully processed, than to be able to return before
prune locks are set, so this switches the order of `m_best_block_index =
block;` and `UpdatePruneLock` statements in `SetBestBlockIndex` to make it more
reliable.

Also since commit f08c9fb0c6a799e3cb75ca5f763a746471625beb, there has been a
race condition in the `coinstatsindex_initial_sync` test. Before that commit,
the atomic index best block pointer `m_best_block_index` was updated as the
last step of `BaseIndex::BlockConnected`, so `BlockUntilSyncedToCurrentChain`
could safely be used in tests to wait for the last `BlockConnected`
notification to be finished before stopping and destroying the index. But
after that commit, calling `BlockUntilSyncedToCurrentChain` is no longer
sufficient, and there is a race between the test shutdown code which destroys
the index object and the new code introduced in that commit calling
`AllowPrune()` and `GetName()` on the index object. Reproducibility
instructions for this are in
https://github.com/bitcoin/bitcoin/issues/25365#issuecomment-1259744133

This commit fixes the `coinstatsindex_initial_sync` race condition, even though
it will require an additional change to silence TSAN false positives,
https://github.com/bitcoin/bitcoin/pull/26188, after it is fixed. So this
partially addresses but does not resolve the bug reporting TSAN errors
https://github.com/bitcoin/bitcoin/issues/25365.

There is no known race condition outside of test code currently, because the
bitcoind `Shutdown` function calls `FlushBackgroundCallbacks` not
`BlockUntilSyncedToCurrentChain` to safely shut down.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Co-authored-by: MacroFake <falke.marco@gmail.com>

* test: Prevent UB in `minisketch_tests.cpp`

* test: Remove confusing DUMMY_P2WPKH_SCRIPT

* docs: fix m_children to be a member of CTxMemPoolEntry

* wallet: have prune error take precedence over assumedvalid

From Russ Yanofsky:

"Agree with all of Marco's points here and think this should be updated

If havePrune and hasAssumedValidChain are both true, better to show
havePrune error message.  Assumed-valid error message is vague and not
very actionable.  Would suggest "Error loading wallet. Wallet requires
blocks to be downloaded, and software does not currently support loading
wallets while blocks are being downloaded out of order though assumeutxo
snapshots. Wallet should be able to load successfully after node sync
reaches height {block_height}"

Co-authored-by: MacroFake <MarcoFalke@gmail.com>
Co-authored-by: Russ Yanofsky <russ@yanofsky.org>

* Fix comment typos in scriptpubkeyman.cpp, wallet.cpp, wallet.h

Fix comment typos:
sigature -> signature
ponter -> pointer
it's key -> its key

* sync: avoid confusing name overlap (Mutex)

Use `MutexType` instead of `Mutex` for the template parameter of
`UniqueLock` because there is already a class named `Mutex` and the
naming overlap is confusing. `MutexType` is used elsewhere in `sync.h`.

* sync: remove DebugLock alias template

Use `UniqueLock` directly. Type deduction works just fine from the first
argument to the constructor of `UniqueLock`, so there is no need to
repeat

```cpp
UniqueLock<typename std::remove_reference<typename std::remove_pointer<decltype(cs)>::type>::type>
```

five times in the `LOCK` macros. Just `UniqueLock` suffices.

* sync: rename AnnotatedMixin::UniqueLock to AnnotatedMixin::unique_lock

This avoids confusion with the global `UniqueLock` and the snake case
is consistent with `UniqueLock::reverse_lock.

* iwyu: Add zmq source files

* Update .cirrus.yml

Co-authored-by: brunoerg <brunoely.gc@gmail.com>
Co-authored-by: James O'Beirne <james.obeirne@pm.me>
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Co-authored-by: MacroFake <falke.marco@gmail.com>
Co-authored-by: fanquake <fanquake@gmail.com>
Co-authored-by: Anthony Towns <aj@erisian.com.au>
Co-authored-by: willcl-ark <will8clark@gmail.com>
Co-authored-by: Aurèle Oulès <aurele@oules.com>
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
Co-authored-by: glozow <gloriajzhao@gmail.com>
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
Co-authored-by: stickies-v <stickies-v@protonmail.com>
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: MacroFake <MarcoFalke@gmail.com>
Co-authored-by: Russ Yanofsky <russ@yanofsky.org>
Co-authored-by: Dimitris Tsapakidis <dimitris@tsapakidis.com>
Co-authored-by: alex v <alex@encrypt-s.com>

* script, test: fix python linter E275 errors with flake8 5.0.4

* script: update python linter dependencies

* wallet: simplify ListCoins implementation

Can remove the locked coins lookup if we include them directly
inside the AvailableCoins result

* wallet: unify FindNonChangeParentOutput functions

The function is only used in ListCoins.

* Use same Python executable for subprocesses as for all-lint.py

* tests: Use unique port for ZMQ tests

The ZMQ interface tests should use unique ports as we do for the p2p and
rpc ports so that multiple instances of the test can be run at the same
time.

* test: add coverage for cross-chain wallet restore

* refactor: use braced init for integer constants instead of c style casts

* rpc: Sanitize label name in various RPCs

- importprivkey
- importaddress
- importpubkey
- listtransactions
- listsinceblock
- importmulti
- importdescriptors

* compat: use STDIN_FILENO over 0

This is already used throughout this file, and is self-documenting.

* debian: remove nonexistent files from copyright

* refactor: Add sanity checks in LabelFromValue

* test: Invalid label name coverage

* refactor: use convenience fn to auto parse non-string parameters

Minimizes code duplication and improves function naming by having
a single (overloaded) convenience function that both checks if
the parameter is a non-string parameter and automatically parses the
value if so.

* refactor: add kernel/cs_main.*

Co-authored-by: Anthony Towns <aj@erisian.com.au>

* test: Fix feature_startupnotify intermittent issue

* refactor: Make `ThreadHTTP` return void

The `bool` return value was introduced in 755aa05174e06effd758eeb78c5af9fb465e9611.

It has been not used since 8d3f46ec3938e2ba17654fecacd1d2629f9915fd.

No behavior change.

* build: remove already tested headers from AC_CHECK_HEADERS

These headers are already included in a default set which are checked
early during configure.

We already use at least sys/types.h and unistd.h unconditionally in
configure.

* refactor: Work around Werror=free-nonheap-object in AssumeCalculateMemPoolAncestors

* build: fix configuring with only bitcoin-util

Fixes the issue presented in #25037 in a single (easily backportable)
diff, with no additional refactoring/changes.

Can be tested with:
```bash
./configure \
  --disable-tests \
  --disable-bench \
  --without-libs \
  --without-daemon \
  --without-gui \
  --disable-fuzz-binary \
  --without-utils \
  --enable-util-util
```

* random: remove windows-only compat.h include in randomenv

Note that this was probably only here to indirectly receive windows.h
via another include in compat.h (windows.h or winreg.h aren't included
there).

Also note that compat.h is already pulled in here for everyone via
util/time.h, so including inside a windows only ifdef is secondarily
redundant.

* randomenv: consolidate WIN32 #ifdefs

Order includes.
Remove // for xyz comments

* test: Add signs P2TR and RAWSCRIPT to MiniWallet sign_tx

* test: Add "include immature coinbase" flag to MiniWallet get_utxos

* test: Add "include mempool" flag to MiniWallet rescan_utxos

* test: Run feature_bip68_sequence.py with MiniWallet

* doc: Update Boost version in doc/dependencies.md

* scripts: add PE Canary check to security-check

* rpc: Return accurate results for scanblocks

This makes use of undo data to accurately verify results
from blockfilters.

* p2p, rpc: don't allow past absolute timestamp in `setban`

* test: add coverage for absolute timestamp in `setban`

* [validation] when quitting early in AcceptPackage, set package_state and tx result

Bug: not setting package_state means package_state.IsValid() == true and
the caller does not know that this failed.

We won't be validating this transaction again, so it makes sense to return this
failure to the caller.

Rename package_state to package_state_quit_early to make it more clear
what this variable is used for and what its scope is.

Co-authored-by: Greg Sanders <gsanders87@gmail.com>

* [test] package validation quits early due to non-policy, non-missing-inputs failure

* [validation] return effective feerate from mempool validation

* ci: Fix ci_native_fuzz_msan CONTAINER_NAME

* ci: Create named symbol for BINS_SCRATCH_DIR

Also, create the dir a bit earlier, right after it was put in the PATH.

* ci: Remove unused busybox workaround

The find workaround is no longer needed after commit
d3d547c545021d8339db666d36e48f10ea478f9c

* ci: Run one task with all tests on credits

* doc: clarify -i2pacceptincoming help documentation

and also hoist the default setting to a constexpr and
remove unused f-string operators in a related functional test.

* doc: update bandwidth section of I2P documentation

* doc: update/clarify/de-emphasize I2P transient address section

* doc: remove recommended I2P router versions

as these go stale and users will generally install the current versions available.

* [validation] return wtxids of other transactions whose fees were used

* [rpc] return effective-feerate in testmempoolaccept and submitpackage

* [rpc] return effective-includes in testmempoolaccept and submitpackage

* [validation] remove PackageMempoolAcceptResult::m_package_feerate

This value creates an extremely confusing interface as its existence is
dependent upon implementation details (whether something was submitted
on its own, etc). MempoolAcceptResult::m_effective_feerate is much more
helpful, as it always exists for submitted transactions.

* [doc] release note effective-feerate and effective-includes RPC results

No release note for submitpackage because it is regtest-only.

* [refactor] rename variables in AcceptPackage for clarity

* [validation] return MempoolAcceptResult for every tx on PCKG_TX failure

This makes the interface more predictable and useful. The caller
understands one or more transactions failed, and can learn what happened
with each transaction. We already have this information, so we might as
well return it.

It doesn't make sense to do this for other PackageValidationResult
values because:
- PCKG_RESULT_UNSET: this means everything succeeded, so the individual
  failures are no longer accurate.
- PCKG_MEMPOOL_ERROR: something went wrong with the mempool logic;
  transaction failures might not be meaningful.
- PCKG_POLICY: this means something was wrong with the package as a
  whole. The caller should use the PackageValidationState to find the
  error, rather than looking at individual MempoolAcceptResults.

* doc: net: fix link to onion address encoding scheme [ONIONADDRESS]

Instead of referring to a fixed line number to a file in master (which
is obviously always quickly outdated), use a permalink tied to the
latest commit.

* doc: fix up -netinfo relaytxes help

Co-authored-by: "MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>"

* scripted-diff: ci: Rework docker naming

DOCKER in names is confusingly used as synonym for "image", "container",
and "ci". Fix the confusion by picking the term that fits the context.

-BEGIN VERIFY SCRIPT-
 ren() { sed -i "s:$1:$2:g" $( git grep -l "$1" ) ; }

 ren DOCKER_PACKAGES CI_BASE_PACKAGES
 # This better reflects that they are the common base for all CI
 # containers.

 ren DOCKER_ID CI_CONTAINER_ID
 # This is according to the documentation of "--detach , -d: Run
 # container in background and print container ID".

 ren DOCKER_NAME_TAG CI_IMAGE_NAME_TAG
 # This avoids confusing with CONTAINER_NAME and clarifies that it is an
 # image.

 ren DOCKER_ADMIN CI_CONTAINER_CAP
 # This clarifies that it is a capability added to the container.

 ren DOCKER_CI_CMD_PREFIX CI_EXEC_CMD_PREFIX
 # This brings it in line with the CI_EXEC naming.

-END VERIFY SCRIPT-

* ci: Stop and remove CI container

* test: Fix intermittent timeout in p2p_permissions.py

* test: Fix wrong types passed to RPCs

* rpc: Run type check against RPCArgs

* doc: add databases/py-sqlite3 to FreeBSD test suite deps

* Wallet/RPC: Allow specifying min & max chain depth for inputs used by fund calls

Enables users to craft BIP-125 replacements with changes to the output
list, ensuring that if additional funds are needed they will be added.

* rpc: add minconf and maxconf options to sendall

* doc: move errant release note to doc/

* test: skip sqlite3 tests if it isn't available

Fixes #26819. Related too #26873.

* doc: remove usages of C++11

Now it's just the standard library.

* Update to mention restoring wallet via GUI

* Change dots to an ellipsis and fix capitalization

Matches ellipsis usage in the "Restore" section.

* test: remove `-spendzeroconfchange` setting from mempool_limit.py

Since this test was changed to use MiniWallet instead of the Bitcoin
Core wallet (see commit d447ded6babebe7c7948e585c9e78bf34dbef226),
the setting doesn't have any effect and hence can be removed.

* ci: Bump vcpkg to the latest version 2023.01.09

Dependency changes (2022.09.27 - 2023.01.09):
 - boost 1.80.0#0 -> 1.81.0#0
 - libevent 2.1.12#6 -> libevent 2.1.12#7
 - sqlite3 3.39.2#0 -> 3.40.0#1

* ci: Bump `ccache` version to the latest 4.7.4 in "Win64 native" task

* test: wallet: add coverage for `-spendzeroconfchange` setting

* build: allow NO_BOOST=1 in depends

* build: allow NO_LIBEVENT=1 in depends

* test: refactor: simplify p2p_permissions.py by using MiniWallet

Also, use the pre-mined chain of the test framework rather than mining
100 blocks manually on each run.

* test: Remove redundant function call

* ci: Use pyenv's `python-build` to install Python in lint task

* ci: Bump lint task image to Ubuntu Jammy

* test: add `rescan_utxos` in MiniWallet's initialization

this simplifies usage when MiniWallet is used with a pre-mined chain.

* test: remove redundant blocks generation logic

those tests already have enough mature utxos from the pre-mined chain.

* test: simplify tests by using the pre-mined chain

* test: Return wtxid from create_self_transfer_multi

This is not used right now, but may be in the future. Also, it
simplifies the create_self_transfer return logic

* test: Refactor MiniWallet sign_tx

To make the code less verbose and easier to read.

* test: Return chain of MiniWallet txs from MiniWallet chain method

* test: Return fee from MiniWallet

* test: Run mempool_packages.py with MiniWallet

* build: move rpc/request from util lib to common

This is JSON RPC request code that doesn't need to be in util, and
should not be required by the kernel.

* test: Add test for missing and omitted required arg

* doc: Fix incorrect sendmany RPC doc

This enables the type check and fixes the wrong docs.

Otherwise the enabled check would lead to test errors, such as:

> "wallet_labels.py", line 96, in run_test
>     node.sendmany(
>
> test_framework.authproxy.JSONRPCException:
>  JSON value of type null is not of expected type string (-3)

* refactor: Remove const to fix performance-move-const-arg clang-tidy errors

The warnings look like:

src/rpc/util.h:192:19: error: std::move of the const variable 'name' has no effect; remove std::move() or make the variable non-const [performance-move-const-arg,-warnings-as-errors]
        : m_names{std::move(name)},
                  ^~~~~~~~~~    ~

* refactor: Introduce is_top_level_arg

* doc: Properly report optional RPC args

* refactor: Remove duplication of clang-tidy's check names

* ci: Add missing lint dependency

* hash: add HashedSourceWriter

This class is the counterpart to CHashVerifier, in that it
writes data to an underlying source stream,
while keeping a hash of the written data.

* addrdb: Only call Serialize() once

The previous logic would call it once for serializing into the filestream,
and then again for serializing into the hasher. If AddrMan was changed
in between these calls by another thread, the resulting peers.dat would
be corrupt with non-matching checksum and data.
Fix this by using HashedSourceWriter, which writes the data
to the underlying stream and keeps track of the hash in one go.

* ci: Bump --combinedlogslen to debug intermittent issues

* doc: Clarify debian copyright comment

* Bump minimum python version to 3.7

* Revert "contrib: Fix capture_output in getcoins.py"

This reverts commit be59bd17ec753af7cc763474f2432d12bfc88c2f
because the changes are no longer needed.

* scripted-diff: Use new python 3.7 keywords

-BEGIN VERIFY SCRIPT-
 sed -i 's/universal_newlines/text/g' $(git grep -l universal_newlines)
-END VERIFY SCRIPT-

* test: add an easy way to run linters locally

Adds a Dockerfile configuration
that allows straightforward running of linters with compatible versions
locally. This removes a ton of annoyance when trying to appease CI,
because many of the linter versions are quite old and difficult to
maintain locally.

I realize that people may not be thrilled to more ancillary tooling to
the repo, but I think this makes a lot of sense given the linter
versions listed in this container configuration are dictated by this
repo (within the CI configuration), so having these things live in
two separate places is a recipe for version mismatches.

Eventually we can likely just use this container on CI directly to avoid
any chance of inconsistencies between local dev experience and CI.

* lint: specify the right commit range when running locally

When running lints on Cirrus, a special envvar is set ($CIRRUS_PR);
emulate this when running linters locally by setting $LOCAL_BRANCH
to any value.

* clang-tidy: Fix `performance-move-const-arg` in headers

See https://clang.llvm.org/extra/clang-tidy/checks/performance/move-const-arg.html

* clang-tidy: Fix `performance-no-automatic-move` in headers

See https://clang.llvm.org/extra/clang-tidy/checks/performance/no-automatic-move.html

* doc: add release note for #25957 (fast wallet rescan)

* test: refactor: simplify p2p_tx_download.py by using MiniWallet

* test: refactor: simplify p2p_eviction.py by using MiniWallet

Also, use the pre-mined chain of the test framework rather than
mining 100 blocks manually on each run.

* Add missing includes to fix gcc-13 compile error

* RPC: make RPCResult::MatchesType return useful errors

* [fuzz] Actually use mocked mempool in tx_pool target

* rpc: Throw more user friendly arg type check error

* mempool: Don't share mempool with dbcache in blocksonly

When -blockonly is set, reduce mempool size to 5MB unless -maxmempool
is also set.

See #9569

* doc: Update blocksonly behaviour in reduce-memory

Changes to the default mempool allocation size now documented.

Provides users with guidance on the mempool implications of -blocksonly
mode, along with instructions on how to re-enable old behaviour.

* doc: release note on mempool size in -blocksonly

Adds a release note detailing the new mempool sizing behaviour when
running in blocksonly mode, and instruction on how to override the new
defaults.

* Add unit test for ComputeTapleafHash

* test: Fix intermittent feature_rbf issue

* scripted-diff: use RPCArg::Optional::OMITTED over OMITTED_NAMED_ARG

-BEGIN VERIFY SCRIPT-
sed -i -e "/Deprecated alias for OMITTED, can be removed/d" src/rpc/util.h src/rpc/util.cpp
sed -i -e "s/OMITTED_NAMED_ARG/OMITTED/g" $(git grep -l "OMITTED_NAMED_ARG" src/)
-END VERIFY SCRIPT-

* doc: improve doc for RPCArg::Optional::OMITTED

* depends: fix systemtap download URL

* depends: systemtap 4.8

* build: use more recommended sqlite3 compile options

See https://www.sqlite.org/compile.html.

DSQLITE_DQS
> This setting disables the double-quoted string literal misfeature.

DSQLITE_DEFAULT_MEMSTATUS
> This setting causes the sqlite3_status() interfaces that track
> memory usage to be disabled.
> This helps the sqlite3_malloc() routines run much faster, and since
> SQLite uses sqlite3_malloc() internally, this helps to make the
> entire library faster.

DSQLITE_OMIT_DEPRECATED
> Omitting deprecated interfaces and features will not help SQLite
> to run any faster.
> It will reduce the library footprint, however. And it is the
> right thing to do.

DSQLITE_OMIT_SHARED_CACHE
> Omitting the possibility of using shared cache allows many
> conditionals in performance-critical sections of the code to be
> eliminated. This can give a noticeable improvement in performance.

Also: https://www.sqlite.org/sharedcache.html
> Shared-cache mode is an obsolete feature.
> The use of shared-cache mode is discouraged.
> Most use cases for shared-cache are better served by WAL mode.
> Applications that build their own copy of SQLite from source code
> are encouraged to use the -DSQLITE_OMIT_SHARED_CACHE compile-time
> option, as the resulting binary will be both smaller and faster.

DSQLITE_OMIT_JSON
Starting with sqlite 3.38.0 the JSON extension became opt-out rather
than opt-in, so we disable it here.

--disable-rtree
> An R-Tree is a special index that is designed for doing range queries.
> R-Trees are most commonly used in geospatial systems...
https://www.sqlite.org/rtree.html

--disable-fts4 --disable-fts5
> FTS5 is an SQLite virtual table module that provides full-text
> search functionality to database applications.

DSQLITE_LIKE_DOESNT_MATCH_BLOBS
> simplifies the implementation of the LIKE optimization and allows
> queries that use the LIKE optimization to run faster.

DSQLITE_OMIT_DECLTYPE
> By omitting the (seldom-needed) ability to return the declared type of
> columns from the result set of query, prepared statements can be made
> to consume less memory.

DSQLITE_OMIT_PROGRESS_CALLBACK
> By omitting this interface, a single conditional is removed from the
> inner loop of the bytecode engine, helping SQL statements to run slightly
> faster.

DSQLITE_OMIT_AUTOINIT
> with the SQLITE_OMIT_AUTOINIT option, the automatic initialization is omitted.
> This helps many API calls to run a little faster
> it also means that the application must call sqlite3_initialize()
manually.

* build: pass --enable-debug to sqlite when DEBUG=1

* wallet: permit mintxfee=0

Fixes #26797

Permit nodes to use a mintxfee of `0` if they choose.
Values below 0 are handled by the ParseMoney() check.

* test: Avoid rpc timeout in p2p_headers_sync_with_minchainwork

* [block encodings] Make CheckBlock mockable for PartiallyDownloadedBlock

* [block encodings] Avoid fuzz blocking asserts in PartiallyDownloadedBlock

* [fuzz] Add PartiallyDownloadedBlock target

* [fuzz] Assert that omitting missing transactions always fails block reconstruction

* build: fix usage of -Wloop-analysis

Looks like I introduced this in
5ced92528362d8a9ea3c15a9752f3dc184108060.

* depends: systemtap: remove variadic params that trigger compiler warnings

* init: Remove sensitive flag from rpcbind

* refactor: Remove c_str from util/check

* streams: Add DataStream without ser-type and ser-version

The moved parts can be reviewed with "--color-moved=dimmed-zebra".
The one-char changes can be reviewed with "--word-diff-regex=.".

* ci: Fix APPEND_APT_SOURCES_LIST trying to modify the host system

* txorphange: Drop redundant originator arg from GetTxToReconsider

* net_processing: only process orphans before messages

Previously, when we processed a new tx we would attempt to ATMP any
orphans that considered the new tx a parent immediately, but would only
accept at most one such tx, leaving any others to be considered on a
future run of ProcessMessages(). With this patch, we don't attempt any
orphan processing immediately after receiving a tx, instead deferring
all of them until the next call to ProcessMessages().

* net_processing: Don't process tx after processing orphans

If we made progress on orphans, consider that enough work for this peer
for this round of ProcessMessages. This also allows cleaning up the api
for TxOrphange:GetTxToReconsider().

* net_processing: indicate more work to do when orphans are ready to reconsider

When PR#15644 made orphan processing interruptible, it also introduced a
potential 100ms delay between processing of the first and second newly
reconsiderable orphan, because it didn't check if the orphan work set
was non-empty after invoking ProcessMessage(). This adds that check, so
that ProcessMessages() will return true if there are orphans to process,
usually avoiding the 100ms delay in CConnman::ThreadMessageHandler().

* Use DataStream where possible

* Remove unused CDataStream::SetType

The last use was removed in the previous commit.

* Comment mcl/bls libs

* fix linter

* fix linter

* fix Makefile.am

---------

Co-authored-by: Andrew Chow <github@achow101.com>
Co-authored-by: Jon Atack <jon@atack.com>
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Co-authored-by: Kristaps Kaupe <kristaps@blogiem.lv>
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
Co-authored-by: Pasta <pasta@dashboost.org>
Co-authored-by: glozow <gloriajzhao@gmail.com>
Co-authored-by: fanquake <fanquake@gmail.com>
Co-authored-by: Aurèle Oulès <aurele@oules.com>
Co-authored-by: stickies-v <stickies-v@protonmail.com>
Co-authored-by: Anthony Towns <aj@erisian.com.au>
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
Co-authored-by: Miles Liu <miles@bung.cc>
Co-authored-by: brunoerg <brunoely.gc@gmail.com>
Co-authored-by: Greg Sanders <gsanders87@gmail.com>
Co-authored-by: Juan Pablo Civile <elementohb@gmail.com>
Co-authored-by: ishaanam <ishaana.misra@gmail.com>
Co-authored-by: John Moffett <john.moff@gmail.com>
Co-authored-by: Kolby ML <31669092+KolbyML@users.noreply.github.com>
Co-authored-by: kouloumos <kouloumosa@gmail.com>
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
Co-authored-by: James O'Beirne <james.obeirne@pm.me>
Co-authored-by: dergoegge <n.goeggi@gmail.com>
Co-authored-by: willcl-ark <will@256k1.dev>
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
Co-authored-by: alex v <alex@nav.community>

Repeat Sync with bitcoin upstream 26/01/22 with no squash

Repeat Sync upstream 260122 with no squash