wallet, sqlite: Enable secure_delete by default #36796

Currently, when we make deletions or updates to the database that require records to be fully deleted and to not persist in free pages afterwards, we need to explicitly call WalletDatabase::Rewrite. This behavior was introduced in #635 to ensure that unencrypted private keys are erased from the database after encrypting a wallet, and initial encryption is the only consumer of this function.

However, initial encryption is not the only functionality that requires secure deletion. l0rinc's LLMs recently found changing the passphrase can leave the original that encryption key record in free pages. We should want to ensure that those records are also deleted, although I do not consider that the presence of such records in free space to actually be a problem. Note that this behavior appears to be present since wallet encryption was initially introduced in #352.

Fortunately, SQLite offers a secure_delete flag that can be enabled on each connection to a database. If set, all deletes or updates will ensure that any data that was deleted or overwritten actually are overwritten with 0's. It is one of the 2 methods in SQLite's documentation for ensuring that records do not persist in free pages, the other method being to call VACUUM after deletions or updates, which is what Rewrite does.

The downside is that secure_delete will increase CPU and disk I/O, which is why this was not set previously. However, it seems like several distros ship with secure_delete enabled by default: Debian, Ubuntu, Fedora, Arch, and NixOS all set SQLite's compile time option to default secure_delete to enabled, and these are just the ones I've looked at. This suggests to me that the performance penalty is not actually that bad. As I run Arch and often do not use depends builds, I had not noticed that secure_delete was always on and did not experience any particularly poor wallet performance that could be attributed to that behavior. In the grand scheme of things, erases and updates are fairly infrequent in the wallet.

This PR enables secure_delete when the database is opened, and also enables it by default in the sqlite depends package. There are also a couple tests to check that all records we want deleted actually are being deleted from the database. WalletDatabase::Rewrite is removed as it it is no longer used.

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

External sites

• Code coverage & benchmarks: https://corecheck.dev/bitcoin/bitcoin/pulls/36796
• Ralph review <img src=https://review.fish.foo/assets/bitcoin/bitcoin/36796.png alt="Ralph review status"> https://git.fish.foo/bitcoin/bitcoin/pulls/36796#author-ralph

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline and AI policy for information on the review process.

If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

<!--5faf32d7da4f0f540f40219e4f7537a3-->

<!--85328a0da195eb286784d51f73fa0af9-->

🚧 At least one of the CI tasks failed. _{Task lint: https://github.com/bitcoin/bitcoin/actions/runs/42328113449/job/128268068670} _{LLM reason (✨ experimental): CI failed because Python lint found six unnecessary semicolons in test/functional/wallet_encryption.py.}

<details><summary>Hints</summary>

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

</details>

Concept ACK

For a wallet created at this head, no copy of the plaintext key stays in wallet.dat-journal after encryptwallet, while the wallet is loaded or after a kill -9.

I ran encryptwallet on a fresh descriptor wallet at 96352f5200 and counted copies of its 32-byte xprv private key in its files. I linked bitcoind against Homebrew SQLite and against depends, both leaving secure_delete off by default, so only the pragma this PR adds can zero the pages. wallet.dat and the journal held 0 copies while loaded, after a kill -9, and after a restart and reload. The same run on the head of #36793, which still relies on VACUUM, left 6 copies in the journal, 1 after the restart.

The script, builds and output are in https://gist.github.com/fametrano/245e7d7aa2a4762d2ce7c102e097b92f/7cf068b0300062264dce40a1838c98d4958504b0.