Depends on #27601, please go there first.
This work aims to define, and implement a base standard mechanism to detect individual change outputs.
Context
Currently, the wallet detects whether an output is change or not based on data stored in the address book.
There is no notion of “change outputs”, the wallet detects change scripts.
Connoting that any address book record modification has implications on all the historical outputs related to that particular destination. Meaning that all those outputs can either be change or not. There is no middle-ground granular distinction.
How Change Detection Currently Works?
The wallet walks-through the transaction outputs, extracts the script destination and verify the following two points:
-
If the destination doesn’t exist in the address book, then the script is a “change address”.
-
If the destination exists in the address book, but it doesn’t have a label, then the script is a “change address”.
Motivation
There are a good number of problems in the current approach:
-
We make the wallet dependent on an external structure, with separate storage. Which has to be updated and maintained along with the wallet state.
-
It cannot be maintained nor recovered across different wallet instances. Cannot re-create the, possibly custom, address book data only by importing the wallet descriptor string.
-
As the address book is an structure that the user can freely modify, the change detection process might differ through different wallets.
-
The current rudimentary assumptions of “no address book entry” or “no label set for the address book entry” to denote that certain script destination is change or not can easily be broken: E.g. derive an address from one of the wallet’s external paths manually. Then send coins to it. As the receive destination wasn’t created inside the wallet, the wallet has no associated address book entry. So, the reception is invalidly detected as change (added a test case for it).
-
The wallet can’t detect change outputs on more complex scripts such as multi-sig change outputs.
-
The wallet is not able to detect change outputs going to an internal address if the internal address has a label. (E.g. the user can manually set a label for the internal address and, doing that, make that all the change outputs, in the wallet history, that were sent to the destination are no longer detected as change).
-
There isn’t a way to distinguish the external reception of coins into an internal address. Coins reception on any internal address are always detected as change.
New Change Detection Mechanism Goals
Aiming to:
-
Define a base mechanism to align different wallet implementations. Preventing each piece of software from diverging on the basic change outputs distinction.
-
Detect change outputs on-demand without requiring to maintain an external data structure synced with the latest wallet state.
-
Independently, and accurately, detect change outputs regardless data stored in structures that the user can freely modify.
-
Granular distinction between change vs non-change outputs that were sent to the same internal address. E.g. the reception of coins, from an external source, on internal addresses will not longer be detected as change anymore.
-
Expand the change detection to more complex scripts such as a multi-sig protected addresses. (While they are added into the wallet on an internal spkm)
Change Output Detection Rules
A transaction output is change if it fulfills the following points:
-
At least one of the parent transaction inputs is from the wallet. (If none of them are, then the wallet is receiving coins on an internal address).
-
The script extracted destination is from the wallet and is located in one of the internal script pub key manager. (e.g. derived from an internal derivation path)
What about legacy wallets?
If the legacy wallet is HD post-split, we have an internal derivation path, so we can follow the same process as descriptors wallet. Unless the destination is on the pre-split key pool, in which case, we fallback to the follow-up case.
If the legacy wallet is pre-split, we continue using the address book as we either have an HD wallet with keys derived only on the external path, or we are using raw keypool.
———————————————————————
Extra Note
This PR, in about 85% at least, is about expanding the current test coverage for the change output detection area.
TO DO (still WIP):
- Save “internal” flag on non-active descriptors so the wallet can use them on the change detection process. (which will fix the currently failing test cases).
- Re-organize commits so tests always pass.
- Verify backwards compatibility.
