wallet: Add separate balance info for non-mempool wallet txs

ajtowns commented at 4:59 pm on October 21, 2025: contributor

Changes getbalances to report the balance of outputs for transactions that aren’t confirmed nor in the mempool (eg due to being part of too long a mempool chain, or spending non-standard outputs, or having a datacarrier output that exceeds -datacarriersize, etc)

 0$ bitcoin-cli -regtest getbalances
 1{
 2  "mine": {
 3    "trusted": 5764.96604310,
 4    "untrusted_pending": 0.00000000,
 5    "immature": 3325.00009446,
 6    "nonmempool": 19.99995580
 7  },
 8  "lastprocessedblock": {
 9    "hash": "3c8e45844bc26b66f49f595d1e4b806635424384a7629534e5c4086e00f0e2c6",
10    "height": 216
11  }
12}

Closes #11887

DrahtBot commented at 4:59 pm on October 21, 2025: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33671.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept NACK	achow101
Concept ACK	vasild, RandyMcMillan

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#27865 (wallet: Track no-longer-spendable TXOs separately by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

ajtowns commented at 5:03 pm on October 21, 2025: contributor

Relevant for #29415 – private relay of our wallet transactions would create spends that weren’t in the mempool and would thus disappear mysteriously from the wallet balance which seems unacceptable. I think I’ve seen similar behaviour when doing lots of consolidations of my signet wallet, which has also been disturbing.

Perhaps see #11020 for a similar previous attempt.

~~This PR currently doesn’t include tests that this works sensibly for nonzero nonmempool balances, hence wip/draft.~~

glozow added the label Wallet on Oct 22, 2025

ajtowns force-pushed on Oct 22, 2025

vasild commented at 10:31 am on November 4, 2025: contributor

Concept ACK

This extends the getbalances RPC. Would be good to also update the GUI (not necessary in this PR).

The new type of transactions would have somehow to be fed to the wallet so that they would be returned by wallet.GetTXOs(), called by GetBalance(). How would that be done?

RandyMcMillan commented at 1:18 pm on November 10, 2025: contributor

Concept ACK

wallet: Add separate balance info for non-mempool wallet txs 52092735a7

ajtowns force-pushed on Nov 19, 2025

tests: Add test for mempool-invalid wallet tx

Uses send rpc to create a tx with oversized OP_RETURN output, verifies
that it doesn't enter the mempool, and that getbalance rpc returns a
nonmempool value.

7af1215027

ajtowns commented at 1:21 am on November 19, 2025: contributor

This extends the getbalances RPC. Would be good to also update the GUI (not necessary in this PR).

Okay, I have a patch for that at https://github.com/ajtowns/bitcoin/commits/202511-wallet-unconf-bal-gui/ ; opened as draft at bitcoin-core/gui#911

The new type of transactions would have somehow to be fed to the wallet so that they would be returned by wallet.GetTXOs(), called by GetBalance(). How would that be done?

I don’t think there’s a way to do this for arbitrary txs that originate outside of the wallet, even if they involve a wallet address. Maybe there should be? (ie, an RPC that lets you pass raw tx to AddToWalletIfInvolvingMe ?)

But for txs that are created by the wallet, eg send or sendall rpc, then I believe they’ll be added to the wallet via CommitTransaction which will also invoke SubmitTxMemoryPoolAndRelay() which in turn invokes broadcastTransaction.

You can create non-mempool txs by limiting -datacarriersize on startup, then using bitcoin-cli send to create a tx with a {"data":"deadbeef..."} output that exceeds the datacarriersize you set. Those will not be rejected by the rpc, and will enter the wallet as described above, but not enter the mempool. If you restart with a sufficiently larger datacarriersize, the tx will enter the mempool (and be trusted, since it’s from you, to you).

I’ve added a test to wallet_send.py that exercises this behaviour.

ajtowns marked this as ready for review on Nov 19, 2025

ajtowns renamed this:
~~[wip] wallet: Add separate balance info for non-mempool wallet txs~~
wallet: Add separate balance info for non-mempool wallet txs
on Nov 19, 2025

achow101 commented at 10:20 pm on November 19, 2025: member

Concept NACK-ish

There’s a lot of reasons that an unconfirmed transaction might be in the wallet but not the mempool, and I think giving an accurate balance of those transactions will be hard to get right. We absolutely do not want to mislead users into thinking they have more coins than they actually do, so I am hesitant to add something like this.

From a cursory glance at the code, I think 2 transactions that conflict with each other but aren’t in the mempool would both get counted towards the nonmempool balance, and which I think is incorrect. But in that case, which one should be counted, if at all?

ajtowns commented at 0:56 am on November 20, 2025: contributor

Concept NACK-ish

There’s a lot of reasons that an unconfirmed transaction might be in the wallet but not the mempool, and I think giving an accurate balance of those transactions will be hard to get right. We absolutely do not want to mislead users into thinking they have more coins than they actually do, so I am hesitant to add something like this.

We currently mislead users into thinking they have less coins than they actually do (due to transactions in the wallet with long mempool chains, eg), which is very scary, and (IMO) very bad from an accounting POV. So I don’t think the current state is acceptable either.

The underlying problem (IMO) is that the accounting is unbalanced: if we see two transactions (A,B) in the wallet, where B spends A (via IsSpent()) but B is not in the mempool, then because B spends A we will ignore our funds from A (believing it’s spent by B) but we will also ignore our funds from B (believing it’s not a real tx). There’s two ways to fix that: we could not ignore the funds that are still in A, or we could not ignore the funds that are in B. Currently this PR tries not ignoring the funds in B.

From a cursory glance at the code, I think 2 transactions that conflict with each other but aren’t in the mempool would both get counted towards the nonmempool balance, and which I think is incorrect. But in that case, which one should be counted, if at all?

That’s accurate:

 0$ bitcoin-cli -regtest -named send outputs='{"bcrt1qperny2w6xvfkjvec44w4qtwwk3ykuw9py0ejrp": 1.0, "data": "54686520717569636b2062726f776e20666f78206a756d7073206f76657220746865206c617a7920646f67"}' fee_rate=1 inputs='[{"txid":"a6e88286d165dbe4f4aab7830f7b68931ef7f5b66d32f0ea573d906594ad6a0c","vout":0}]'
 1{
 2  "txid": "42ef1c7d7c2afee4bb2519fbfcf284ef40ae156ea193ce86a1fa670183a12322",
 3  "complete": true
 4}
 5$ bitcoin-cli -regtest -named send outputs='{"bcrt1qperny2w6xvfkjvec44w4qtwwk3ykuw9py0ejrp": 2.0, "data": "54686520717569636b2062726f776e20666f78206a756d7073206f76657220746865206c617a7920646f67"}' fee_rate=2 inputs='[{"txid":"a6e88286d165dbe4f4aab7830f7b68931ef7f5b66d32f0ea573d906594ad6a0c","vout":0}]'
 6{
 7  "txid": "ebbd3bf4807d8e5683d124cd726de982a4f36e2bae264ff5b91545dae41bacbe",
 8  "complete": true
 9}
10$ bitcoin-cli -regtest getbalances
11{
12  "mine": {
13    "trusted": 450.00000000,
14    "untrusted_pending": 0.00000000,
15    "immature": 5000.00000000,
16    "nonmempool": 99.99999415
17  },
18  "lastprocessedblock": {
19    "hash": "33e1b4932fb3a464aef972470ffed7650795a35f26214cfa58f76a787d548768",
20    "height": 110
21  }
22}

ie spending a single 50 rBTC output twice suddenly gets reported as ~100 rBTC ex fees.

I don’t think there’s an easy way to avoid that with the “don’t ignore B’s funds” approach: if you have confirmed utxos A (1 BTC), B (2 BTC), C (3 BTC), you could have non-mempool txs D spending A,B (3 BTC out, 2.9 BTC change) and E spending A,C (4 BTC 3.8 BTC change). You subtract 6 BTC from your balance because each of A,B,C have non-abandoned, non-block/mempool-conflicted spends in the wallet, but if D were to confirm you should add 5.9 BTC (D+C), if E were to confirm you should add 5.8 BTC (E+B), and if you pretended both were to confirm you’d add 6.7 BTC (D+E).

The other approach, then, is to say “D and E aren’t in the mempool, so A B and C aren’t -really- spent”, and not remove the 6 BTC from A,B,C from your balance; though IMO it still makes sense to separate those out into “possibly-spent-by-nonmempool-txs”. I believe that easily avoids the double counting if you have conflicting possible spends.

With that approach, if I spend an immature coinbase, that changes my balances from:

 0$ bitcoin-cli -regtest getbalances
 1{
 2  "mine": {
 3    "trusted": 450.00000000,
 4    "untrusted_pending": 0.00000000,
 5    "immature": 5000.00000000,
 6    "nonmempool": 50.00000000
 7  },
 8  "lastprocessedblock": {
 9    "hash": "33e1b4932fb3a464aef972470ffed7650795a35f26214cfa58f76a787d548768",
10    "height": 110
11  }
12}
13$ bitcoin-cli -regtest getbalances
14{
15  "mine": {
16    "trusted": 450.00000000,
17    "untrusted_pending": 0.00000000,
18    "immature": 4950.00000000,
19    "nonmempool": 100.00000000
20  },
21  "lastprocessedblock": {
22    "hash": "33e1b4932fb3a464aef972470ffed7650795a35f26214cfa58f76a787d548768",
23    "height": 110
24  }
25}

Double spending the same tx at that point doesn’t change the result. If I do the “A,B,C” (50 rBTC each) spent by conflicting “D,E”, I get:

 0$ bitcoin-cli -regtest getbalances
 1{
 2  "mine": {
 3    "trusted": 5949.99999220,
 4    "untrusted_pending": 0.00000000,
 5    "immature": 3200.00000780,
 6    "nonmempool": 100.00000000
 7  },
 8  "lastprocessedblock": {
 9    "hash": "3ab4582226d5e8ad76438db48d76e822c31bce2cdbc7ba82a5d974a277515d0d",
10    "height": 221
11  }
12}
13...
14$ bitcoin-cli -regtest getbalances
15{
16  "mine": {
17    "trusted": 5899.99999220,
18    "untrusted_pending": 0.00000000,
19    "immature": 3200.00000780,
20    "nonmempool": 150.00000000
21  },
22  "lastprocessedblock": {
23    "hash": "3ab4582226d5e8ad76438db48d76e822c31bce2cdbc7ba82a5d974a277515d0d",
24    "height": 221
25  }
26}

That approach is still “misleading” in the sense that those nonmempool txs are reported as how much money you’re spending, rather than how much you’re potentially keeping as change if those spends go through.

Another approach would be to report it as:

0trusted: 6049.99
1untrusted_pending: 0
2immature: 3200
3nonmempool-pending: -150

ie “your balance is this, but you have X funds that are currently not in your mempool that you might lose some/all of”

Could also perhaps do it more fine-grained as:

0trusted: 6049.99 [-100 non-mempool spends]
1untrusted_pending: 0
2immature: 3200 [-50 non-mempool spends]

[EDIT: You could perhaps also account for locked coins similarly to non-mempool spends, which might be logical if you’re locking coins because you’re spending them elsewhere in ways you don’t expect will make it to your mempool]

Happy to switch to one of those approaches, or something else, if that seems more acceptable.

wallet: Add separate balance info for non-mempool wallet txs #33671

Code Coverage & Benchmarks

Reviews

Conflicts