wallet: Add separate balance info for non-mempool wallet txs

ajtowns commented at 4:59 pm on October 21, 2025: contributor

Changes getbalances to report the sum of txos spent by transactions that aren’t confirmed nor in the mempool (eg due to being part of too long a mempool chain, or spending non-standard outputs, or having a datacarrier output that exceeds -datacarriersize, etc). Those values are added to the trusted/untrusted_pending/immature/used fields as appropriate (where previously they were skipped), and subtracted from the new nonmempool field, so that the sum of all fields remains the same.

For example:

 0$ bitcoin-cli -regtest getbalances
 1{
 2  "mine": {
 3    "trusted": 6049.99999220,
 4    "untrusted_pending": 0.00000000,
 5    "immature": 3200.00000780,
 6    "nonmempool": -100.00000000
 7  },
 8  "lastprocessedblock": {
 9    "hash": "3ab4582226d5e8ad76438db48d76e822c31bce2cdbc7ba82a5d974a277515d0d",
10    "height": 221
11  }
12}

Closes #11887

DrahtBot commented at 4:59 pm on October 21, 2025: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33671.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept NACK	achow101
Concept ACK	vasild, RandyMcMillan

If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#33392 (wallet, rpc: add UTXO set check and incremental rescan to importdescriptors by musaHaruna)
#27865 (wallet: Track no-longer-spendable TXOs separately by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

ajtowns commented at 5:03 pm on October 21, 2025: contributor

Relevant for #29415 – private relay of our wallet transactions would create spends that weren’t in the mempool and would thus disappear mysteriously from the wallet balance which seems unacceptable. I think I’ve seen similar behaviour when doing lots of consolidations of my signet wallet, which has also been disturbing.

Perhaps see #11020 for a similar previous attempt.

~~This PR currently doesn’t include tests that this works sensibly for nonzero nonmempool balances, hence wip/draft.~~

glozow added the label Wallet on Oct 22, 2025

ajtowns force-pushed on Oct 22, 2025

vasild commented at 10:31 am on November 4, 2025: contributor

Concept ACK

This extends the getbalances RPC. Would be good to also update the GUI (not necessary in this PR).

The new type of transactions would have somehow to be fed to the wallet so that they would be returned by wallet.GetTXOs(), called by GetBalance(). How would that be done?

RandyMcMillan commented at 1:18 pm on November 10, 2025: contributor

Concept ACK

ajtowns force-pushed on Nov 19, 2025

ajtowns commented at 1:21 am on November 19, 2025: contributor

This extends the getbalances RPC. Would be good to also update the GUI (not necessary in this PR).

Okay, I have a patch for that opened as draft at bitcoin-core/gui#911

The new type of transactions would have somehow to be fed to the wallet so that they would be returned by wallet.GetTXOs(), called by GetBalance(). How would that be done?

I don’t think there’s a way to do this for arbitrary txs that originate outside of the wallet, even if they involve a wallet address. Maybe there should be? (ie, an RPC that lets you pass raw tx to AddToWalletIfInvolvingMe ?)

But for txs that are created by the wallet, eg send or sendall rpc, then I believe they’ll be added to the wallet via CommitTransaction which will also invoke SubmitTxMemoryPoolAndRelay() which in turn invokes broadcastTransaction.

You can create non-mempool txs by limiting -datacarriersize on startup, then using bitcoin-cli send to create a tx with a {"data":"deadbeef..."} output that exceeds the datacarriersize you set. Those will not be rejected by the rpc, and will enter the wallet as described above, but not enter the mempool. If you restart with a sufficiently larger datacarriersize, the tx will enter the mempool (and be trusted, since it’s from you, to you).

I’ve added a test to wallet_send.py that exercises this behaviour.

ajtowns marked this as ready for review on Nov 19, 2025

ajtowns renamed this:
~~[wip] wallet: Add separate balance info for non-mempool wallet txs~~
wallet: Add separate balance info for non-mempool wallet txs
on Nov 19, 2025

achow101 commented at 10:20 pm on November 19, 2025: member

Concept NACK-ish

There’s a lot of reasons that an unconfirmed transaction might be in the wallet but not the mempool, and I think giving an accurate balance of those transactions will be hard to get right. We absolutely do not want to mislead users into thinking they have more coins than they actually do, so I am hesitant to add something like this.

From a cursory glance at the code, I think 2 transactions that conflict with each other but aren’t in the mempool would both get counted towards the nonmempool balance, and which I think is incorrect. But in that case, which one should be counted, if at all?

ajtowns commented at 0:56 am on November 20, 2025: contributor

Concept NACK-ish

There’s a lot of reasons that an unconfirmed transaction might be in the wallet but not the mempool, and I think giving an accurate balance of those transactions will be hard to get right. We absolutely do not want to mislead users into thinking they have more coins than they actually do, so I am hesitant to add something like this.

We currently mislead users into thinking they have less coins than they actually do (due to transactions in the wallet with long mempool chains, eg), which is very scary, and (IMO) very bad from an accounting POV. So I don’t think the current state is acceptable either.

The underlying problem (IMO) is that the accounting is unbalanced: if we see two transactions (A,B) in the wallet, where B spends A (via IsSpent()) but B is not in the mempool, then because B spends A we will ignore our funds from A (believing it’s spent by B) but we will also ignore our funds from B (believing it’s not a real tx). There’s two ways to fix that: we could not ignore the funds that are still in A, or we could not ignore the funds that are in B. Currently this PR tries not ignoring the funds in B.

From a cursory glance at the code, I think 2 transactions that conflict with each other but aren’t in the mempool would both get counted towards the nonmempool balance, and which I think is incorrect. But in that case, which one should be counted, if at all?

That’s accurate:

 0$ bitcoin-cli -regtest -named send outputs='{"bcrt1qperny2w6xvfkjvec44w4qtwwk3ykuw9py0ejrp": 1.0, "data": "54686520717569636b2062726f776e20666f78206a756d7073206f76657220746865206c617a7920646f67"}' fee_rate=1 inputs='[{"txid":"a6e88286d165dbe4f4aab7830f7b68931ef7f5b66d32f0ea573d906594ad6a0c","vout":0}]'
 1{
 2  "txid": "42ef1c7d7c2afee4bb2519fbfcf284ef40ae156ea193ce86a1fa670183a12322",
 3  "complete": true
 4}
 5$ bitcoin-cli -regtest -named send outputs='{"bcrt1qperny2w6xvfkjvec44w4qtwwk3ykuw9py0ejrp": 2.0, "data": "54686520717569636b2062726f776e20666f78206a756d7073206f76657220746865206c617a7920646f67"}' fee_rate=2 inputs='[{"txid":"a6e88286d165dbe4f4aab7830f7b68931ef7f5b66d32f0ea573d906594ad6a0c","vout":0}]'
 6{
 7  "txid": "ebbd3bf4807d8e5683d124cd726de982a4f36e2bae264ff5b91545dae41bacbe",
 8  "complete": true
 9}
10$ bitcoin-cli -regtest getbalances
11{
12  "mine": {
13    "trusted": 450.00000000,
14    "untrusted_pending": 0.00000000,
15    "immature": 5000.00000000,
16    "nonmempool": 99.99999415
17  },
18  "lastprocessedblock": {
19    "hash": "33e1b4932fb3a464aef972470ffed7650795a35f26214cfa58f76a787d548768",
20    "height": 110
21  }
22}

ie spending a single 50 rBTC output twice suddenly gets reported as ~100 rBTC ex fees.

I don’t think there’s an easy way to avoid that with the “don’t ignore B’s funds” approach: if you have confirmed utxos A (1 BTC), B (2 BTC), C (3 BTC), you could have non-mempool txs D spending A,B (3 BTC out, 2.9 BTC change) and E spending A,C (4 BTC 3.8 BTC change). You subtract 6 BTC from your balance because each of A,B,C have non-abandoned, non-block/mempool-conflicted spends in the wallet, but if D were to confirm you should add 5.9 BTC (D+C), if E were to confirm you should add 5.8 BTC (E+B), and if you pretended both were to confirm you’d add 6.7 BTC (D+E).

The other approach, then, is to say “D and E aren’t in the mempool, so A B and C aren’t -really- spent”, and not remove the 6 BTC from A,B,C from your balance; though IMO it still makes sense to separate those out into “possibly-spent-by-nonmempool-txs”. I believe that easily avoids the double counting if you have conflicting possible spends.

With that approach, if I spend an immature coinbase, that changes my balances from:

 0$ bitcoin-cli -regtest getbalances
 1{
 2  "mine": {
 3    "trusted": 450.00000000,
 4    "untrusted_pending": 0.00000000,
 5    "immature": 5000.00000000,
 6    "nonmempool": 50.00000000
 7  },
 8  "lastprocessedblock": {
 9    "hash": "33e1b4932fb3a464aef972470ffed7650795a35f26214cfa58f76a787d548768",
10    "height": 110
11  }
12}
13$ bitcoin-cli -regtest getbalances
14{
15  "mine": {
16    "trusted": 450.00000000,
17    "untrusted_pending": 0.00000000,
18    "immature": 4950.00000000,
19    "nonmempool": 100.00000000
20  },
21  "lastprocessedblock": {
22    "hash": "33e1b4932fb3a464aef972470ffed7650795a35f26214cfa58f76a787d548768",
23    "height": 110
24  }
25}

Double spending the same tx at that point doesn’t change the result. If I do the “A,B,C” (50 rBTC each) spent by conflicting “D,E”, I get:

 0$ bitcoin-cli -regtest getbalances
 1{
 2  "mine": {
 3    "trusted": 5949.99999220,
 4    "untrusted_pending": 0.00000000,
 5    "immature": 3200.00000780,
 6    "nonmempool": 100.00000000
 7  },
 8  "lastprocessedblock": {
 9    "hash": "3ab4582226d5e8ad76438db48d76e822c31bce2cdbc7ba82a5d974a277515d0d",
10    "height": 221
11  }
12}
13...
14$ bitcoin-cli -regtest getbalances
15{
16  "mine": {
17    "trusted": 5899.99999220,
18    "untrusted_pending": 0.00000000,
19    "immature": 3200.00000780,
20    "nonmempool": 150.00000000
21  },
22  "lastprocessedblock": {
23    "hash": "3ab4582226d5e8ad76438db48d76e822c31bce2cdbc7ba82a5d974a277515d0d",
24    "height": 221
25  }
26}

That approach is still “misleading” in the sense that those nonmempool txs are reported as how much money you’re spending, rather than how much you’re potentially keeping as change if those spends go through.

Another approach would be to report it as:

0trusted: 6049.99
1untrusted_pending: 0
2immature: 3200
3nonmempool-pending: -150

ie “your balance is this, but you have X funds that are currently not in your mempool that you might lose some/all of”

Could also perhaps do it more fine-grained as:

0trusted: 6049.99 [-100 non-mempool spends]
1untrusted_pending: 0
2immature: 3200 [-50 non-mempool spends]

[EDIT: You could perhaps also account for locked coins similarly to non-mempool spends, which might be logical if you’re locking coins because you’re spending them elsewhere in ways you don’t expect will make it to your mempool]

Happy to switch to one of those approaches, or something else, if that seems more acceptable.

achow101 commented at 8:40 pm on November 21, 2025: member

Another approach would be to report it as:
0trusted: 6049.99
1untrusted_pending: 0
2immature: 3200
3nonmempool-pending: -150
ie “your balance is this, but you have X funds that are currently not in your mempool that you might lose some/all of”

I would prefer this approach and docs just need to be clear that some of that amount may come back as change.

wallet: Have GetBalance report used amount directly without two calls 81e763f1e5

wallet: Add separate balance info for non-mempool wallet txs 183921910d

tests: Add test for mempool-invalid wallet tx

Uses send rpc to create a tx with oversized OP_RETURN output, verifies
that it doesn't enter the mempool, and that getbalance rpc returns a
nonmempool value.

e7200379f3

ajtowns force-pushed on Nov 27, 2025

ajtowns commented at 5:46 am on November 27, 2025: contributor

Okay, so when a “transfer to self” tx paying 1310 sats in fees from 100 BTC of inputs doesn’t hit the mempool, it currently looks like:

0{
1  "mine": {
2    "trusted": 5949.99999220,
3    "untrusted_pending": 0.00000000,
4    "immature": 3200.00000780
5  }
6}

vs when the tx does hit the mempool:

0{
1  "mine": {
2    "trusted": 6049.99997910,
3    "untrusted_pending": 0.00000000,
4    "immature": 3200.00000780
5  }
6}

That is, there’s 99.9999869 BTC missing from your balance while the transaction is in the wallet but not in the mempool.

With the current version of this PR, it now looks like this when the tx is not in the mempool:

0{
1  "mine": {
2    "trusted": 6049.99999220,
3    "untrusted_pending": 0.00000000,
4    "immature": 3200.00000780,
5    "nonmempool": -100.00000000
6  }
7}

So if you add those figures up, you’re still 100 BTC down, but at least you have a clue as to what’s going on.

When it does hit the mempool (or gets mined), it becomes:

0{
1  "mine": {
2    "trusted": 6049.99997910,
3    "untrusted_pending": 0.00000000,
4    "immature": 3200.00000780,
5    "nonmempool": 0.00000000
6  }
7}

which is the same as previously (with the addtion of nonmempool=0).

I also refactored it to calculate the “used” value directly (ie funds held in coins with addresses that are already spent from, for wallets that care about that), since I couldn’t figure out how to get the nonmempool figure correct otherwise, but I think that’s an improvement anyway.

I believe there was a “bug” in the cleanup decorator for wallet_v3_txs in that coins in a wallet spent by non-mempool wallet txs wouldn’t be cleared out by the sendall calls. That wasn’t showing up in the balance checks previously, but does show up when the nonmempool figures are also reported. Addressed that by abandoning any unconfirmed wallet txs prior to calling sendall.

ajtowns force-pushed on Nov 27, 2025

DrahtBot added the label CI failed on Nov 27, 2025

DrahtBot commented at 7:28 am on November 27, 2025: contributor

🚧 At least one of the CI tasks failed. Task test each commit: https://github.com/bitcoin/bitcoin/actions/runs/19727553059/job/56521733216 LLM reason (✨ experimental): Git rebase failed due to uncommitted changes in the index.

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

ajtowns force-pushed on Nov 27, 2025

DrahtBot removed the label CI failed on Nov 27, 2025

wallet: Add separate balance info for non-mempool wallet txs #33671

Code Coverage & Benchmarks

Reviews

Conflicts