ischange / listsinceblock/listtransactions bug

Crypto2 commented at 7:24 pm on June 6, 2019: none

After updating to 0.18.0 we’ve started experiencing issues where TXes to random addresses in our wallet aren’t showing up in listsinceblock/listtransactions. Trying to narrow down when I do a getaddressinfo on them I saw ischange = true and that seems to be the issue is some of the addresses have been marked as change somehow; note these were made with getnewaddress and/or addwitnessaddress so are not change addresses. Doing a “setlabel address ‘’” removes the flag and fixes it but we can’t go through millions of addresses with getaddressinfo :)

promag commented at 7:34 pm on June 6, 2019: member

What was your previous version? 0.17.0 - better, which version was used to create those addresses?

Crypto2 commented at 1:39 am on June 7, 2019: none

Previous version was 0.17.1, the addresses could be from as far back as 0.16.0 though based on the dates we have for them.

MarcoFalke added the label Wallet on Jun 7, 2019

MarcoFalke commented at 7:52 am on June 7, 2019: member

I think addwitnessaddress was only meant for testing.

Crypto2 commented at 7:04 pm on June 7, 2019: none

In the beginning of segwit it was the only API call available to make segwit addresses, we switched to getnewaddress once it was updated to support segwit as well though.

Not sure why it would set a change flag on random addresses though? (And some of these addresses would have been made after we switched from using it as well.)

Crypto2 commented at 5:22 pm on June 13, 2019: none

Any update on this? It’s getting really annoying and we can’t go through millions of addresses checking them for ischange: true or not.

sipa commented at 5:33 pm on June 13, 2019: member

This certainly sounds like a bug, but without a way to reproduce it there isn’t much to investigate.

Crypto2 commented at 8:59 pm on June 13, 2019: none

Well not reporting TXes in listtransactions/listsinceblock just because the address is a change address would be a bug in itself. If they fix that then this will be fixed as well.

Crypto2 commented at 4:02 pm on June 14, 2019: none

I’m not super familiar with Bitcoin internals so I don’t want to just blindly try to fix this myself, but it looks like removing this might fix the bug? https://github.com/bitcoin/bitcoin/blob/d0f81a96d9c158a9226dc946bdd61d48c4d42959/src/wallet/wallet.cpp#L1756

sipa commented at 4:10 pm on June 14, 2019: member

@Crypto2 Not reporting payments that are actual change is by design, as otherwise every transaction would show up multiple times.

There are two issues here:

When a payment from the outside comes in to a change address, it shouldn’t be treated as change (only when it’s from-self).
You somehow ended up with payments to change addresses.

The first is an improvement that may be worthwhile, but I wouldn’t consider it a bug. You shouldn’t be receiving incoming payments as transactions with change outputs. We may want to improve how to deal with this situation (as I agree not listing them at all is bad), but there is only so much that can be done.

The second, depending on how you got in this situation, worries me a lot more. If Bitcoin Core was ever giving out change addresses as public payment addresses, we have a serious issue, so I’d like to see a way to reproduce that.

promag commented at 8:49 pm on June 14, 2019: member

Doing a “setlabel address ‘’” removes the flag

Actually there is no such flag:

https://github.com/bitcoin/bitcoin/blob/c165df198d6e9ea7292af0578024113803b8a6a2/src/wallet/wallet.cpp#L1394-L1414

As you can see by setting a label (saved in mapAddressBook) the address is no longer considered a change address (L1410).

If you go back to 0.17 with a copy of the wallet, does those transactions show up?

Crypto2 commented at 6:57 pm on June 15, 2019: none

They definitely weren’t ever change addresses, they were all made with getnewaddress and/or addwitnessaddress.

We can’t go back to 0.17.x since this is a production system and it takes forever for Bitcoin to start up if you include the time to unlock the wallet. We for sure didn’t experience this with 0.17.x though, we never had this issue a single time then after updating to 0.18.0 started seeing a ticket for this at least once daily.

Crypto2 commented at 8:38 pm on June 20, 2019: none

Any update on this? It’s getting ridiculous at this point it’s been 2 weeks and no fix?

promag commented at 8:44 pm on June 20, 2019: member

Like @sipa said above, you should help to reproduce the but.

sipa commented at 8:50 pm on June 20, 2019: member

@Crypto2 The fact that your addresses are being treated as change is potentially a very serious issue. However, there really is no way to know where to start looking without a way to reproduce the issue.

Crypto2 commented at 5:45 pm on June 21, 2019: none

What about that line I posted above, would removing it correct the “sends to change addresses not being shown in listtransactions/listsinceblock” issue without breaking other things? https://github.com/bitcoin/bitcoin/blob/d0f81a96d9c158a9226dc946bdd61d48c4d42959/src/wallet/wallet.cpp#L1756

imo it’s really a bad feature on it’s own, listtransactions/listsinceblock should show all sends to the wallet regardless of if it’s to a change address or not. Otherwise if you ever wanted to audit the wallet you’d have to get each individual transaction over the RPC.

Crypto2 commented at 5:43 pm on June 25, 2019: none

Any update if removing the check is fine or not? I’m not an expert on Bitcoin internals so don’t want to break anything.

sipa commented at 5:45 pm on June 25, 2019: member

I’ve told you already we can’t do that:

@Crypto2 Not reporting payments that are actual change is by design, as otherwise every transaction would show up multiple times.

Making the software detect transactions to change addresses that are in fact payments, despite not paying to any payment addresses would be a useful feature, but not something that will happen in the short term.

The bug here is that somehow your wallet thinks payment addresses are changes. That needs to be fixed, but we really need a way to reproduce this issue.

Crypto2 commented at 2:24 am on June 28, 2019: none

I did fix it by going ahead and removing that line. So anyone else if you have this trouble that will do it.

Having the receive to the change address (or in this case one it thinks is one) show up isn’t a real issue, better safe than sorry should be the policy and show them.

MarcoFalke commented at 4:15 pm on April 26, 2020: member

Is this still an issue with a recent version of Bitcoin Core? If yes, what are the steps to reproduce?

MarcoFalke closed this on Apr 26, 2020

Crypto2 commented at 3:19 pm on April 27, 2020: none

@MarcoFalke afaik I know it still is although I still don’t know exactly how to reproduce. My fix above does still work.

MarcoFalke commented at 3:24 pm on April 27, 2020: member

It seems unlikely that at this point someone is going to dig up the root cause of the bug which seemed to correlate with addwitnessaddress, which is long removed.

If there is a bug fix for Bitcoin Core, we are happy to take it. Though, as long as there are no steps to reproduce or even indication that more users than just yourself are affected, I don’t think anyone is going to submit a fix.

du2zy commented at 1:40 pm on November 16, 2020: none

I faced such problem twice: using the node reserving (#19856) and when I restore from backup (#20293). Very easy steps to reproduce the problem is described in issue (#20293)

Did I get it right that this problem cannot be solved without bitcoin core compilation?

du2zy commented at 9:23 am on November 17, 2020: none

@MarcoFalke please reopen the issue for further discussion. Steps to reproduce problem are shown at #20293. I’ll duplicate it here:

Run bitcoin core in regtest mode
Create a new address A and generate 101 blocks with sending a reward to the created address A
Create a wallet backup
Create new address B
Send transaction to address B, write TXID
Generate 1 block with sending a reward to address A
View getaddressinfo B. Verify labels are not an empty list
View listtransactions. Make sure there is a TXID transaction present
Stop bitcoin core
Restore backup
Run bitcoin core
View getaddressinfo B and listtransactions. Detect that the address does not have a label set and that listtransactions does not contain a TXID transaction.

DrahtBot locked this on Feb 15, 2022

ischange / listsinceblock/listtransactions bug #16160