After max chain of unconfirmed change transactions, last tx is missing from memory until rescan #10004

Summary:

https://github.com/bitcoin/bitcoin/blob/57b34599b2deb179ff1bd97ffeab91ec9f904d85/doc/release-notes/release-notes-0.12.0.md

"Bitcoin Core 0.12 also introduces new default policy limits on the length and size of unconfirmed transaction chains that are allowed in the mempool (generally limiting the length of unconfirmed chains to 25 transactions, with a total size of 101 KB)."

1. Generate a chain of transactions with unconfirmed change outputs until this maximum is hit.
2. The last tx in this chain does is not accessible from gettransaction, does not appear in listunspent and will not be mined into a block until the node is restarted with -rescan (luckily 0.14.0 saves mempool to disk so the tx is "found" on rescan)

Demonstration:

Start node, generate 101 blocks so only a single 50 BTC output is mature enough to spend:

$ ./bitcoind -daemon -regtest
Bitcoin server starting
$ ./bitcoin-cli -regtest getinfo
{
  "version": 140000,
  "protocolversion": 70015,
  "walletversion": 130000,
  "balance": 0.00000000,
  ...
}
$ ./bitcoin-cli -regtest generate 101
[
  "271d2c725a2279976bee7650bf20f0a16ccbbc626e07150fcb5e8e8429909b09", 
  ... 
  "13e07e2376aa2ce1d161497e05156632262571a17bb6e1cbd333392e23b973cc"
]
$ ./bitcoin-cli -regtest getinfo
{
  "version": 140000,
  "protocolversion": 70015,
  "walletversion": 130000,
  "balance": 50.00000000,
  ...
}

Now attempt to create 30 x 0.5 BTC transactions, forcing bitcoind to spend unconfirmed change outputs in a chain until the limit is hit:

$ for n in {1..30}; do ./bitcoin-cli -regtest sendtoaddress 2NCY1Q1Mu6V5uha7fDzGNYYPUs5F1zSdXCL 0.5; done
fb15572accafe0b6d2b7b2806f31bda16280c7d97a91d99ca929d4236e5099c1
c7cea6a964ab01f20bfc54e647fc2ea9b86f1a73f64ae547dcb5fb77a9071180
1cca3556fa953a0826d34d5096353a6809df1479996caff9e47954b1f6f81dce
2ae2141d550c8d290fdc0f0313684eac0c5813836b051ab275cc3663059ce3ef
b660559b13939d4d77e2ee01907841008b09b473ed6c215388f319692312ba4b
e79a92d89b027c064c0c24df438c795b3595627ee823e532ab7c18ea87f98434
7383970de305e47f781de8451303ebe84469e31f75308f72672dd52dcdd3886b
2355d572f3cfd28f6ecae42c6ec04a739283087091512aae1360880a4b4056fe
873cb3977698284a118ef29be6ed307badab9da635049ab5a6b0c9a253bf35a3
bf66983fe782efbc6ba9ce36c5e6bfba691f8721e35a16bfcb4b35505903ebf5
f80c7d4863bb9f14175f8915ced763176b7f6d4f18680793a5929ed2671d8e5b
21fa4a20a41be5f64156ec9df037077611269fabd7de0703fa11c660014c0aa9
6aad90127ab5e3fe67fa8b9b1a13eb03892a0f0b73798ecd7cf8af53f3a59e2c
e68af2ecb142efb3b5885b5fc885905c3e08a3ebd5cbeff9ace73893aeb9451f
18980ff65d020f6e333d7562bb0cf77aa2a8dca7d3f3f53f5b3f5986cc5b43f5
8de3f866aa42c71836927d3db6e764fadb462c61c82f398a22c035104a7d5dee
a9d139b2763c5ae4000cdfe78521d41c8c4647b50de3c86e2e2d0e7daa8cc97f
0ee976f345a5805e1a260a2f2d570edf3354cf0c434bdcf42ce16752ae6cc96f
6b5327eafcd979c3f63ecd8de1139b73651b21525157f2c681b81dfdd43acf9b
acd5a269a60dc43622c30a0ecaa8b1e393f074efb61f00e225cac0c028e7809f
52204cba5012dceb2f0bdc0b1dbec771d728cf1345da5545704c0ef004523d46
824b8ab2bef5a742f0ca3fad8b58f7a226daf3c5565b0b364fdfe7013d98e8f5
fbd88cdff1a7b648b5103699321c8d035698c867a39aa1a7b0c49c1a087ab879
4fa4e846deb03bbc7a7e07364c0087d6f43cd339792a220892c53e50eb60a463
c3d33580c52cd4eee88a0d76b5df2c77653f92537a10c3d04d27c063b543d2d9
3a8b99ed748f3418a8bf9ef9ea3b167df2bfabb28d660c0027405769e1d67174
error code: -6
error message:
Insufficient funds
error code: -6
error message:
Insufficient funds
error code: -6
error message:
Insufficient funds
error code: -6
error message:
Insufficient funds

Ok looks like 26 transactions were generated. The first one spends from a coinbase and the next 25 spend unconfirmed change outputs until bitcoind errors out with "Insufficient funds". That's ok, I should still have some change left in my wallet balance right?

$ ./bitcoin-cli -regtest listunspent 0
[
]
$ ./bitcoin-cli -regtest getbalance
0.00000000
$ ./bitcoin-cli -regtest getbalance "*" 0
36.99884200

Unexpected behavior 1: Unconfirmed change is not reflected in unspent output list, but unconfirmed balance seems correct. Let's mine a block and see what happens to my transaction chain:

$ ./bitcoin-cli -regtest generate 1
[
  "5d2e7dd5bb7366fecfc36acfc6ca33346114f05d040ebb020b84465db2401e9f"
]
$ ./bitcoin-cli -regtest getblock 5d2e7dd5bb7366fecfc36acfc6ca33346114f05d040ebb020b84465db2401e9f
{
  "hash": "5d2e7dd5bb7366fecfc36acfc6ca33346114f05d040ebb020b84465db2401e9f",
  "confirmations": 1,
  "strippedsize": 5783,
  "size": 5783,
  "weight": 23132,
  "height": 102,
  "version": 536870912,
  "versionHex": "20000000",
  "merkleroot": "f7692c07ffc26637b3f1df3dcbf9077f0a6292f8e445289fc7ce817ce4c3ae82",
  "tx": [
    "d2d05cc73f95abad38c8fa59af38629f04da8dc40d638ef46852aeb0604a20ed", 
    "fb15572accafe0b6d2b7b2806f31bda16280c7d97a91d99ca929d4236e5099c1", 
    "c7cea6a964ab01f20bfc54e647fc2ea9b86f1a73f64ae547dcb5fb77a9071180", 
    "1cca3556fa953a0826d34d5096353a6809df1479996caff9e47954b1f6f81dce", 
    "2ae2141d550c8d290fdc0f0313684eac0c5813836b051ab275cc3663059ce3ef", 
    "b660559b13939d4d77e2ee01907841008b09b473ed6c215388f319692312ba4b", 
    "e79a92d89b027c064c0c24df438c795b3595627ee823e532ab7c18ea87f98434", 
    "7383970de305e47f781de8451303ebe84469e31f75308f72672dd52dcdd3886b", 
    "2355d572f3cfd28f6ecae42c6ec04a739283087091512aae1360880a4b4056fe", 
    "873cb3977698284a118ef29be6ed307badab9da635049ab5a6b0c9a253bf35a3", 
    "bf66983fe782efbc6ba9ce36c5e6bfba691f8721e35a16bfcb4b35505903ebf5", 
    "f80c7d4863bb9f14175f8915ced763176b7f6d4f18680793a5929ed2671d8e5b", 
    "21fa4a20a41be5f64156ec9df037077611269fabd7de0703fa11c660014c0aa9", 
    "6aad90127ab5e3fe67fa8b9b1a13eb03892a0f0b73798ecd7cf8af53f3a59e2c", 
    "e68af2ecb142efb3b5885b5fc885905c3e08a3ebd5cbeff9ace73893aeb9451f", 
    "18980ff65d020f6e333d7562bb0cf77aa2a8dca7d3f3f53f5b3f5986cc5b43f5", 
    "8de3f866aa42c71836927d3db6e764fadb462c61c82f398a22c035104a7d5dee", 
    "a9d139b2763c5ae4000cdfe78521d41c8c4647b50de3c86e2e2d0e7daa8cc97f", 
    "0ee976f345a5805e1a260a2f2d570edf3354cf0c434bdcf42ce16752ae6cc96f", 
    "6b5327eafcd979c3f63ecd8de1139b73651b21525157f2c681b81dfdd43acf9b", 
    "acd5a269a60dc43622c30a0ecaa8b1e393f074efb61f00e225cac0c028e7809f", 
    "52204cba5012dceb2f0bdc0b1dbec771d728cf1345da5545704c0ef004523d46", 
    "824b8ab2bef5a742f0ca3fad8b58f7a226daf3c5565b0b364fdfe7013d98e8f5", 
    "fbd88cdff1a7b648b5103699321c8d035698c867a39aa1a7b0c49c1a087ab879", 
    "4fa4e846deb03bbc7a7e07364c0087d6f43cd339792a220892c53e50eb60a463", 
    "c3d33580c52cd4eee88a0d76b5df2c77653f92537a10c3d04d27c063b543d2d9"
  ],
  "time": 1489607281,
  "mediantime": 1489607020,
  "nonce": 0,
  "bits": "207fffff",
  "difficulty": 4.656542373906925e-10,
  "chainwork": "00000000000000000000000000000000000000000000000000000000000000ce",
  "previousblockhash": "13e07e2376aa2ce1d161497e05156632262571a17bb6e1cbd333392e23b973cc"
}

Unexpected behavior 2: There is a transaction missing! The last one I generated in my unspent-change chain (3a8b99ed748f3418a8bf9ef9ea3b167df2bfabb28d660c0027405769e1d67174) did not make it into the block. Hm, ok maybe that is correct miner policy, let's see if it gets into the next block:

$ ./bitcoin-cli -regtest generate 1
[
  "2b842bed9f8e8d17b3026b8772416c6f015cfe434c0dc0ea24d8e4bfab6bc0fc"
]
$ ./bitcoin-cli -regtest getblock 2b842bed9f8e8d17b3026b8772416c6f015cfe434c0dc0ea24d8e4bfab6bc0fc
{
  "hash": "2b842bed9f8e8d17b3026b8772416c6f015cfe434c0dc0ea24d8e4bfab6bc0fc",
  "confirmations": 1,
  "strippedsize": 227,
  "size": 227,
  "weight": 908,
  "height": 103,
  "version": 536870912,
  "versionHex": "20000000",
  "merkleroot": "6efa911037aae4e7b809374ab0f812894e9db446b5fd02da3606fc9f80bacf31",
  "tx": [
    "6efa911037aae4e7b809374ab0f812894e9db446b5fd02da3606fc9f80bacf31"
  ],
  "time": 1489607296,
  "mediantime": 1489607021,
  "nonce": 1,
  "bits": "207fffff",
  "difficulty": 4.656542373906925e-10,
  "chainwork": "00000000000000000000000000000000000000000000000000000000000000d0",
  "previousblockhash": "5d2e7dd5bb7366fecfc36acfc6ca33346114f05d040ebb020b84465db2401e9f"
}

Nope, that tx is still not getting mined. Any luck with my balance or unspent output list?

$ ./bitcoin-cli -regtest listunspent 0
[
  {
    "txid": "e0aafa4fa10960b32a506d42777c48be646caa88aac223a5d3f4bee353a7187e",
    "vout": 0,
    "address": "myuKG99nWAVXUdePtE5V66jA4DQRkshUdY",
    "scriptPubKey": "2103bc5af0d827a5f6b9ae4911f343ef864f4040f078dad86059dce2a7a12e376b5eac",
    "amount": 50.00000000,
    "confirmations": 101,
    "spendable": true,
    "solvable": true
  }, 
  {
    "txid": "69274f8265b50907bb486a28a782562a41f2f1fc71e531e4806b9d16e3cc9ac5",
    "vout": 0,
    "address": "myuKG99nWAVXUdePtE5V66jA4DQRkshUdY",
    "scriptPubKey": "2103bc5af0d827a5f6b9ae4911f343ef864f4040f078dad86059dce2a7a12e376b5eac",
    "amount": 50.00000000,
    "confirmations": 102,
    "spendable": true,
    "solvable": true
  }
]
$ ./bitcoin-cli -regtest getbalance
100.00000000
$ ./bitcoin-cli -regtest getbalance "*" 0
136.99884200

No! I am still missing coins! Where is the leftover change from my transaction chain? Ok, let's restart the node, rescan, and then check my balance:

$ ./bitcoin-cli -regtest stop
Bitcoin server stopping
$ ./bitcoind -daemon -regtest -rescan
Bitcoin server starting
$ ./bitcoin-cli -regtest listunspent 0
[
  {
    "txid": "3a8b99ed748f3418a8bf9ef9ea3b167df2bfabb28d660c0027405769e1d67174",
    "vout": 1,
    "address": "mpNrtzsAU2eEeKk9NMjwMmKvvrkTBFwL8Q",
    "scriptPubKey": "76a91461314c7a9c7e4481797b9be3a2754313a45314f888ac",
    "amount": 36.99884200,
    "confirmations": 0,
    "spendable": true,
    "solvable": true
  }, 
  {
    "txid": "e0aafa4fa10960b32a506d42777c48be646caa88aac223a5d3f4bee353a7187e",
    "vout": 0,
    "address": "myuKG99nWAVXUdePtE5V66jA4DQRkshUdY",
    "scriptPubKey": "2103bc5af0d827a5f6b9ae4911f343ef864f4040f078dad86059dce2a7a12e376b5eac",
    "amount": 50.00000000,
    "confirmations": 101,
    "spendable": true,
    "solvable": true
  }, 
  {
    "txid": "69274f8265b50907bb486a28a782562a41f2f1fc71e531e4806b9d16e3cc9ac5",
    "vout": 0,
    "address": "myuKG99nWAVXUdePtE5V66jA4DQRkshUdY",
    "scriptPubKey": "2103bc5af0d827a5f6b9ae4911f343ef864f4040f078dad86059dce2a7a12e376b5eac",
    "amount": 50.00000000,
    "confirmations": 102,
    "spendable": true,
    "solvable": true
  }
]

There it is! The missing tx and my chain's final change output. Now can we mine it?

$ ./bitcoin-cli -regtest generate 1
[
  "01b49edd77e4c1a6eb19860013c8b93ea79cc4cb7c2f8f4739dc8f2478db41ba"
]
$ ./bitcoin-cli -regtest getblock 01b49edd77e4c1a6eb19860013c8b93ea79cc4cb7c2f8f4739dc8f2478db41ba
{
  "hash": "01b49edd77e4c1a6eb19860013c8b93ea79cc4cb7c2f8f4739dc8f2478db41ba",
  "confirmations": 1,
  "strippedsize": 450,
  "size": 450,
  "weight": 1800,
  "height": 104,
  "version": 536870912,
  "versionHex": "20000000",
  "merkleroot": "810e3b7be253da6114b8d6e3b644f7dcfc467e0809bc3387d23a672234901b92",
  "tx": [
    "502876265ef9c5238331c1635d2c949de9d64f2bd880ede4660dd756cb2e6f71", 
    "3a8b99ed748f3418a8bf9ef9ea3b167df2bfabb28d660c0027405769e1d67174"
  ],
  "time": 1489607415,
  "mediantime": 1489607021,
  "nonce": 0,
  "bits": "207fffff",
  "difficulty": 4.656542373906925e-10,
  "chainwork": "00000000000000000000000000000000000000000000000000000000000000d2",
  "previousblockhash": "2b842bed9f8e8d17b3026b8772416c6f015cfe434c0dc0ea24d8e4bfab6bc0fc"
}

Yes we can!

A few comments here:

• your node is probably running with the default limitancestorcount / limitdescendant count, which prevents chains of >25 transactions entering your mempool. You can override those with command line arguments.
• when you send the 26th transaction, it gets added your wallet, but is rejected by your mempool. You should see something like this in debug.log:

2017-03-15 21:41:07 AddToWallet 4c1473ae23f8c087116beb04ccb8cf1ee01226bb5a77db4cba1c5a44c8b52de6  new
2017-03-15 21:41:07 CommitTransaction(): Transaction cannot be broadcast immediately, too-long-mempool-chain

• after you generate the next block, the transaction does not get resubmitted to your mempool automatically. You can find the transaction in your wallet using gettransaction, but getrawtransaction won't show it (because it's not in the mempool):

[ubuntu] /home/ubuntu/bitcoin
→ bitcoin-cli getrawtransaction 4c1473ae23f8c087116beb04ccb8cf1ee01226bb5a77db4cba1c5a44c8b52de6
error code: -5
error message:
No such mempool or blockchain transaction. Use gettransaction for wallet transactions.
[ubuntu] /home/ubuntu/bitcoin
→ bitcoin-cli gettransaction 4c1473ae23f8c087116beb04ccb8cf1ee01226bb5a77db4cba1c5a44c8b52de6
{
  "amount": -0.50000000,
 ...
 "hex": "02000000015415b85a301a420bac34ec12d3786e1b9d8608c29c9a083306da5e0a0a065286010000006b483045022100ca16a86854d1c51c975bf3e55386bb9ca9929c252524eb1118cfb27121e4d97902206d6acedf4596b9edaea93f411474d39746cc3d5348c9de9f43d9c9c937310de5012103880da4e6b6451859b1eed794b76    7a2e5f2a8e7898aa4cc7a83f2aad8165faa13feffffff02a88e0492000000001976a9149e2f8afc219f0db384ce131d7c028753ac350a4a88ac80f0fa020000000017a914d396f368f3b4348ced0ed1f4138bb2906e12f7df8765000000"
}

• at this point, you could resubmit the transaction to your mempool by running sendrawtransaction <tx hex serialization>
• persistent mempool has nothing to do with the transaction being readded after stop-start (since the transaction never made it into the mempool in the first place). The transaction is actually in your wallet file, so when you stop-start, that's where it's being found. You can prove this to yourself by deleting mempool.dat between stopping and starting.

I think that the correct behaviour here is that the wallet should reject the transaction if it's in a chain that's too long to get accepted by the node's mempool.

Ahhh thank you for the explanation! This makes sense. Yeah I suppose the expected behavior for me would be if sendtoaddress failed with a "chain too long" error before creating that last tx, leaving one last unspent output available for a retry or wait for confirmation...

FYI there's some relevant context in #9262, #9302, and #9752.

your node is probably running with the default limitancestorcount / limitdescendant count, which prevents chains of >25 transactions entering your mempool @jnewbery Isn't it 64? I don't have that set and I just got this error: 64: too-long-mempool-chain, when I did this. (On the testnet.)

64 is the error code.

64 is the error code.

Thanks. That explains why the error message on my GUI had less than 64 transaction ids in it.

Linking to the relevant line in the code for reference: https://github.com/bitcoin/bitcoin/blob/a89221873a3ee2451c73b41bbe2d99d36f439d31/src/wallet/wallet.cpp#L3036-L3042

Is there a solution using LockUnspent ?

Generate 24 transactions with SendToAddress Using TXID of the 24th transaction GetRawTransaction txid, true (or 1 if version before 0.14.0)

• examine the vout: array; in our case there should only be two, the spend and the change
• find the vout: for the change the amount is != the amount spent the address is != the address the spend was to LockUnspent false, txid, vout
• lock that vout: so it can not be used as the UTXO for any further spend.

Lather, rinse, repeat.

Seems to me a lot simpler than loading amounts into multiple addresses, using the deprecated Account feature to identify them, and then SetAccount every group of 24 to force the wallet to move onto a different (confirmed) UTXO so that a new chain will be built.

Will this approach work ?

Stop On Apr 6, 2018 8:15 PM, "smartwombat" notifications@github.com wrote:

Is there a solution using LockUnspent ?

Generate 24 transactions with SendToAddress Using TXID of the 24th transaction GetRawTransaction txid, true (or 1 if version before 0.14.0)

• examine the vout: array; in our case there should only be two, the spend and the change
• find the vout: for the change the amount is != the amount spent the address is != the address the spend was to LockUnspent false, txid, vout
• lock that vout: so it can not be used as the UTXO for any further spend.

Lather, rinse, repeat.

Seems to me a lot simpler than loading amounts into multiple addresses, using the deprecated Account feature to identify them, and then SetAccount every group of 24 to force the wallet to move onto a different (confirmed) UTXO so that a new chain will be built.

Will this approach work ?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bitcoin/issues/10004#issuecomment-379364753, or mute the thread https://github.com/notifications/unsubscribe-auth/Ai3zhr9OqGO9n6GsIrtGPaI0xTDz86xcks5tl8zngaJpZM4Mee-O .

CommitTransaction(): Transaction cannot be broadcast immediately, no-witness-yet anyone else getting this error

My question is how to catch this relays from my api service automatically?