`getrawtransaction` fails (0.18.0) #16142

<!-- Describe the issue -->

getrawtransaction fails

<!--- What behavior did you expect? -->

Return decoded raw tx

<!--- What was the actual behavior (provide screenshots if the issue is GUI-related)? -->

• bitcoin-qt -prune=8192 -server -rpcuser=rpc -rpcpassword=rpc -onlynet=ipv4 -rpcbind=127.0.0.1:18332 -testnet
  • BTC 0.18.0
  • getrawtransaction b423b2b22d6d74376de2887c07f7c84ae890a93d541fcc6dbf3686f2923b2753 1
  • BTC 0.17.1
  • getrawtransaction b423b2b22d6d74376de2887c07f7c84ae890a93d541fcc6dbf3686f2923b2753 1

<!--- How reliably can you reproduce the issue, what are the steps to do so? -->

Request getrawtransaction.

<!-- What version of Bitcoin Core are you using, where did you get it (website, self-compiled, etc)? -->

Official from site.

<!-- What type of machine are you observing the error on (OS/CPU and disk type)? -->

Linux pentagon 4.9.0-9-amd64 [#1](/bitcoin-bitcoin/1/) SMP Debian 4.9.168-1+deb9u2 (2019-05-13) x86_64 GNU/Linux, u5200, 2x8Gb, SSD

<!-- For the GUI-related issue on Linux provide names and versions of a distro, a desktop environment and a graphical shell (if relevant). -->

QT

<!-- Any extra information that might be useful in the debugging process. -->

<!--- This is normally the contents of a `debug.log` or `config.log` file. Raw text or a link to a pastebin type site are preferred. -->

Please see the error message for alternatives.

For some reason it doesn't mention that you can also provide the block hash as parameter to getrawtransaction. Imo, it should.

In your case getrawtransaction b423b2b22d6d74376de2887c07f7c84ae890a93d541fcc6dbf3686f2923b2753 1 00000000000000409e394d133807fd4f6b0fd27ade24a6e717ee257e825167e1

This is intended behavior. Transactions that have outputs in the UTXO set are no longer looked up in getrawtransaction. See #15159

@MarcoFalke

In your case getrawtransaction b423b2b22d6d74376de2887c07f7c84ae890a93d541fcc6dbf3686f2923b2753 1 00000000000000409e394d133807fd4f6b0fd27ade24a6e717ee257e825167e1 @achow101

Look:

• i use SAME data from 0.17.1 on 0.18.1
• i open CLEAN console JUST after start
• i type getrawtransaction b423b2b22d6d74376de2887c07f7c84ae890a93d541fcc6dbf3686f2923b2753 1
• And i get those 2 errors (!)
  • it write executing commant using wallet (probably not error, maybe just notification for me)
  • it can't find TX data in saved blockchain part

There are NOT only getrawtransaction failed, there are really strange behavior for 0.18.0.

@nikitasius Again, this (breaking) change was intended for 0.18.0. Could you please type in

getrawtransaction b423b2b22d6d74376de2887c07f7c84ae890a93d541fcc6dbf3686f2923b2753 1 00000000000000409e394d133807fd4f6b0fd27ade24a6e717ee257e825167e1

Note that it uses three arguments. You can use help getrawtransaction to see their help text.

@MarcoFalke this way it work.

So: shortly now everyone obliged to call this call with adding blockhash information?

There are no more way just give txid to client? Everyone must call listunspent to get unspent tx and then call idk what else need to look btc manual to get blockhash for tx i need. I'm correct?

yes,

• listunspent if needed
• gettransaction with tx id
• getrawtransaction to get what i really need. @MarcoFalke on next 0.18.1 it will be 1 more call to do?

@nikitasius Note that you can turn on txindex, which will enable using getrawtransaction with just the txid. But you need to enable it, because it now requires an index to query for these things efficiently.

@nikitasius Also, if it's about wallet transactions, you can just only call gettransaction with the decode flag enabled; no need for getrawtransaction ever.

All that was removed is the ability to use getrawtransaction for non-wallet transactions, without enabling txindex.

@nikitasius Also, if it's about wallet transactions, you can just only call gettransaction with the decode flag enabled; no need for getrawtransaction ever.

All that was removed is the ability to use getrawtransaction for non-wallet transactions, without enabling txindex.

Before i was requesting wallet transactions and non-wallet (parent tx), to get info about parent tx.

About txindex:

• i have it enabled on my prod
• on dev and laptop i use with prune and it was handy to get info about recent transactions who sent me coins on testnet.

Then, for example, i pick an old tx from blockchain:

• curl -u rpc:rpc -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","id":"1","method":"getrawtransaction","params":["f38888cd5f99b54bbd32c9f4c0444345277f8f8d5803dc93c5af319378997953",1,"00000000001012f9088a1f33694881732c6f168bc13c9894eaeb3dcbfc3d6893"]}' http://127.0.0.1:18332
  • on laptop without txindex {"result":null,"error":{"code":-1,"message":"Block not available"},"id":"1"} - no result
  • on dev with txindex {"result":{"in_active_chain":true,"txid":"f38888cd5f99b54bbd32c9f4c0444345277f8f8d5803dc9.... - correct result

So, with this actual changes we:

• lost abillity to get raw information about ALL latest transactions (what inside blockchain data we stored) on prunes instances @sipa you allow in 0.18.0 to get raw information when you specify blockhash, but it's still limited by the blockchain you store (8Gb for example).

This is totally unuseful update. It can be useful in one situation:

• you allow to get information about ANY transaction on blockhain no matter how much you store.

In such case YES, this is useful to get blockhash and use it.

But actually:

• we lost functionality we had without any good reasons for.

And this is very bad.

It would be actually useful to have a new RPC call to query the blockId of a transaction by using the UTXO set, so at least people have an alternative which does not rely on the wallet.

It would be actually useful to have a new RPC call to query the blockId of a transaction by using the UTXO set, so at least people have an alternative which does not rely on the wallet.

it will be useful to add functionality we had before:

• call with txid only.

Due 0.18.0 allow of browse in the data we stored anyway. And this stupid update forcing people to get blockhash.

Look the ORIGINAL idea: #3220 (comment) @jimpo explained really nice. And i personally like this idea.

But actual realisation sucks.

upd: on production less calls you do - better you are. Actually this update increase processing time to ~ 30% at least.

@NicolasDorier You can do that using gettxout, but it's generally unreliable, as we can't find transactions which don't have any unspent outputs left. The goal here is making it more predictable what will work:

• For wallet transactions, use gettransaction.
• For mempool transactions, use getrawtransaction with just the txid.
• For confirmed transactions, if you have txindex enabled, use getrawtransaction with just txid.
• For confirmed transactions, if you don't have txindex enabled, use getrawtransaction with txid/block hash arguments.

Oh I was completely unaware that gettxout existed.

@sipa could you please give your opinion why it's like this:

• 0.18.0
• 0.17.1

This is my tx (from 0.17.1):

@nikitasius It's a confirmed transaction, so now you need to either enable txindex, or use the blockhash version. In 0.17 this was possible without but it was (a) unreliable (b) confusing (c) slow. That's why the functionality was removed.

@nikitasius You can use gettransaction for wallet transactions. (It also includes the raw hex)

@nikitasius It's a confirmed transaction, so now you need to either enable txindex, or use the blockhash version. In 0.17 this was possible without but it was (a) unreliable (b) confusing (c) slow. That's why the functionality was removed. @sipa but i still can't go deeper eve with blockhash, i posted my example here: #16142 (comment)

short (from the link): i picked up random block and tx from. On my dev server with txindex i can pick the data even without blockhash, for old and for recent txs.

But on my laptop (prune 8 Gb) i obliged to precise blockhash for all TX what i have in my stored data. I have no access to this old tx.

Why you can't get blockhash internally, and add it internally when rpc client asking getrawtransaction for a recent tx?

Internal (inside bitcoin client) it will be faster, while rpc clients (who work with pruned data) oblliged to pick the blockhash before.

With this update you ruined flow for all small clients. For me this is looks like "use txindex OR send 1 more request per tx". 1 more request per tx it;s a collosal waste of time when you process many tx, due client (rpc) must establish a connection with server (rpc).

@nikitasius You can use gettransaction for wallet transactions. (It also includes the raw hex) @MarcoFalke , i don't use it only for wallet tx. I use it for all tx

• my wallet tx
• parent tx, where funds was sent before someone sent it to me

@nikitasius Then you should enable txindex. It'll make getrawtransaction work like before, and much faster.

@sipa how can i work on 0.18.0 like before (=on 0.17.1) without txindex (=wasting my space) and requesting blockhash (=wasting processing time)?

You can't, by design, because it was always unreliable.

Try it, in 0.17, go back a couple of txids, it will only work when there is at least one unspent output of the txid remaining. As that was unpredictable, the functionality was removed.

You can't, by design, because it was always unreliable.

Try it, in 0.17, go back a couple of txids, it will only work when there is at least one unspent output of the txid remaining. As that was unpredictable, the functionality was removed.

from example:

• https://live.blockcypher.com/btc-testnet/tx/f38888cd5f99b54bbd32c9f4c0444345277f8f8d5803dc93c5af319378997953/

They are all spent.

I run curl -u rpc:rpc -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","id":"1","method":"getrawtransaction","params":["f38888cd5f99b54bbd32c9f4c0444345277f8f8d5803dc93c5af319378997953",1]}' http://127.0.0.1:18332 in 0.17.1 and i get:

{
	"result": {
		"txid": "f38888cd5f99b54bbd32c9f4c0444345277f8f8d5803dc93c5af319378997953",
		"hash": "f38888cd5f99b54bbd32c9f4c0444345277f8f8d5803dc93c5af319378997953",
		"version": 1,
		"size": 590,
		"vsize": 590,
		"weight": 2360,
		"locktime": 0,
		"vin": [{
			"txid": "a9e268e17c031cd5a4bb5263e84668493cf38e34b46338b5c4d5b1ce4f944d1d",
			"vout": 0,
			"scriptSig": {
				"asm": "3045022100eea6b8b4ffa8aa5c3893810231c0f39c4d33cde7cc6956a98128e393076aefc302206806317c984a1045494198727bacc9f8acbd50d22df18dd290c3c3498ced22dd[ALL] 0226ec2235e760f31ffcfa8911c244070dc012c077b5864f983fcd82894ae63efa",
				"hex": "483045022100eea6b8b4ffa8aa5c3893810231c0f39c4d33cde7cc6956a98128e393076aefc302206806317c984a1045494198727bacc9f8acbd50d22df18dd290c3c3498ced22dd01210226ec2235e760f31ffcfa8911c244070dc012c077b5864f983fcd82894ae63efa"
			},
			"sequence": 4294967295
		}, {
			"txid": "b38b902dabb0dad3227ddf8ecb2647ed1b23b033f678558f2db9dcec6ab78dcc",
			"vout": 3,
			"scriptSig": {
				"asm": "3045022013e9281d2c97519983f655612af2e1b67aa8222c880a83d4ae95a082981cb3d90221009dd4f34c30aeb77118a9250bef157d270a7e99eeced15520e6cd7a5c1ee47fdc[ALL] 022dfcddf66d99d48b1f34ae286cb1ca58439a5ca3651f8c3c287479bb10c0baf1",
				"hex": "483045022013e9281d2c97519983f655612af2e1b67aa8222c880a83d4ae95a082981cb3d90221009dd4f34c30aeb77118a9250bef157d270a7e99eeced15520e6cd7a5c1ee47fdc0121022dfcddf66d99d48b1f34ae286cb1ca58439a5ca3651f8c3c287479bb10c0baf1"
			},
			"sequence": 4294967295
		}, {
			"txid": "a868e9d25d76991ba79f2c186accd111a8957f2601328f4b9e79334460f2a9e0",
			"vout": 1,
			"scriptSig": {
				"asm": "3045022100ab56e741949243668b75c9beff9c2fc372238e4ca51c2e4ec3f21c3d8e528afa02200a61685980aedcb0e2522ae4d3113f08a3aca312668daba7ab193be68655f9df[ALL] 03f9576892cdd721d39a7a2e8d54acd364c8b3925d8fa77a192700c9694a6467d4",
				"hex": "483045022100ab56e741949243668b75c9beff9c2fc372238e4ca51c2e4ec3f21c3d8e528afa02200a61685980aedcb0e2522ae4d3113f08a3aca312668daba7ab193be68655f9df012103f9576892cdd721d39a7a2e8d54acd364c8b3925d8fa77a192700c9694a6467d4"
			},
			"sequence": 4294967295
		}],
		"vout": [{
			"value": 0.01200000,
			"n": 0,
			"scriptPubKey": {
				"asm": "OP_DUP OP_HASH160 25d8205c309cb74973c1df74c9615bf56805b100 OP_EQUALVERIFY OP_CHECKSIG",
				"hex": "76a91425d8205c309cb74973c1df74c9615bf56805b10088ac",
				"reqSigs": 1,
				"type": "pubkeyhash",
				"addresses": ["miy4CsrzKRabbnNTS4kMCpjFGFKeWkwNj8"]
			}
		}, {
			"value": 0.01800000,
			"n": 1,
			"scriptPubKey": {
				"asm": "OP_DUP OP_HASH160 81281226ab26d105d3eb560ef2ac4ee7c86df714 OP_EQUALVERIFY OP_CHECKSIG",
				"hex": "76a91481281226ab26d105d3eb560ef2ac4ee7c86df71488ac",
				"reqSigs": 1,
				"type": "pubkeyhash",
				"addresses": ["msHsTyX3DGdMTKBejFnYN8oraMZHLMAfg3"]
			}
		}, {
			"value": 0.02400000,
			"n": 2,
			"scriptPubKey": {
				"asm": "OP_DUP OP_HASH160 7bba7362acf752c8c6a75d39da3f7de49d79de12 OP_EQUALVERIFY OP_CHECKSIG",
				"hex": "76a9147bba7362acf752c8c6a75d39da3f7de49d79de1288ac",
				"reqSigs": 1,
				"type": "pubkeyhash",
				"addresses": ["mroAmUN7HxZZ2us5h2j2u2vCHRBJJTZRgG"]
			}
		}, {
			"value": 0.01246997,
			"n": 3,
			"scriptPubKey": {
				"asm": "OP_DUP OP_HASH160 05243244d1accdda3d93bb4356235b3838576df5 OP_EQUALVERIFY OP_CHECKSIG",
				"hex": "76a91405243244d1accdda3d93bb4356235b3838576df588ac",
				"reqSigs": 1,
				"type": "pubkeyhash",
				"addresses": ["mfz92ypn7cccGBEaENTE2PxxLU9VqheDSh"]
			}
		}],
		"hex": "01000000031d4d944fceb1d5c4b53863b4348ef33c496846e86352bba4d51c037ce168e2a9000000006b483045022100eea6b8b4ffa8aa5c3893810231c0f39c4d33cde7cc6956a98128e393076aefc302206806317c984a1045494198727bacc9f8acbd50d22df18dd290c3c3498ced22dd01210226ec2235e760f31ffcfa8911c244070dc012c077b5864f983fcd82894ae63efaffffffffcc8db76aecdcb92d8f5578f633b0231bed4726cb8edf7d22d3dab0ab2d908bb3030000006b483045022013e9281d2c97519983f655612af2e1b67aa8222c880a83d4ae95a082981cb3d90221009dd4f34c30aeb77118a9250bef157d270a7e99eeced15520e6cd7a5c1ee47fdc0121022dfcddf66d99d48b1f34ae286cb1ca58439a5ca3651f8c3c287479bb10c0baf1ffffffffe0a9f2604433799e4b8f3201267f95a811d1cc6a182c9fa71b99765dd2e968a8010000006b483045022100ab56e741949243668b75c9beff9c2fc372238e4ca51c2e4ec3f21c3d8e528afa02200a61685980aedcb0e2522ae4d3113f08a3aca312668daba7ab193be68655f9df012103f9576892cdd721d39a7a2e8d54acd364c8b3925d8fa77a192700c9694a6467d4ffffffff04804f1200000000001976a91425d8205c309cb74973c1df74c9615bf56805b10088ac40771b00000000001976a91481281226ab26d105d3eb560ef2ac4ee7c86df71488ac009f2400000000001976a9147bba7362acf752c8c6a75d39da3f7de49d79de1288ac15071300000000001976a91405243244d1accdda3d93bb4356235b3838576df588ac00000000",
		"blockhash": "00000000001012f9088a1f33694881732c6f168bc13c9894eaeb3dcbfc3d6893",
		"confirmations": 1367877,
		"time": 1386558263,
		"blocktime": 1386558263
	},
	"error": null,
	"id": "1"
}

As you changed this part, do you have an example? tx id (or couple) what will fail?

You must have enabled txindex at some point.

You must have enabled txindex at some point.

I got what you mean, i had it in my logs.

here is my laptop, prune 8Gb (so no txindex):

• 0.17.1
  • have unspent https://live.blockcypher.com/btc-testnet/tx/6974bfdd8137136854c57c7527be95911a08b6c301d5232bcd17ca8cabe73b95/
  • all spent https://live.blockcypher.com/btc-testnet/tx/618151a2a6ce401fa4ab98b2d97c1cf8221b5ae1ac1001816b90630aca75d353/
• 0.18.0
  • no error for ....30aca75d353

This is nice improvement, you made it stable and predictable.

But what about:

• When user request tx by txid and do not setup blockhash, btc client look for this field itself.

This improvement is possible? To avoid an additional API calls to RPC server (=wasting of time on processing).