[blockchain] make `getrawtransaction` useful again on pruned nodes #16182

in the version 0.18.0:

The getrawtransaction RPC & REST endpoints no longer check the unspent UTXO set for a transaction. The remaining behaviors are as follows: 1. If a blockhash is provided, check the corresponding block. 2. If no blockhash is provided, check the mempool. 3. If no blockhash is provided but txindex is enabled, also check txindex.

• I run pruned nodes for dev purposes. And i have problems:
  • #16142
• Here i explain why i walk on blockchain and use this call:
  • #15159 (comment)

Whats wrong:

• without txindex enabled, getrawtransaction are became worthless
• to process wallet tx we need to do x2 more calls:
  • use gettransaction to get blockhash
  • then call getrawtransaction (with blockhash) or decoderawtransaction (with hash) to get rawtransaction body
• to process non wallet transactions it completely fails :scream:

Here #16181 @MarcoFalke offers to decode rawtx data inside gettransaction. This is unuseful overload for an existing limited method (=api call what work only for in-wallet tx).

What i offer:

• Check the mempool and return if tx is found
• if txindex enabled look for this tx and return tx or an error what tx not found
• if txindex disabled look inside the data, stored on the node (prune options) and return if tx is found or error message if tx not found

Like this we can use getrawtransaction to work with transactions on blockchain (fully or partially).

We store data, 4Gb or 8Gb - let the client use it!

greatly explained idea is here: #16181 (comment)

Concept NACK, I don't think linearly searching through the block data is remotely acceptable without txindex (it'd easily take ~hour per RPC even on a fast system).

An alternative discussed elsewhere is making txindex work with a pruned blockchain. I suspect that's not very useful, as it means applications built on top still couldn't rely on getrawtransaction working, and would need a fallback anyway. I'm eager to hear other opinions, though.

I don't think we should provide an unstable API to a partial txindex (which is what this would effectively be), since it'll generate user confusion and questions. Anyone who built on this API would need to deal with cases where the tx is in a block that has been pruned anyway.

@jnewbery there are no problem to deal with pruned data, otherwise you should to remove this option. I use 16Gb prunes data on dev nodes and 8Gb on laptop, this offer me a huge date period.

About perfomance for partial txindex: i don't know, how you made your txindex. But storing txid:blockhash in binary tables is a cheap and fast solution. If you have tricky binary tables, you will be able find data faster i guess.

Discussion should move to the issues linked from here rather than on this closed PR.

I would like to make one observation about the last comment (#15159 (comment)). A lot of the complexity comes from the fact that you're trying to apply a 'from address'. Such a concept doesn't exist in bitcoin. A transaction can have multiple txins, each of which refer to a txout that may have a scriptPubKey that encodes to an address.

Addresses should be seen more as single-use invoice identifiers. Trying to apply a from/to address model doesn't fit with the Bitcoin UTXO model and will only cause you headaches! @jnewbery yes, you right about vin's, but this complexity is simplified on user side. I just "show only first address" for user. But i can keep multiple addresses for statistic purposes (if i need to check which addresses used often, etc). This is how such concept exists inside the code realisation what uses bitcoin core wallet.

The product i do and i'm a part of providing centralised and unified UI for bitcoin and other cryptocurrencies, allowing to use features of each other, sure.

P.S. with XMR it fail i know, this is exception. But while we can get list of addresses, who has been involved into transaction, we can display then to user, and display a tx id too.

An alternative discussed elsewhere is making txindex work with a pruned blockchain. I suspect that's not very useful, as it means applications built on top still couldn't rely on getrawtransaction working, and would need a fallback anyway. I'm eager to hear other opinions, though. @sipa with manual pruning it might make sense in order to save resources - the user prunes because he knows those blocks/transactions are no longer needed.

The feature request didn't seem to attract much attention in the past. Also, the issue seems not important enough right now to keep it sitting around idle in the list of open issues.

Closing due to lack of interest. Pull requests with improvements are always welcome.