By specifying -txindex when initializing the database, a txid-to-diskpos index is maintained in the blktree database. This database is used to help answering getrawtransaction() RPC queries, when enabled.
Changing the -txindex value requires a -reindex; the client will abort at startup if the database and the specified -txindex mismatch.
224+ READWRITE(*(CDiskBlockPos*)this);
225+ READWRITE(VARINT(nTxOffset));
226+ )
227+
228+ CDiskTxPos(const CDiskBlockPos &blockIn, unsigned int nTxOffsetIn) : CDiskBlockPos(blockIn.nFile, blockIn.nPos), nTxOffset(nTxOffsetIn) {
229+ }
} at the end of the above line?
I’m not sure I see the issue. Yes you can make a node do lots of disk IO, but downloading the block chain does that too. It’d be nice to fix, but it’s not a new attack vector.
Apps that need indexes like that will end up just using blockchain.info or various random other sites/protocols to get what they want. It’s not like those apps will go away if the P2P network doesn’t give them what they need.
The only thing keeping history is necessary for, is bootstrapping fully validating nodes. However, that doesn’t require an index (or even a Bitcoin node at all - it could be provided by an HTTP-based file service or other protocols - something that web has plenty of). I’m sure that making a feature available on the P2P will result in infrastructure depending on it, something that would burden the nodes that provide archive data. Yes, making it an optional feature makes this much less of a problem, but I still prefer to be conservative with functions the P2P network provides unless there is a very clear use.
As far as I know, all versions before 0.7.0 maintained a full tx-to-diskpos index, and there was no way at all to query it (not even RPC). I’m quite sure this was a deliberate choice by Satoshi. In a mail about maintaining per-txout spendability of wallet transactions, he clearly said not to rely on the txindex for this, as it wouldn’t always be available.
EDIT: I dislike this feature as whole (even as RPC), as I think that services that are built not to depend on infinite history have better scalability. It’s extremely useful for debugging though, so I added it. I’m sure people will use it for other purposes as well, and that’s clearly preferable to having them depend on a centralized service for it. Providing it via P2P doesn’t add much to that, IMHO.
By specifying -txindex when initializing the database, a txid-to-diskpos
index is maintained in the blktree database. This database is used to
help answering getrawtransaction() RPC queries, when enabled.
Changing the -txindex value requires a -reindex; the client will abort
at startup if the database and the specified -txindex mismatch.
Test plan that I’ll finish executing after lunch:
Default arguments, getrawtransaction w. arbitrary old, not-in-your-wallet txid, fully-spent txn EXPECT: transaction not found
Rerun with -txindex=1 EXPECT: startup fails with “must -reindex” message
Note disk space used by blktree/ subdirectory Rerun with -txindex=1 -reindex=1 -logtimestamps=1 EXPECT: long background process to rebuild index (Q: how long?) Wait until “Reindexing finished” appears in debug.log, then: getrawtransaction EXPECT: success Q: How much extra disk space ?
Rerun with -txindex=0: EXPECT: startup fails with “must -reindex” message
Note disk space used by blktree/ Rerun with -txindex=0 -reindex=1 -logtimestamps=1 EXPECT: long background process to rebuild index (Q: how long?) Wait until “Reindexing finished” appears in debug.log, then: getrawtransaction w. old txid EXPECT: transaction not found EXPECT: txindex disk space freed from blktree/
@gavinandresen RE test plan:
Otherwise the plan looks correct and complete to me.
EDIT: -reindex doesn’t cause a slow startup, the startup is always instant, but importing (and thus building the txindex) happens in the background.
Test results: success!
Disk space: blktree/ 30M –> 600M
Time to -reindex: (note: running -g build, no -dbcache set) 1 hour 50 minutes (same time, with/without -fullindex)
I’m going to pull.
Milestone
0.8.0