Add listsinceblock command for retrieving all wallet transactions in blocks after the specified block #199

1. 
   cdhowie commented at 4:22 pm on May 6, 2011: contributor

   This commit adds a “listsinceblock” RPC command that will accept a block hash and list all of the wallet transactions that have occurred after that block. If no block is specified, it will list all wallet transactions. The output also includes the block ID of the latest block on the main chain.

   This is designed to simplify the process of reconciling transactions it bitcoind with another database; you store the most recent block you have seen (the “latest block” result from your last reconcile) and, on the next reconcile pass this block identifier. bitcoind will then tell you about all the transactions you need to process (possibly including transactions you have already processed, if the latest block the last time you reconciled has since been orphaned).

2. 
   gavinandresen commented at 1:58 pm on May 7, 2011: contributor

   How did you test that this does the right thing through a block split/reorg?
3. 
   cdhowie commented at 5:35 pm on May 7, 2011: contributor

   I didn’t, since I’m not sure how I could create such a situation with the amount of computing power I have. I believe it will do the right thing since I’ve read the CBlockLocator sources.
4. 
   gavinandresen commented at 6:41 pm on May 7, 2011: contributor

   You should be able to pretty easily simulate block splits by setting up a small “testnet-in-a-box” network – see https://sourceforge.net/projects/bitcoin/files/Bitcoin/testnet-in-a-box/

   Process would be something like:

   Send a transaction or three while generating on testnet-in-a-box. Verify that the transactions are confirmed. Then shutdown that testnet-in-a-box and start a new one.

   Generate a longer chain.

   Then join the two chains (restart a -nolisten testnet-in-a-box node from the shorter chain, and -connect it to the longer-chain testnet-in-a-box) and make sure listsinceblock does what you expect.

   In case it doesn’t, you should probably save away the shorter and longer chains (just save the datadirs somewhere), so re-testing is easy.

5. 
   sipa commented at 12:13 pm on May 8, 2011: member

   CMerkleTx’s (and by extension CWalletTx’s) support looking up in which block they are (see CMerkleTx::GetDepthInMainChain(CBlockIndex* &)). I believe it is better to loop over all wallet transactions, and filter them by looking up where they are in the main chain, than to loop over the whole block chain (loading it from disk), and then finding wallet transactions that match it. The block chain won’t be available in client-only mode anyway.
6. 
   cdhowie commented at 6:59 pm on May 25, 2011: contributor

   Gavin, I did the test today and the command does do what I expect. This was after modifying the command’s implementation per sipa’s suggestion.
7. 
   brandonjank commented at 9:01 pm on June 3, 2011: none

   Patched 322rc4 with this the other day. Seems to be working, great feature. Merge :D
8. 
   pixelglow commented at 3:35 am on June 28, 2011: none

   Might be useful to have block id against each txn, not just the last block id. For example, perhaps I’m searching for all txn’s with confirmations > n. If I find some txn’s with confirmations <= n, I have to note them down in the sync’ed database and recheck them later. But if we got block id’s back for the txn’s, we could restart the scan somewhere else other than the last block to look for these confirmations. Also would be good to get some sense of time ordering of the block id’s as well.
9. 
   cdhowie commented at 7:40 pm on June 28, 2011: contributor

   @pixelglow: I had that in at some point and Gavin asked that I remove it. :) But it’s a one line change to add it.

   The transactions should be listed sorted by block. So in terms of the time that the blocks were created, you can infer that from the time the transaction happened. If all you care about is which blocks came before which blocks, then you don’t even have to look at the transaction times, just their order.

10. 
    pixelglow commented at 11:58 pm on June 28, 2011: none

    I suppose there are two ways of adding block id’s for txn’s. You could tag each txn with the block id e.g.[ { txid: 1, block: abc }, {txid: 2, block: abc} … ] but since there are multiple txn per block and you are already sorted by block order you can do a more normalized { block: abc; txns: [ { txid: 1}, {txid : 2} …] }. The feature in general would make it so convenient to pick an arbitrary block to get txn’s forward from e.g. if you are looking for confirmed txn only.
11. 
    gavinandresen commented at 2:02 pm on June 29, 2011: contributor

    Block chain re-orgs change what blocks a given transaction is in, which is why I’m dead-set against putting them in transaction information. Transactions already have confirmations, which tells you how far back in the best-block-chain to look for the block that contains that transaction.
12. 
    cdhowie commented at 5:32 pm on June 29, 2011: contributor

    I’m thinking that a “minimum confirmations” parameter would solve this nicely. If you only want transactions with six confirmations or more, you call “listsinceblock (block-id) 6” and all of the recent transactions with 6+ confirmations would be reported. The “lastblock” property on the result object would then contain the hash of the sixth most recent block instead of the most recent. This way, when you run the command later, transactions that had under 6 confirmations last time will be reported if they have 6+ now.
13. 
    pixelglow commented at 11:20 pm on June 29, 2011: none

    @cdhowie that is an excellent idea and solves exactly the issue I have with confirmations. I can’t see any other reason why someone would need to look at earlier blocks than the last, other than checking confirmations. (But then again, I’m a newbie when it comes to bitcoin :-) ).
14. 
    cdhowie commented at 8:25 pm on July 6, 2011: contributor

    I’ve implemented the requested change; the second parameter specifies the depth of the block you want to obtain as the “lastblock” return property. The default of 1 will return the latest block ID, while 2 will return the second-latest block ID, and so on.

    The command now outputs transactions with zero confirmations in addition to those in the newer blocks so that applications can act on them in some way, such as show them as pending in a UI.

    Note that the second parameter does not in any way affect which transactions are returned; if you only care about transactions with N confirms, you will pass N as the second parameter, and then filter the results down to only those with N+ confirmations. This will ensure that the next time you run the command with the new lastblock value you won’t miss any transactions that have now reached N confirmations. But it will allow applications that want to specially handle transactions with <N confirmations to do so as well. So this command neatly hits both use-cases.

    Edit: I’ve also rebased against master.

15. Add listsinceblock command for retrieving all wallet transactions in blocks after the specified block 5b2f35167f
16. listsinceblock now shows txns with 0 confirms, as well as allows the lastblock return property to be targeted to the block with the specified depth 76aed0141c
17. 
    pixelglow commented at 7:05 am on July 18, 2011: none

    I’ve integrated this into my bitcoin exchange (currently only as a JSON API to the backend) and it works quite well in replaying transactions into the PostgreSQL backend. Thanks. Can’t wait to see this in the mainline.

    http://forum.bitcoin.org/index.php?topic=29057.0

18. gavinandresen merged this on Oct 5, 2011
19. gavinandresen closed this on Oct 5, 2011

    ---

20. dexX7 referenced this in commit 2052bc14e2 on Nov 15, 2014
21. sipa referenced this in commit 9d09322b41 on Mar 27, 2015
22. TheBlueMatt referenced this in commit 582b2934e6 on Oct 20, 2015
23. btcdrak referenced this in commit b7f69edd34 on Nov 28, 2015
24. deadalnix referenced this in commit d9e49ba3aa on Jan 3, 2017
25. deadalnix referenced this in commit fc8285f18a on Jan 19, 2017
26. lateminer referenced this in commit 49af99bab4 on Dec 9, 2017
27. lateminer referenced this in commit 337e9b3ee7 on Dec 9, 2017
28. lateminer referenced this in commit bf2535d495 on Dec 9, 2017
29. classesjack referenced this in commit d93a783c9d on Jan 2, 2018
30. nining referenced this in commit 0bddaaed1d on Jan 3, 2018
31. attilaaf referenced this in commit 9c38726c91 on Jan 13, 2020
32. rajarshimaitra referenced this in commit 114387e822 on Aug 5, 2021
33. DrahtBot locked this on Sep 8, 2021

Contributors
cdhowie gavinandresen sipa brandonjank pixelglow

Labels
Feature