Make all RPCs take a blockhash argument / allow rpc leasing? #24471

issue JeremyRubin opened this issue on March 4, 2022
  1. JeremyRubin commented at 12:08 AM on March 4, 2022: contributor

    One thing I heard interviewing some users is that it's very common to sequence RPCs with periodic tip hash checks and restart if a new block comes in.

    Something that might be helpful would be if Core could support a few different things in RPCs:

    1. a tip hash argument that causes failure if the passed_tip != actual_tip
    2. a hash argument that causes failure if the passed_tip \not \in header_chain

    another thing that might help these types of users could be node leases whereby:

    1. the node is given a lease which expires e.g. N seconds after the lease is granted/no message has been sent by the leasor, and the node will not process new messages until the lease is free or let go

    Optionally, we could allow a distinct non expiring lease.

    One might use such an API like (made up RPCs)

    finished = false
    while not finished:
        try:
            h <- getbestblockhash
            b <- getwalletbalance h
            a <- getnewaddress h
            send b a
            finished = true
    

    or

    L <- get_lease
    b <- getwalletbalance h
    a <- getnewaddress h
    send b a
    clear_lease L
    

    Leasing might be better than just a blockhash argument because you can guarantee e.g. repeated calls to mempool operations are consistent. This could be problematic e.g. if a txn is in the mempool and you're trying to coordinate wallet software to detect txns paying you so that you might fee bump them with CPFP, it might disappear while your flow of txns is occuring.

  2. JeremyRubin added the label Feature on Mar 4, 2022
  3. maflcko commented at 2:10 PM on March 7, 2022: member

    See also #18567

  4. JeremyRubin commented at 3:13 PM on March 7, 2022: contributor

    @danielabrozzoni @afilini do you have thoughts on this?

  5. afilini commented at 1:37 PM on March 8, 2022: none

    The approach described in #18567 is very reasonable and probably much easier to implement than locks and whatnot.

    Assuming the calling software can complete its task within the block time (which probably applies to most applications) it should work very well.

    I could try working on that as soon as I'm done with other BDK stuff.

  6. JeremyRubin commented at 4:29 PM on March 8, 2022: contributor

    will that be sufficient though? I would assume you'd want to e.g. lock the mempool? Maybe we can allow locking to just lock mempool and not blocks?

  7. danielabrozzoni commented at 5:41 PM on March 9, 2022: contributor

    I agree that returning the hash might not be sufficient for some use cases, but it's better than nothing.

    Ideally, being able to lock the mempool and/or the blocks would be great (even tho I can't really think of any use case where one wants to lock just the mempool and not the blocks) - but, as Alekos was saying, it seems to be a bit complicated to implement.

  8. promag commented at 5:21 PM on March 21, 2022: contributor

    I agree that returning the hash might not be sufficient for some use cases, but it's better than nothing.

    Can you describe those use cases?

  9. danielabrozzoni commented at 9:22 PM on March 23, 2022: contributor

    The main issue would be: your application wants to sync with bitcoind, in doing so it does subsequent calls to the RPC, all the calls return the same blockhash, so you would assume the calls' results are consistent between each others, but they might not. This can happen if the calls depend on the mempool, and the mempool somehow changed between the calls.

    The RPC calls that come to my mind that would have this problem are listunspent and listtransactions. (If there's a way to ask listunspent and listtransactions to ignore the mempool, I'm not aware of it).

  10. promag commented at 10:36 PM on March 23, 2022: contributor

    The RPC calls that come to my mind that would have this problem are listunspent and listtransactions. (If there's a way to ask listunspent and listtransactions to ignore the mempool, I'm not aware of it).

    It's the wallet that would have to ignore the mempool, I think that could be done. A wallet could be loaded but be "disconnected" from the mempool and the chain. In this case, the wallet RPC's should return the latest block processed by the wallet. However, I think the wallet should be "readonly" (like watchonly) and all spend calls should fail. The GUI would have to be aware of this as well.

    I understand leases/locks can simplify sync on the client-side. However, I don't think it's that great because:

    • if bitcoind is shared then one client can block the others
    • if a client crashes its locks/leases could be dangling resulting in an unusable bitcoind
    • the client would still have to handle re-sync after some time and/or some notifications, ideally not from scratch

    The following are what I consider good tips for sync methods:

    • be idempotent, re-processing the same data (block/tx/notification) should not result in more changes in the client state, this helps incremental syncs
    • listen for zmq notifications on startup and use them to increment or invalidate some client state
    • batch process, for instance, avoid listing all transactions
  11. maflcko commented at 7:40 AM on March 24, 2022: member

    What about (optionally) also returning the mempool sequence number?

  12. JeremyRubin commented at 5:58 PM on March 24, 2022: contributor

    i think it wouldn't be too useful because very likely things will change during your call and you'll end up spinning for a long time.

  13. JeremyRubin commented at 5:59 PM on March 24, 2022: contributor

    wild option:

    add a small LISP with some special ops for handling JSON (i think we already have something like this in the Gui console?) and then allow sending a fully atomic sequence of operations.

  14. ryanofsky commented at 11:16 PM on March 24, 2022: contributor

    add a small LISP with some special ops for handling JSON (i think we already have something like this in the Gui console?) and then allow sending a fully atomic sequence of operations.

    You may be interested in #20273 which ports the gui nested logic to the cli. Presumably it could ported to run atomically in the server, too.

    Also has anybody suggested extending JSONRPCExecBatch to support atomic operations (or just letting multiple RPC calls share state)?

  15. willcl-ark commented at 2:50 PM on October 14, 2024: member

    There doesn't seem to be much action on implementing this feature.

    Please feel free to submit a pull request if this is something that you still want. This could be done for specific RPCs.

  16. willcl-ark closed this on Oct 14, 2024

  17. bitcoin locked this on Oct 14, 2025

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-15 21:14 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me