Reverse cs_main, cs_wallet lock order and reduce cs_main locking #16426

pull ariard wants to merge 5 commits into bitcoin:master from ariard:2019-07-remove-more-locking-chain changing 11 files +257 −379
  1. ariard commented at 6:25 am on July 20, 2019: member

    This change is intended to make the bitcoin node and its rpc, network and gui interfaces more responsive while the wallet is in use. Currently, because the node’s cs_main mutex is always locked before the wallet’s cs_wallet mutex (to prevent deadlocks), cs_main currently stays locked while the wallet does relatively slow things like creating and listing transactions.

    Switching the lock order so cs_main is acquired after cs_wallet allows cs_main to be only locked intermittently while the wallet is doing slow operations, so the node is not blocked waiting for the wallet.

    To review the present PR, most of getting right the move is ensuring any LockAssertion in Chain::Lock method is amended as a LOCK(cs_main). And in final commit, check that any wallet code which was previously locking the chain is now calling a method, enforcing the lock taking job. So far the only exception I found is handleNotifications, which should be corrected.

  2. fanquake added the label Refactoring on Jul 20, 2019
  3. fanquake added the label Wallet on Jul 20, 2019
  4. DrahtBot commented at 6:53 am on July 20, 2019: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #18699 (wallet: Avoid translating RPC errors by MarcoFalke)
    • #18698 (Make g_chainman internal to validation by MarcoFalke)
    • #18592 (rpc: replace raw pointers with shared_ptrs by brakmic)
    • #18554 (wallet: ensure wallet files are not reused across chains by mrwhythat)
    • #18531 (rpc: Assert that RPCArg names are equal to CRPCCommand ones by MarcoFalke)
    • #18354 (Protect wallet by using shared pointers by bvbfan)
    • #18244 (rpc: fundrawtransaction and walletcreatefundedpsbt respect locks even with manual coin selection by Sjors)
    • #18202 (refactor: consolidate sendmany and sendtoaddress code by Sjors)
    • #16549 ([WIP] UI external signer support (e.g. hardware wallet) by Sjors)
    • #16546 (External signer support - Wallet Box edition by Sjors)
    • #16224 (gui: Bilingual GUI error messages by hebasto)
    • #15719 (Drop Chain::requestMempoolTransactions method by ryanofsky)
    • #14707 ([RPC] Include coinbase transactions in receivedby RPCs by andrewtoth)
    • #14582 (wallet: always do avoid partial spends if fees are within a specified range by kallewoof)
    • #10102 ([experimental] Multiprocess bitcoin by ryanofsky)
    • #9381 (Remove CWalletTx merging logic from AddToWallet by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  5. ryanofsky commented at 11:17 am on July 20, 2019: member

    Great work! I’d suggest changing the PR title to something like “Reverse cs_main, cs_wallet lock order and reduce cs_main locking” to motivate it better and describe the change.

    The current title “Remove Chain::Lock interface” and starting sentence “Follow-up in the set of Chain interface refactoring” make this sound mostly like a refactoring. But this is more of a locking change, and a change to make the wallet more asynchronous and event driven than a refactoring change.

  6. ariard renamed this:
    Remove Chain::Lock interface
    Reverse cs_main, cs_wallet lock order and reduce cs_main locking
    on Jul 20, 2019
  7. practicalswift commented at 11:00 am on July 22, 2019: contributor

    Concept ACK

    Excellent work!

  8. MarcoFalke commented at 8:43 pm on July 22, 2019: member
    Concept ACK. This might help IBD, because cs_main had to be acquired due to the lock order requirement. If it doesn’t help IBD, the change will hopefully still speed up the msghand thread because the wallets take the main lock less.
  9. jnewbery commented at 9:32 pm on July 22, 2019: member
    Big concept ACK! Thanks @ariard
  10. DrahtBot added the label Needs rebase on Jul 24, 2019
  11. promag commented at 2:58 am on July 25, 2019: member
    Concept ACK, mother of god, not locking cs_main!
  12. in src/wallet/wallet.cpp:2258 in a052c3748b outdated
    2251@@ -2274,13 +2252,13 @@ bool CWalletTx::InMempool() const
    2252     return fInMempool;
    2253 }
    2254 
    2255-bool CWalletTx::IsTrusted(interfaces::Chain::Lock& locked_chain) const
    2256+bool CWalletTx::IsTrusted() const
    2257 {
    2258     // Quick answer in most cases
    2259-    if (!locked_chain.checkFinalTx(*tx)) {
    2260+    if (!pwallet->chain().checkFinalTx(*tx)) {
    


    ryanofsky commented at 6:18 pm on November 5, 2019:

    Instead of adding an interfaces::Chain::checkFinalTx method, it might be better to call IsFinalTx directly with the wallet’s height and time, avoiding going through CheckFinalTx.

    This could perform better since it wouldn’t require locking cs_main, and could make wallet calls return more internally consistent information when the wallet is catching up to the chain.


    ryanofsky commented at 5:12 pm on December 16, 2019:

    re: #16426 (review)

    Instead of adding an interfaces::Chain::checkFinalTx method, it might be better to call IsFinalTx

    #17443 is a step in this direction (https://github.com/bitcoin/bitcoin/pull/17443#pullrequestreview-332692782)

  13. ariard force-pushed on Nov 11, 2019
  14. DrahtBot removed the label Needs rebase on Nov 11, 2019
  15. ariard commented at 7:33 pm on November 11, 2019: member

    Finally rebased after merge of #15931, PR should be ready for review.

    Apart of 3efe38d which use the new m_last_block_processed_height to avoid querying the chainstate and introduce some modifications, other commits are pretty straight-forward. It’s just taking cs_main inside the Chain implementation instead of using Chain::lock. Lock order is reversed only in last commit f057aed to avoid any deadlock issue.

    If you still feel PR is hard to review, I can subsplit easily in another PR.

  16. ryanofsky commented at 8:41 pm on November 11, 2019: member

    Approach ACK. Code changes here are very simple after #15931.

    All the changes before the last commit f057aedf80d8bd6083c2227e42d4887be4c2933b seem straightforward and don’t change behavior other than locking cs_main in new places recursively, so all the new locks are no-ops.

    Only the last commit f057aedf80d8bd6083c2227e42d4887be4c2933b is the big scary change that removes cs_main locks all over the wallet, leaving us to hope that remaining locking is sufficient and that stretches of wallet code that used to run under cs_main aren’t making assumptions about the tip not changing and other wallet threads not running.


    I’ll review this PR more closely this week. It might be nice to simplify the PR description now that #15931 is merged. I think the description just basically needs to say that interfaces::Chain::Lock is a wrapper around around cs_main, and that this PR changes wallet code to not lock and unlock cs_main directly anymore, and not keep cs_main locked while it locks cs_wallet and does other work. Instead, after this PR, wallet code only locks cs_main indirectly and intermittently when it needs to look up bits of chain information, and never holds onto cs_main while it does other work.

  17. jnewbery commented at 9:01 pm on November 11, 2019: member

    Restating my concept ACK. I plan to review this fully soon.

    Thanks for rebasing this so quickly

  18. ariard force-pushed on Nov 11, 2019
  19. ariard commented at 10:03 pm on November 11, 2019: member

    Only the last commit f057aed is the big scary change that removes cs_main locks all over the wallet, leaving us to hope that remaining locking is sufficient and that stretches of wallet code that used to run under cs_main aren’t making assumptions about the tip not changing and other wallet threads not running.

    Most of the code making assumptions about the tip is confined in the rescan logic, so I think this one should get the focus, you can grep for all methods fetching tip like getHeight, getBlockHeight getX and check no decision is made on return value of 2 different calls of them. Concerning other wallet threads, it should be cover by wallet lock in itself.

  20. ariard force-pushed on Nov 12, 2019
  21. ariard force-pushed on Nov 12, 2019
  22. ariard force-pushed on Nov 12, 2019
  23. ariard force-pushed on Nov 12, 2019
  24. ariard force-pushed on Nov 12, 2019
  25. ariard force-pushed on Nov 21, 2019
  26. meshcollider commented at 7:56 pm on November 22, 2019: contributor
    Concept ACK
  27. in src/wallet/rpcdump.cpp:559 in be2651fd89 outdated
    564@@ -565,7 +565,7 @@ UniValue importwallet(const JSONRPCRequest& request)
    565         if (!file.is_open()) {
    566             throw JSONRPCError(RPC_INVALID_PARAMETER, "Cannot open wallet dump file");
    567         }
    568-        Optional<int> tip_height = locked_chain->getHeight();
    569+        Optional<int> tip_height = pwallet->chain().getHeight();
    


    ryanofsky commented at 9:31 pm on December 2, 2019:
    Why use chain height instead of wallet height here and other places? Anywhere the wallet is locked and the chain isn’t locked it would seem safer to use the wallet height.

    ariard commented at 7:13 pm on December 3, 2019:

    I think I can switch for the one in CreateTransaction, all left are tied to the rescan logic which doesn’t call BlockConnected and so doesn’t update m_last_block_processed accurately. I can rework a bit the rescan logic to make it work on m_last_block_processed_height but I felt it was a bit of scope and an increased burden for reviewers.

    As you know I plan to rework the rescan logic after this PR. At term it would remove all getHeight callsites`.


    ryanofsky commented at 7:27 pm on December 3, 2019:
    Thanks, chain height seems fine for rescan logic, and is what I’d expect there. I think the other places should use wallet height if there isn’t an explicit reason not to.

    ariard commented at 7:57 pm on December 3, 2019:
    Corrected on 24f40fc, there was only one occurrence in CreateTransaction. All others are tied to rescan logic, including ones in rpcdump/rpcwallet.
  28. ariard force-pushed on Dec 3, 2019
  29. DrahtBot added the label Needs rebase on Dec 4, 2019
  30. jkczyz commented at 4:07 pm on December 4, 2019: contributor

    When compiling with these changes, I get a new compilation warning related to holding cs_wallet:

    0  CXX      bitcoin_wallet-bitcoin-wallet.o
    1wallet/wallet.cpp:2528:61: warning: reading variable 'm_last_block_processed_height' requires holding mutex
    2      'cs_wallet' [-Wthread-safety-analysis]
    3    txNew.nLockTime = GetLocktimeForNewTransaction(chain(), m_last_block_processed_height);
    4                                                            ^
    
  31. jonatack commented at 4:35 pm on December 4, 2019: member
    @jkczyz, what platform, compiler, and configure flags did you use so I can try to reproduce? My build (Debian, gcc 8.3, –enable-debug, –enable-bench) didn’t show that warning. Perhaps I should use clang -c -Wthread-safety.
  32. jonatack commented at 4:37 pm on December 4, 2019: member

    Approach ACK, light tested ACK as I have only made a cursory code review. It looks like you got all the call sites. Can LockAssertion in sync.h be removed? It appears to be unused after these changes.

    Built with no warnings, ran tests and bitcoind with –enable-debug for the -DDEBUG_LOCKORDER compiler flag. No debug log warnings so far apart from the continual n of last 100 blocks have unexpected version ones. Would like to do the same rebased to current master but the rebase isn’t clean; will wait for rebase.

    02019-12-04T15:41:23Z Bitcoin Core version v0.19.99.0-24f40fc1af (debug build)
    
  33. jkczyz commented at 4:55 pm on December 4, 2019: contributor

    @jkczyz, what platform, compiler, and configure flags did you use so I can try to reproduce? My build (Debian, gcc 8.3, –enable-debug, –enable-bench) didn’t show that warning. Perhaps I should use clang -c -Wthread-safety. @jonatack Built on MacOS using clang-1001.0.46.4. No special configuration flags. I performed make clean && make at a7aec7ad9 followed by make at 24f40fc1a to confirm the warning only appeared at the latter commit. I haven’t tried building on my Linux box yet.

  34. jkczyz commented at 5:09 pm on December 4, 2019: contributor

    @jkczyz, what platform, compiler, and configure flags did you use so I can try to reproduce? My build (Debian, gcc 8.3, –enable-debug, –enable-bench) didn’t show that warning. Perhaps I should use clang -c -Wthread-safety.

    @jonatack Built on MacOS using clang-1001.0.46.4. No special configuration flags. I performed make clean && make at a7aec7a followed by make at 24f40fc to confirm the warning only appeared at the latter commit. I haven’t tried building on my Linux box yet.

    Confirmed no warning on Ubuntu:

    0$ gcc --version
    1gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
    
  35. ariard force-pushed on Dec 4, 2019
  36. ariard commented at 5:17 pm on December 4, 2019: member

    Thanks for reviews, @jkczyz it should be good now on df508be, following Russ remark I switchted GetLocktimeForNewTransaction to use m_last_block_processed_height but without remembering the reason I avoid to do it at first, which was to avoid tweaking lock order issue. I just changed GetLocktimeForNewTransaction after lock taking in CreateTransaction, it shouldn’t change anything.

    Can LockAssertion in sync.h be removed? @jonatack I would prefer to let it like this. We may need again this thread assertion helper in the future. IIRC clang thread safety detection only works on Mac for now

  37. jonatack commented at 5:17 pm on December 4, 2019: member

    @jonatack Built on MacOS using clang-1001.0.46.4. No special configuration flags. I performed make clean && make at a7aec7a followed by make at 24f40fc to confirm the warning only appeared at the latter commit. I haven’t tried building on my Linux box yet. @jkczyz Thanks! Found this warning with Debian/clang version 7.0.1-8:

    0  CXX      libbitcoin_common_a-merkleblock.o
    1wallet/wallet.cpp:2528:61: warning: reading variable 'm_last_block_processed_height' requires holding mutex 'cs_wallet' [-Wthread-safety-analysis]
    2    txNew.nLockTime = GetLocktimeForNewTransaction(chain(), m_last_block_processed_height);
    3                                                            ^
    41 warning generated.
    
    0  CC            = /usr/bin/ccache clang
    1  CFLAGS        = -g -O2
    2  CPPFLAGS      =  -DDEBUG -DDEBUG_LOCKORDER  -Qunused-arguments
    3                   -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS
    4  CXX           = /usr/bin/ccache clang++ -std=c++11
    5  CXXFLAGS      =  -O0 -g3 -ftrapv  -Wstack-protector -fstack-protector-all  -Wall -Wextra
    6                   -Wformat -Wvla -Wswitch -Wformat-security -Wthread-safety-analysis
    7                   -Wrange-loop-analysis -Wredundant-decls  -Wno-unused-parameter
    8                   -Wno-self-assign -Wno-unused-local-typedef -Wno-deprecated-register
    9                   -Wno-implicit-fallthrough
    
  38. jonatack commented at 5:37 pm on December 4, 2019: member

    it should be good now on df508be

    IIRC clang thread safety detection only works on Mac for now

    Yes, rebuilt and the warning is now gone in df508be. As commented just above, I reproduced the warning on 24f40fc with Debian and Clang (passing –enable-debug).

    0~/projects/bitcoin/bitcoin/src ((HEAD detached at origin/pr/16426))$ bitcoind
    12019-12-04T17:41:08Z Bitcoin Core version v0.19.99.0-df508be136 (debug build)
    
  39. practicalswift commented at 5:37 pm on December 4, 2019: contributor
    Would be interesting to see the impact (if any) on IBD :)
  40. DrahtBot removed the label Needs rebase on Dec 4, 2019
  41. ariard commented at 7:54 pm on December 4, 2019: member
    @jonatack that’s a compiler warning and --enable-debug is about turning on our deadlock detection with conditional compilation they are not related.
  42. jkczyz commented at 9:15 pm on December 4, 2019: contributor
    I’m getting the warning at df508be on MacOS using clang as before.
  43. ariard force-pushed on Dec 4, 2019
  44. ariard commented at 9:34 pm on December 4, 2019: member
    @jkczyz at 3773b41, add a AssertLockHeld in CreateTransaction after taking cs_wallet, it should avoid clang false positive warning according to documentation “utility class for indicating to compiler thread analysis that a mutex is locked (when it couldn’t be determined otherwise)”
  45. ariard force-pushed on Dec 4, 2019
  46. jkczyz commented at 10:10 pm on December 4, 2019: contributor

    @jkczyz at 3773b41, add a AssertLockHeld in CreateTransaction after taking cs_wallet, it should avoid clang false positive warning according to documentation “utility class for indicating to compiler thread analysis that a mutex is locked (when it couldn’t be determined otherwise)”

    Looks like I had a problem on my end. I don’t get the warning with df508be after all. Sorry about the confusion!

  47. jonatack commented at 10:40 pm on December 4, 2019: member

    @jonatack that’s a compiler warning and --enable-debug is about turning on our deadlock detection with conditional compilation they are not related.

    Thanks. Agreed. Verified locally: warning present on 24f40fc with Debian+Clang without passing –enable-debug.

    0  CC            = /usr/bin/ccache clang
    1  CFLAGS        = -g -O2
    2  CPPFLAGS      =   -U_FORTIFY_SOURCE  -D_FORTIFY_SOURCE=2  -Qunused-arguments
    3                    -DHAVE_BUILD_INFO  -D__STDC_FORMAT_MACROS
    4  CXX           = /usr/bin/ccache clang++ -std=c++11
    5  CXXFLAGS      =   -Wstack-protector -fstack-protector-all  -Wall -Wextra -Wformat
    6                    -Wvla -Wswitch -Wformat-security -Wthread-safety-analysis
    7                    -Wrange-loop-analysis -Wredundant-decls  -Wno-unused-parameter
    8                    -Wno-self-assign -Wno-unused-local-typedef -Wno-deprecated-register
    9                    -Wno-implicit-fallthrough   -g -O2
    
  48. jonatack commented at 10:43 pm on December 4, 2019: member
    @ariard what changed from df508be to df17e36 to 3773b41?
  49. ariard commented at 11:36 pm on December 4, 2019: member
    @jonatack Adding the AssertLockHeld to be sure to not raise false positive warning from the compiler.
  50. laanwj added this to the "Blockers" column in a project

  51. ariard commented at 7:27 pm on December 5, 2019: member
    @jonatack src/wallet/wallet.cpp L2553 on head commit. But you should really recompile latest git to get git range-diff it changes your life!
  52. instagibbs commented at 7:33 pm on December 5, 2019: member
    could you update the OP? It’s slightly outdated due to dependent PRs being merged
  53. ariard commented at 4:38 pm on December 6, 2019: member
    OP updated, see also for more context https://bitcoincore.reviews/16426.html
  54. in src/wallet/wallet.cpp:3406 in b844282ba6 outdated
    3411-                    // ... and all their affected keys
    3412-                    std::map<CKeyID, int>::iterator rit = mapKeyFirstBlock.find(keyid);
    3413-                    if (rit != mapKeyFirstBlock.end() && *height < rit->second)
    3414-                        rit->second = *height;
    3415-                }
    3416+        // ... which are already in a block
    


    promag commented at 11:29 pm on December 8, 2019:

    b844282ba6539954e6ad9ba899497e28fed60d91

    IMO drop this comment, or adjust and move to L3412.

  55. in src/wallet/wallet.h:762 in b844282ba6 outdated
    757@@ -758,6 +758,9 @@ class CWallet final : public WalletStorage, private interfaces::Chain::Notificat
    758     /** Interface to assert chain access and if successful lock it */
    759     std::unique_ptr<interfaces::Chain::Lock> LockChain() { return m_chain ? m_chain->lock() : nullptr; }
    760 
    761+    /** Interface to assert chain access */
    762+    bool HaveChain() { return m_chain ? true : false; }
    


    promag commented at 11:30 pm on December 8, 2019:

    b844282ba6539954e6ad9ba899497e28fed60d91

    nit, const?

  56. in src/wallet/wallet.cpp:3386 in b844282ba6 outdated
    3417+        for (const CTxOut& txout : wtx.tx->vout) {
    3418+            // iterate over all their outputs
    3419+            for (const auto& keyid : GetAffectedKeys(txout.scriptPubKey, *spk_man)) {
    3420+                // ... and all their affected keys
    3421+                std::map<CKeyID, int>::iterator rit = mapKeyFirstBlock.find(keyid);
    3422+                if (rit != mapKeyFirstBlock.end() && wtx.m_confirm.block_height < rit->second)
    


    promag commented at 11:31 pm on December 8, 2019:

    b844282ba6539954e6ad9ba899497e28fed60d91

    nit, since this line is no longer different add braces?


    ariard commented at 11:38 pm on December 10, 2019:
    Is this a style requirement? IMO don’t find code easier to read doing so.

    kallewoof commented at 2:14 am on January 5, 2020:

    Yes, it is a style requirement. Same line or add {}.

    From https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#coding-style-c

    If an if only has a single-statement then-clause, it can appear on the same line as the if, without braces. In every other case, braces are required, and the then and else clauses must appear correctly indented on a new line.

  57. in src/wallet/wallet.cpp:2454 in e64f0909c6 outdated
    2451     if (chain.isInitialBlockDownload()) {
    2452         return false;
    2453     }
    2454     constexpr int64_t MAX_ANTI_FEE_SNIPING_TIP_AGE = 8 * 60 * 60; // in seconds
    2455-    if (locked_chain.getBlockTime(*locked_chain.getHeight()) < (GetTime() - MAX_ANTI_FEE_SNIPING_TIP_AGE)) {
    2456+    auto locked_chain = chain.lock();
    


    promag commented at 11:36 pm on December 8, 2019:

    e64f0909c66e39896c1e41df77576cccc3001088

    nit, I’m aware this goes away later in 6b4d68ec25f024f009efc79b99816fa9ff6d89de but maybe the interfaces::Chain::Lock& locked_chain arg should be dropped later instead - so this commit would keep locked_chain and just adde block_height.


    ariard commented at 11:42 pm on December 10, 2019:
    Going to let it like this, one of both commits have to clean it anyway unless adding a temporary one line is really gross..
  58. in src/interfaces/chain.cpp:334 in 3773b4159c outdated
    330@@ -344,6 +331,7 @@ class ChainImpl : public Chain
    331     }
    332     std::unique_ptr<Handler> handleNotifications(Notifications& notifications) override
    333     {
    334+        LOCK(::cs_main);
    


    promag commented at 0:17 am on December 9, 2019:

    3773b4159c37db993b9bb7c3f5d7dd7a51a39b09

    Is this going to change in a follow up? Maybe add a comment explaining this lock?


    ariard commented at 11:44 pm on December 10, 2019:
    Adding a comment only for this method would seem obscure compare to other methods for future reviewers after refactoring, that’s why it’s in commit message “Lock cs_main in handleNotifications as it was relaying on its caller to do so before”, or do you have a different opinion ?

    ryanofsky commented at 8:56 pm on December 13, 2019:

    re: #16426 (review)

    Adding a comment only for this method would seem obscure compare to other methods for future reviewers after refactoring

    I think this lock is a little different than the locks in the other Chain methods because it is not obvious why locking is needed for something that appears to just be a memory allocation. If this lock really is necessary, having a comment in the code would be helpful to me at least. Or if it is possible to drop the lock or move it closer to where it might be needed in the NotificationsHandlerImpl constructor that would be even better.


    ariard commented at 1:00 am on December 19, 2019:

    Okay after thinking about this, by removing lock, we may miss block connections between end of rescan and registering with validation interface. That was the meaning of the comment I think, which is about consistency of chain viewed by the wallet. To avoid that I moved handleNotifications before rescan and after cs_wallet lock taking. That’s way block connections notifications are going to be pending until rescan is done. We may have block processing duplicata but at least we should guarantee wallet consistency.

    Long-term solution is to unify both rescan and Chain::Notifications under same interface.

  59. ariard commented at 11:46 pm on December 10, 2019: member
    Thanks for review @promag, corrected some nits on f87558b.
  60. ariard force-pushed on Dec 10, 2019
  61. ariard commented at 7:25 pm on December 12, 2019: member
    @practicalswift after a run on bitcoinperf, there is no seen performance change on IBD. It still may be an improvement with a heavy workload on wallet RPCs (but harder to simulate).
  62. in src/interfaces/chain.h:93 in f87558b49b outdated
    127+    //! Get block height above genesis block. Returns 0 for genesis block,
    128+    //! 1 for following block, and so on. Returns nullopt for a block not
    129+    //! included in the current chain.
    130+    virtual Optional<int> getBlockHeight(const uint256& hash) = 0;
    131+
    132+    //! Get block hash. Height must be valid or this function will abort.
    


    ryanofsky commented at 5:52 pm on December 13, 2019:

    For getBlockHash, getBlockTime, and getBlockMedianTimePast, I don’t think it safe anymore for these methods to abort when an invalid height is passed. Previously it was possible to guarantee that an invalid height would never be passed because cs_main would be held before calling these (and if an invalid height was passed it indicated a bug). But now that cs_main isn’t held, there could be a race condition where invalidateblock is called, or there is a reorg to a shorter chain that would cause aborts now.

    A possible fix would be to implement error handling for race conditions in the places where these methods are called, maybe changing these methods to throw an exception or return Optional<> values or success bools to indicate failure instead of aborting.

    Another solution would be to drop these methods, and replace them with a more general method similar to findBlock that is capable of returning multiple pieces of information atomically so there aren’t new race conditions the wallet has to handle now because it isn’t holding cs_main.


    ariard commented at 1:05 am on December 19, 2019:

    You’re right, even if you need bad-timed reorgs to trigger them. After thinking about this, I picked the option to return Optional<> because that’s one giving us more control. Throwing exception don’t fit everywhere where an absent block isn’t fatal. Return boolean would mean to pass pointer to values which is a bit different than other interface methods (like getHeight).

    On dropping these methods, I think getBlockTime would be the easiest one if we can store block time in confirmation struct of transactions but we’re going to be limited by not being able to serialize them. getBlockMedianTimePast and getBlockHash can go by refactoring rescan logic. Most of the further dry-ing up refactoring is going to be stuck between serialization/rescan..


    ryanofsky commented at 7:09 pm on December 26, 2019:

    “this function will abort” comments are out of date now and could be updated

    Just to clarify my earlier suggestion about dropping these methods, I wasn’t trying to suggest caching more block information in the wallet (though this could be nice). I was just trying to suggest looking up block information by hash instead of height, because heights aren’t a stable way of identifying blocks if cs_main isn’t locked, but hashes are.


    ariard commented at 0:17 am on January 4, 2020:

    I’ve been through function triplet callsites again and I think using hash instead of height wouldn’t make code safer that much.

    getBlockHash is always called by rescan code which making the assumption of chain lock not being held.

    getBlockTime/getBlockMedianTimePast, given than block time must always be higher than parent MTP and not to be more than 2h in the future by consensus rules and than wallet code is using time range to take decision it doesn’t matter than height match with block hash A or block hash B.

  63. in src/interfaces/wallet.cpp:66 in f87558b49b outdated
    59@@ -60,16 +60,16 @@ WalletTx MakeWalletTx(CWallet& wallet, const CWalletTx& wtx)
    60 }
    61 
    62 //! Construct wallet tx status struct.
    63-WalletTxStatus MakeWalletTxStatus(interfaces::Chain::Lock& locked_chain, const CWalletTx& wtx)
    64+WalletTxStatus MakeWalletTxStatus(CWallet& wallet, const CWalletTx& wtx)
    65 {
    66     WalletTxStatus result;
    67-    result.block_height = locked_chain.getBlockHeight(wtx.m_confirm.hashBlock).get_value_or(std::numeric_limits<int>::max());
    68+    result.block_height = wtx.m_confirm.block_height;
    


    ryanofsky commented at 5:58 pm on December 13, 2019:
    Is behavior changing here? Does this need need to be changed to continue returning std::numeric_limits<int>::max() in the case of an abandoned or unconfirmed transaction, or does it not matter?

    ariard commented at 7:53 pm on December 18, 2019:
    I think it doesn’t matter, and maybe even confusing to display std::numeric_limit<int>::max() as block height. An abandoned one will have is_abandoned sets to true. An unconfirmed one will have depth_in_main_chain sets to 0. It seems to me meaningful enough for end user but I can still do a ternary and if hashBlock is null set it to fallback value if you think so.

    ryanofsky commented at 8:09 pm on December 18, 2019:

    I think it doesn’t matter, and maybe even confusing to display std::numeric_limit<int>::max() as block height. An abandoned one will have is_abandoned sets to true. An unconfirmed one will have depth_in_main_chain sets to 0. It seems to me meaningful enough for end user but I can still do a ternary and if hashBlock is null set it to fallback value if you think so.

    I think anything that doesn’t break the GUI is fine. If GUI treats 0x7FFFFFFFF and 0xFFFFFFFFF the same, the current code is fine. If GUI treats them differently, that should just be addressed somehow. I’m just asking because I’m not sure if this is a real change or not.


    ryanofsky commented at 7:22 pm on December 26, 2019:

    Haven’t tested, but it looks like this change might cause unconfirmed transaction to sort to the bottom instead of the top:

    https://github.com/bitcoin/bitcoin/blob/1dbf3350c683f93d7fc9b861400724f6fd2b2f1d/src/qt/transactionrecord.cpp#L169-L174

    Could fix by updating qt code to handle block height -1, or preserving max int behavior here.


    ariard commented at 0:12 am on January 4, 2020:

    Back to result.block_height = wtx.m_confirm.block_height > 0 ? wtx.m_confirm.block_height std::numeric_limits<int>::max();

    Should avoid to break qt code, without further digging into it.

  64. in src/interfaces/wallet.cpp:321 in f87558b49b outdated
    315@@ -330,14 +316,13 @@ class WalletImpl : public Wallet
    316         if (mi == m_wallet->mapWallet.end()) {
    317             return false;
    318         }
    319-        if (Optional<int> height = locked_chain->getHeight()) {
    320-            num_blocks = *height;
    321-            block_time = locked_chain->getBlockTime(*height);
    322+        num_blocks = m_wallet->GetLastBlockHeight();
    323+        if (num_blocks >= 0) {
    324+            block_time = m_wallet->chain().getBlockTime(num_blocks);
    


    ryanofsky commented at 9:18 pm on December 13, 2019:

    I think to avoid the race condition mentioned #16426 (review), where there is a block reorg between the CWallet::GetLastBlockHeight and Chain::getBlockTime calls, it would be safer to avoid triggering asserts by getting the time by hash rather than height. You could tweak GetLastBlock to optionally return the block hash:

    0int GetLastBlock(uint256* block_hash = nullptr);
    

    for

    0uint256 block_hash;
    1num_blocks = m_wallet->GetLastBlock(&block_hash);
    2if (num_blocks < 0 || !m_wallet.chain().findBlock(block_hash, nullptr, &block_time)) {
    3    block_time = -1;
    4}
    

    There are probably other places in this PR where similar changes would be appropriate.


    ariard commented at 1:10 am on December 19, 2019:
    Changed it in cbd2c1a, covered with other reorg mitigations.
  65. DrahtBot added the label Needs rebase on Dec 16, 2019
  66. in src/wallet/rpcdump.cpp:359 in f87558b49b outdated
    354@@ -358,8 +355,7 @@ UniValue importprunedfunds(const JSONRPCRequest& request)
    355         throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Something wrong with merkleblock");
    356     }
    357 
    358-    auto locked_chain = pwallet->chain().lock();
    359-    Optional<int> height = locked_chain->getBlockHeight(merkleBlock.header.GetHash());
    360+    Optional<int> height = pwallet->chain().getBlockHeight(merkleBlock.header.GetHash());
    


    ryanofsky commented at 7:40 pm on December 17, 2019:

    I think the LOCK(pwallet->cs_wallet); line below should move above this getBlockHeight() call. Otherwise there could be a race condition if there is a reorg and the transaction is no longer confirmed by the time AddToWallet is called below, but the wallet considers it confirmed anyway.

    I’m not sure if it would be a good idea, but perhaps the CWallet::chain() method should be annotated EXCLUSIVE_LOCKS_REQUIRED(cs_wallet) to catch other cases like this.


    ariard commented at 1:08 am on December 19, 2019:
    Thanks, I’ve modified it, but didn’t add EXCLUSIVE_LOCKS_REQUIRED to the CWallet::chain() as it would change lock assumptions for RPC calls like bumpfee or fundtransaction..
  67. ryanofsky commented at 8:06 pm on December 17, 2019: member

    I was reading the review club discussion, and it sounded like there was some confusion about what motivated the PR. I think it would help to simplify the PR description. If I were describing this PR, I would write something like:

    re: #16426#issue-299541584

    Chain::Lock reason was mostly a temporary interface to help separation of Wallet from Node (#10973) without modifying behavior of legacy wallet code relying on chain state. It was a relief of review burden in the aforementioned separation PR, at the cost of keeping synchronicity between Wallet and Node execution.

    Caching more state in CWallet (#15931), let us remove the more used calls like getBlockDepth and move toward a more asynchronous interface. The current PR 1) move all remaining Chain::Lock methods to simple one and ensure cs_main lock is taken inside the interface 2) remove locking of cs_main inside any wallet code, which is another asynchronicity gain. Lock order has to be swapped atomically at once to avoid potential deadlock issues raised by thread sync verification stuff.

    Especially now that #15931 is merged, this mostly feels like historical information that doesn’t help make the PR description more understandable. I would probably drop these paragraphs from the PR description or maybe move them to footnotes.

    Some future works which would enhance Chain interface:

    • move ScanForWalletTransactions and most we can of rescanning logic on the node side, would let us remove directly methods like guessVerificationProgress or getBlockHeight, meanwhile relieving of the the hardship of multiple simultaneous rescanning processing
    • split listsinceblock into a wallet part and node one, #15931 caching block height in tx, I think that’s possible
    • move fee estimation queries to a notification interface and let the wallet cache it, fee estimation being driven by node mempool, wallet should be the consumer
    • obviously remove the RPC specific methods by letting the wallet processing RPC requests without going through the node, need to add --server and --rpcserver options

    I’m also not sure so much detail about future todos belongs in the PR description here. I’ve been trying to keep a list of cleanup todos at the top of src/interfaces/chain.h, so you could definitely merge this list into that list, and keep discussion of followup changes in the PR description more brief and basic.

    To review the present PR, most of getting right the move is ensuring any LockAssertion in Chain::Lock method is amended as a LOCK(cs_main). And in final commit, check that any wallet code which was previously locking the chain is now calling a method, enforcing the lock taking job. So far the only exception I found is handleNotifications, which should be corrected.

    This paragraph is helpful and seems useful to keep, except I’m not clear about the handleNotifications comment. Would that be addressed by following up to #16426 (review)?

  68. in src/interfaces/wallet.cpp:334 in e64f0909c6 outdated
    329@@ -330,11 +330,10 @@ class WalletImpl : public Wallet
    330         if (mi == m_wallet->mapWallet.end()) {
    331             return false;
    332         }
    333-        if (Optional<int> height = locked_chain->getHeight()) {
    334-            num_blocks = *height;
    335-            block_time = locked_chain->getBlockTime(*height);
    336+        num_blocks = m_wallet->GetLastBlockHeight();
    337+        if (num_blocks >= 0) {
    


    fjahr commented at 11:19 am on December 18, 2019:
    nit: This is already asserted in GetLastBlockHeight, so this check could be removed
  69. in src/wallet/rpcwallet.cpp:1605 in e64f0909c6 outdated
    1600@@ -1601,8 +1601,8 @@ static UniValue listsinceblock(const JSONRPCRequest& request)
    1601 
    1602     bool include_removed = (request.params[3].isNull() || request.params[3].get_bool());
    1603 
    1604-    const Optional<int> tip_height = locked_chain->getHeight();
    1605-    int depth = tip_height && height ? (1 + *tip_height - *height) : -1;
    1606+    int tip_height = pwallet->GetLastBlockHeight();
    1607+    int depth = tip_height >= 0 && height ? (1 + tip_height - *height) : -1;
    


    fjahr commented at 11:25 am on December 18, 2019:
    nit: check not needed; see above
  70. in src/wallet/rpcwallet.cpp:1637 in e64f0909c6 outdated
    1633@@ -1634,7 +1634,7 @@ static UniValue listsinceblock(const JSONRPCRequest& request)
    1634         --*altheight;
    1635     }
    1636 
    1637-    int last_height = tip_height ? *tip_height + 1 - target_confirms : -1;
    1638+    int last_height = tip_height >= 0 ? tip_height + 1 - target_confirms : -1;
    


    fjahr commented at 11:25 am on December 18, 2019:
    nit: check not needed; see above
  71. in src/wallet/rpcdump.cpp:785 in f87558b49b outdated
    779@@ -790,9 +780,9 @@ UniValue dumpwallet(const JSONRPCRequest& request)
    780     // produce output
    781     file << strprintf("# Wallet dump created by Bitcoin %s\n", CLIENT_BUILD);
    782     file << strprintf("# * Created on %s\n", FormatISO8601DateTime(GetTime()));
    783-    const Optional<int> tip_height = locked_chain->getHeight();
    784-    file << strprintf("# * Best block at time of backup was %i (%s),\n", tip_height.get_value_or(-1), tip_height ? locked_chain->getBlockHash(*tip_height).ToString() : "(missing block hash)");
    785-    file << strprintf("#   mined on %s\n", tip_height ? FormatISO8601DateTime(locked_chain->getBlockTime(*tip_height)) : "(missing block time)");
    786+    int tip_height = pwallet->GetLastBlockHeight();
    787+    file << strprintf("# * Best block at time of backup was %i (%s),\n", tip_height, tip_height >= 0 ? pwallet->chain().getBlockHash(tip_height).ToString() : "(missing block hash)");
    788+    file << strprintf("#   mined on %s\n", tip_height >= 0 ? FormatISO8601DateTime(pwallet->chain().getBlockTime(tip_height)) : "(missing block hash)");
    


    fjahr commented at 12:04 pm on December 18, 2019:
    nit: tip_height checks not needed here, see above
  72. fjahr commented at 12:55 pm on December 18, 2019: member

    ACK f87558b49bc63922a70c9ff02c33c0b1ed1f44b1

    Reviewed code, built and ran tests locally, ran it as my testnet node and manually tested some wallet RPCs.

    One minor nit (occuring multiple times), feel free to ignore.

  73. ryanofsky commented at 6:05 pm on December 18, 2019: member

    A suggestion for other reviewers might be to review last commit first: “Remove locked_chain from CWallet, its RPCs and tests” (f87558b49bc63922a70c9ff02c33c0b1ed1f44b1), since this is the commit that actually changes behavior, while other commits are moving and refactoring code.

    Note to Antoine: If it would make it easier to update the PR, I think it’d be fine to squash it into just one or two commits. Personally I’m mostly reviewing this PR by looking at the overall diff, because it’s less work to focus on the end result when reviewing parts of the code that are changing more than once.

  74. ariard force-pushed on Dec 19, 2019
  75. ariard commented at 1:13 am on December 19, 2019: member

    Thanks one thousands times @ryanofsky for review and advices. Implemented most of them, including updating OP and tracking TODO in interfaces/chain.h. Also squash a bunch of commits, only non-squashed are the ones doing refactor.

    I’m still looking for low-hanging fruits to dry-up the interface like dropping checkFinalTx to ease review.

  76. DrahtBot removed the label Needs rebase on Dec 19, 2019
  77. in src/interfaces/chain.cpp:170 in 88b41a7c88 outdated
    154+    }
    155+    Optional<int64_t> getBlockTime(int height) override
    156+    {
    157+        LOCK(::cs_main);
    158+        CBlockIndex* block = ::ChainActive()[height];
    159+        assert(block != nullptr);
    


    ryanofsky commented at 7:16 pm on December 26, 2019:
    It looks like getBlockTime and getBlockMedianTimePast are still aborting instead of returning nullopt if the block isn’t found. Unintended change?

    ariard commented at 0:09 am on January 4, 2020:
    Ooops, my bad, thanks should be corrected on latest tip.

    ryanofsky commented at 9:10 pm on April 21, 2020:

    In commit “[wallet] Move getBlockHash from Chain::Lock interface to simple Chain” (c6fa030ca8a282a7906260186e93d8c8902aa5a4)

    re: #16426 (review)

    Ooops, my bad, thanks should be corrected on latest tip.

    Assert is still present (maybe added back again)


    ariard commented at 0:36 am on April 23, 2020:
    Corrected, effectively that’s a rebase error. It’s still present on master but removing locked_chain means you may have a reorg while calling getBlockHash in wallet creation rescan.
  78. ariard force-pushed on Jan 4, 2020
  79. ariard commented at 0:20 am on January 4, 2020: member
    Updated e4da874. Picked up Russ’s last review suggestions/findings, thanks.
  80. in src/wallet/rpcdump.cpp:795 in 1533b075aa outdated
    789@@ -790,9 +790,9 @@ UniValue dumpwallet(const JSONRPCRequest& request)
    790     // produce output
    791     file << strprintf("# Wallet dump created by Bitcoin %s\n", CLIENT_BUILD);
    792     file << strprintf("# * Created on %s\n", FormatISO8601DateTime(GetTime()));
    793-    const Optional<int> tip_height = locked_chain->getHeight();
    794-    file << strprintf("# * Best block at time of backup was %i (%s),\n", tip_height.get_value_or(-1), tip_height ? locked_chain->getBlockHash(*tip_height).ToString() : "(missing block hash)");
    795-    file << strprintf("#   mined on %s\n", tip_height ? FormatISO8601DateTime(locked_chain->getBlockTime(*tip_height)) : "(missing block time)");
    796+    int tip_height = pwallet->GetLastBlockHeight();
    797+    file << strprintf("# * Best block at time of backup was %i (%s),\n", tip_height, locked_chain->getBlockHash(tip_height).ToString());
    798+    file << strprintf("#   mined on %s\n", tip_height >= 0 ? FormatISO8601DateTime(locked_chain->getBlockTime(tip_height)) : "(missing block hash)");
    


    kallewoof commented at 1:51 am on January 5, 2020:
    If tip_height < 0 ever occurs, locked_chain->getBlockHash(tip_height) above will crash. pwallet->GetLastBlockHeight() will also never return a value < 0, so not sure why the tip_height >= 0 ? part is necessary.
  81. in src/wallet/rpcdump.cpp:1369 in 1533b075aa outdated
    1364@@ -1365,7 +1365,7 @@ UniValue importmulti(const JSONRPCRequest& mainRequest)
    1365         EnsureWalletIsUnlocked(pwallet);
    1366 
    1367         // Verify all timestamps are present before importing any keys.
    1368-        const Optional<int> tip_height = locked_chain->getHeight();
    1369+        const Optional<int> tip_height = pwallet->chain().getHeight();
    1370         now = tip_height ? locked_chain->getBlockMedianTimePast(*tip_height) : 0;
    


    kallewoof commented at 1:53 am on January 5, 2020:
    Behavior change as tip_height can now be 0 for a chain with only the genesis block, which before would give truth here but now will give false. I would just remove the ternary part as it makes no sense anymore (I think?).
  82. in src/wallet/rpcwallet.cpp:1637 in 1533b075aa outdated
    1633@@ -1634,8 +1634,7 @@ static UniValue listsinceblock(const JSONRPCRequest& request)
    1634         --*altheight;
    1635     }
    1636 
    1637-    int last_height = tip_height ? *tip_height + 1 - target_confirms : -1;
    1638-    uint256 lastblock = last_height >= 0 ? locked_chain->getBlockHash(last_height) : uint256();
    1639+    uint256 lastblock = tip_height + 1 - target_confirms >= 0 ? locked_chain->getBlockHash(tip_height + 1 - target_confirms) : uint256();
    


    kallewoof commented at 1:59 am on January 5, 2020:

    Slightly prefer the two line version here:

    0int lastheight = tip_height + 1 - target_confirms;
    1uint256 lastblock = lastheight >= 0 ? locked_chain->getBlockHash(lastheight) : uint256();
    
  83. kallewoof commented at 2:15 am on January 5, 2020: member
    Code review up to and including eff956c1d63975d386d68d0fe764b372a2ed81ea (Move getBlockHash from Chain::Lock interface to simple Chain).
  84. in src/wallet/rpcwallet.cpp:2917 in e4da874564 outdated
    2887@@ -2914,9 +2888,8 @@ static UniValue listunspent(const JSONRPCRequest& request)
    2888         cctl.m_avoid_address_reuse = false;
    2889         cctl.m_min_depth = nMinDepth;
    2890         cctl.m_max_depth = nMaxDepth;
    2891-        auto locked_chain = pwallet->chain().lock();
    2892         LOCK(pwallet->cs_wallet);
    


    promag commented at 11:43 pm on January 5, 2020:
    Move this up and remove the lock in L2922.

    ariard commented at 5:30 am on January 8, 2020:
    You mean the main wallet lock just after ? I think some other wallet locks can also be removed in other places and that would be better to do it in a follow-up PR with dedicated thinking.
  85. ariard force-pushed on Jan 8, 2020
  86. ariard commented at 5:28 am on January 8, 2020: member
    Thanks @kallewoof for review, took most of the suggestions on ac05c8b, some of them were already fixed in latter commits anyway, I can squash commits more if it suits more (not my feeling but if it fits reviewers better let’s go for it)
  87. DrahtBot added the label Needs rebase on Jan 8, 2020
  88. in src/interfaces/chain.h:75 in ac05c8b811 outdated
    45@@ -51,79 +46,74 @@ class Wallet;
    46 //! * The handleRpc, registerRpcs, rpcEnableDeprecated methods and other RPC
    47 //!   methods can go away if wallets listen for HTTP requests on their own
    48 //!   ports instead of registering to handle requests on the node HTTP port.
    49+//!
    50+//! * Move fee estimation queries to an asynchronous interface and let the
    51+//!   wallet cache it, fee estimation being driven by node mempool, wallet
    52+//!   should be the consumer.
    53+//!
    54+//! * The `guesVerificationProgress`, `getBlockHeight`, `getBlockHash`, etc
    


    jnewbery commented at 11:09 pm on January 14, 2020:
    s/guesVerificationProgress/guessVerificationProgress

    kallewoof commented at 5:08 am on April 28, 2020:
    This typo is still present in current code.
  89. in src/wallet/wallet.cpp:2540 in ac05c8b811 outdated
    2402@@ -2412,11 +2403,10 @@ bool CWallet::FundTransaction(CMutableTransaction& tx, CAmount& nFeeRet, int& nC
    2403 
    2404     // Acquire the locks to prevent races to the new locked unspents between the
    


    jnewbery commented at 11:11 pm on January 14, 2020:
    S/Acquire the locks/Acquire the wallet lock
  90. in src/wallet/wallet.cpp:3906 in ac05c8b811 outdated
    3775@@ -3800,24 +3776,34 @@ std::shared_ptr<CWallet> CWallet::CreateWalletFromFile(interfaces::Chain& chain,
    3776     // Try to top up keypool. No-op if the wallet is locked.
    3777     walletInstance->TopUpKeyPool();
    3778 
    3779-    auto locked_chain = chain.lock();
    3780     LOCK(walletInstance->cs_wallet);
    3781 
    3782+    // Register wallet with validationinterface. It's done before rescan to avoid
    


    jnewbery commented at 11:24 pm on January 14, 2020:

    This comment is a bit difficult to parse. My suggested wording:

    Register wallet with validationinterface. We do this before rescan so there isn’t a window between the end of the rescan and subscribing to the validation interface where the wallet could miss block connections.

    Because the wallet lock is held, block connection notifications will be queued in the validation interface until the lock is released. It’s possible that some of the blocks in the rescan will also be recieved over the validation interface, but we guarantee that wallet state is correct after the block connected notifications.

    I suggest you drop the “This is temporary until rescan and notifications delivery are unified under same interface.” part. It’s better to avoid adding code comments which say what you want to do in future work.


    ryanofsky commented at 10:29 pm on January 15, 2020:

    It’s better to avoid adding code comments which say what you want to do in future work.

    I kind of have the opposite reaction, but maybe a compromise would be to phrase it more like “here is a limitation of the current code, and how it could be overcome” instead of “this is temporary and will be fixed later”

  91. jnewbery commented at 3:59 pm on January 15, 2020: member
    I’ve started skimming through the changes here. I have a few trivial comments so far.
  92. ryanofsky commented at 11:04 pm on January 15, 2020: member

    Thanks for the updates Antoine, and sorry I’ve been taking so long to review this. I’ve mostly just been stuck on the point about block heights not being a reliable way to refer to blocks when the chain isn’t locked. But I did some work this week to try to figure things out.

    re: #16426 (review)

    I’ve been through function triplet callsites again and I think using hash instead of height wouldn’t make code safer that much.

    I think I agree that safety isn’t a big issue most of these cases because failures should be obvious and just retrying anything that fails should fix things. But I guess I look at this type of safety as more of a binary choice than a sliding scale. I.e. code can either be correct and have race conditions, or be incorrect and not have them.

    So I’ve been working on a change built on top of this PR and #15719 to get rid of all the Chain height methods: 0bc4a8d9530804f82fc6aa272c3cb543ed9f2263. Working on this has helped me understand this PR better, and I think I can break some pieces out of that commit to smaller PRs that remove Chain::Lock usages, so this PR can be smaller, safer, and easier to understand. Even though this will have the drawback of making overall changes larger, I think the end result should be safer, and individual changes easier to verify.

    Just wanted to give this update. I should have more comments and maybe some separate PRs related to this one soon.

  93. ryanofsky commented at 7:53 pm on January 17, 2020: member

    re: #16426#pullrequestreview-343566228

    I think I can break some pieces out of that commit to smaller PRs that remove Chain::Lock usages

    I started doing this in draft PR #17954. Will continue to update

  94. in src/interfaces/wallet.cpp:334 in 45703bee7e outdated
    336-        } else {
    337-            num_blocks = -1;
    338-            block_time = -1;
    339-        }
    340+        num_blocks = m_wallet->GetLastBlockHeight();
    341+        block_time = locked_chain->getBlockTime(num_blocks);
    


    ryanofsky commented at 8:58 pm on February 5, 2020:

    In commit “Move getHeight from Chain::Lock interface to simple Chain” (45703bee7e368f4d32676b8b22541c9fdf962181):

    Note #17954 takes a different approach, returning the actual block time of the last wallet block processed instead of trying to call locked_chain->getBlockTime which could return a different block time or even assert false if the height is too big.

  95. in src/wallet/rpcdump.cpp:569 in 45703bee7e outdated
    564@@ -565,8 +565,8 @@ UniValue importwallet(const JSONRPCRequest& request)
    565         if (!file.is_open()) {
    566             throw JSONRPCError(RPC_INVALID_PARAMETER, "Cannot open wallet dump file");
    567         }
    568-        Optional<int> tip_height = locked_chain->getHeight();
    569-        nTimeBegin = tip_height ? locked_chain->getBlockTime(*tip_height) : 0;
    570+        Optional<int> tip_height = pwallet->chain().getHeight();
    571+        nTimeBegin = locked_chain->getBlockTime(*tip_height);
    


    ryanofsky commented at 9:02 pm on February 5, 2020:

    In commit “Move getHeight from Chain::Lock interface to simple Chain” (45703bee7e368f4d32676b8b22541c9fdf962181):

    Note #17954 takes a different approach, using block time of the last wallet block processed, which should give a more correct rescan time

  96. in src/wallet/rpcdump.cpp:795 in 45703bee7e outdated
    789@@ -790,9 +790,9 @@ UniValue dumpwallet(const JSONRPCRequest& request)
    790     // produce output
    791     file << strprintf("# Wallet dump created by Bitcoin %s\n", CLIENT_BUILD);
    792     file << strprintf("# * Created on %s\n", FormatISO8601DateTime(GetTime()));
    793-    const Optional<int> tip_height = locked_chain->getHeight();
    794-    file << strprintf("# * Best block at time of backup was %i (%s),\n", tip_height.get_value_or(-1), tip_height ? locked_chain->getBlockHash(*tip_height).ToString() : "(missing block hash)");
    795-    file << strprintf("#   mined on %s\n", tip_height ? FormatISO8601DateTime(locked_chain->getBlockTime(*tip_height)) : "(missing block time)");
    796+    int tip_height = pwallet->GetLastBlockHeight();
    797+    file << strprintf("# * Best block at time of backup was %i (%s),\n", tip_height, locked_chain->getBlockHash(tip_height).ToString());
    798+    file << strprintf("#   mined on %s\n", FormatISO8601DateTime(locked_chain->getBlockTime(tip_height)));
    


    ryanofsky commented at 9:05 pm on February 5, 2020:

    In commit “Move getHeight from Chain::Lock interface to simple Chain” (45703bee7e368f4d32676b8b22541c9fdf962181):

    Note #17954 takes a different approach, using block hash and time of the last wallet block processed, which gives more correct information and avoids possibility of getBlockHash and getBlockTime asserting false

  97. in src/wallet/rpcdump.cpp:1369 in 45703bee7e outdated
    1364@@ -1365,8 +1365,8 @@ UniValue importmulti(const JSONRPCRequest& mainRequest)
    1365         EnsureWalletIsUnlocked(pwallet);
    1366 
    1367         // Verify all timestamps are present before importing any keys.
    1368-        const Optional<int> tip_height = locked_chain->getHeight();
    1369-        now = tip_height ? locked_chain->getBlockMedianTimePast(*tip_height) : 0;
    1370+        const Optional<int> tip_height = pwallet->chain().getHeight();
    1371+        now = locked_chain->getBlockMedianTimePast(*tip_height);
    


    ryanofsky commented at 9:10 pm on February 5, 2020:

    In commit “Move getHeight from Chain::Lock interface to simple Chain” (45703bee7e368f4d32676b8b22541c9fdf962181):

    Note #17954 takes a different approach, using block hash and time of the last wallet block processed, which should choose a better rescan time and avoid possibility of getBlockMedianTimePast asserting false

  98. in src/wallet/rpcwallet.cpp:1638 in 45703bee7e outdated
    1633@@ -1634,8 +1634,8 @@ static UniValue listsinceblock(const JSONRPCRequest& request)
    1634         --*altheight;
    1635     }
    1636 
    1637-    int last_height = tip_height ? *tip_height + 1 - target_confirms : -1;
    1638-    uint256 lastblock = last_height >= 0 ? locked_chain->getBlockHash(last_height) : uint256();
    1639+    int lastheight = tip_height + 1 - target_confirms;
    1640+    uint256 lastblock = lastheight >= 0 ? locked_chain->getBlockHash(lastheight) : uint256();
    


    ryanofsky commented at 9:17 pm on February 5, 2020:

    In commit “Move getHeight from Chain::Lock interface to simple Chain” (45703bee7e368f4d32676b8b22541c9fdf962181):

    Note: #17954 takes a different approach here, setting lastblock to the ancestor block of the last wallet block processed, which should be more correct and avoid the possibility of getBlockHash asserting false

  99. in src/wallet/rpcwallet.cpp:3530 in 45703bee7e outdated
    3526@@ -3527,7 +3527,7 @@ UniValue rescanblockchain(const JSONRPCRequest& request)
    3527     uint256 start_block, stop_block;
    3528     {
    3529         auto locked_chain = pwallet->chain().lock();
    3530-        Optional<int> tip_height = locked_chain->getHeight();
    3531+        Optional<int> tip_height = pwallet->chain().getHeight();
    


    ryanofsky commented at 9:21 pm on February 5, 2020:

    In commit “Move getHeight from Chain::Lock interface to simple Chain” (45703bee7e368f4d32676b8b22541c9fdf962181):

    Note: #17954 takes a different approach here, using the height of the last wallet block processed, which gives better error feedback if the wallet sync is behind and avoids the possibility of getBlockHash asserting false below

  100. in src/wallet/test/wallet_tests.cpp:158 in 45703bee7e outdated
    151@@ -152,6 +152,10 @@ BOOST_FIXTURE_TEST_CASE(importmulti_rescan, TestChain100Setup)
    152     // after.
    153     {
    154         std::shared_ptr<CWallet> wallet = std::make_shared<CWallet>(chain.get(), WalletLocation(), WalletDatabase::CreateDummy());
    155+        {
    156+            LOCK(wallet->cs_wallet);
    157+            wallet->SetLastBlockProcessed(::ChainActive().Height(), ::ChainActive().Tip()->GetBlockHash());
    158+        }
    


    ryanofsky commented at 9:27 pm on February 5, 2020:

    In commit “Move getHeight from Chain::Lock interface to simple Chain” (45703bee7e368f4d32676b8b22541c9fdf962181):

    Note: #17954 is pretty similar but adds SetLastBlockProcessed in slightly different places and uses WITH_LOCK shorthand

  101. in src/wallet/wallet.cpp:1554 in 45703bee7e outdated
    1550@@ -1551,7 +1551,7 @@ int64_t CWallet::RescanFromTime(int64_t startTime, const WalletRescanReserver& r
    1551     {
    1552         auto locked_chain = chain().lock();
    1553         const Optional<int> start_height = locked_chain->findFirstBlockWithTimeAndHeight(startTime - TIMESTAMP_WINDOW, 0, &start_block);
    1554-        const Optional<int> tip_height = locked_chain->getHeight();
    1555+        const Optional<int> tip_height = chain().getHeight();
    


    ryanofsky commented at 9:36 pm on February 5, 2020:

    In commit “Move getHeight from Chain::Lock interface to simple Chain” (45703bee7e368f4d32676b8b22541c9fdf962181):

    Note: #17954 takes a different approach here, using the last wallet processed height as the tip height which I think makes the log statement more technically correct (even though it’s kind of a weird number to print since the eventual number of blocks scanned could be different).

  102. in src/wallet/wallet.cpp:2454 in 45703bee7e outdated
    2451         return false;
    2452     }
    2453     constexpr int64_t MAX_ANTI_FEE_SNIPING_TIP_AGE = 8 * 60 * 60; // in seconds
    2454-    if (locked_chain.getBlockTime(*locked_chain.getHeight()) < (GetTime() - MAX_ANTI_FEE_SNIPING_TIP_AGE)) {
    2455+    auto locked_chain = chain.lock();
    2456+    if (locked_chain->getBlockTime(block_height) < (GetTime() - MAX_ANTI_FEE_SNIPING_TIP_AGE)) {
    


    ryanofsky commented at 9:48 pm on February 5, 2020:

    In commit “Move getHeight from Chain::Lock interface to simple Chain” (45703bee7e368f4d32676b8b22541c9fdf962181):

    Note: #17954 takes a different approach here, using the block time of the last wallet processed block, instead of calling getBlockTime with an out of sync height number which potentially return a strange value or assert false

  103. in src/wallet/rpcdump.cpp:363 in 4ac9330ad5 outdated
    358@@ -359,7 +359,8 @@ UniValue importprunedfunds(const JSONRPCRequest& request)
    359     }
    360 
    361     auto locked_chain = pwallet->chain().lock();
    362-    Optional<int> height = locked_chain->getBlockHeight(merkleBlock.header.GetHash());
    363+    LOCK(pwallet->cs_wallet);
    364+    Optional<int> height = pwallet->chain().getBlockHeight(merkleBlock.header.GetHash());
    


    ryanofsky commented at 9:54 pm on February 5, 2020:

    In commit “Move getBlockHeight from Chain::Lock interface to simple Chain” (4ac9330ad5d61e63983e2cf749e365389cef15c9)

    Note: #17954 takes a slightly different approach and is a little more strict, raising an error if merkle block is not an ancestor of the wallet last processed block (even if it is an ancestor of the node tip)

  104. in src/wallet/wallet.cpp:3403 in 4ac9330ad5 outdated
    3408+            // iterate over all their outputs
    3409+            for (const auto& keyid : GetAffectedKeys(txout.scriptPubKey, *spk_man)) {
    3410+                // ... and all their affected keys
    3411+                std::map<CKeyID, int>::iterator rit = mapKeyFirstBlock.find(keyid);
    3412+                if (rit != mapKeyFirstBlock.end() && wtx.m_confirm.block_height < rit->second) {
    3413+                    rit->second = wtx.m_confirm.block_height;
    


    ryanofsky commented at 7:54 pm on February 6, 2020:

    In commit “Move getBlockHeight from Chain::Lock interface to simple Chain” (4ac9330ad5d61e63983e2cf749e365389cef15c9)

    There doesn’t seem to be anything here excluding conflicted transactions now that getBlockHeight is called. Also it seems like getBlockTime call below could assert false if the wallet is behind sync and height is out of bounds. #17954 takes a different approach here just sticking with confirmation data in the wallet and not complicating things by involving the current node tip & active chain

  105. in src/interfaces/chain.h:127 in 51addd56ec outdated
    121@@ -125,6 +122,10 @@ class Chain
    122     //! included in the current chain.
    123     virtual Optional<int> getBlockHeight(const uint256& hash) = 0;
    124 
    125+    //! Get block hash. Returns nullopt for a block not included in the
    126+    //! current chain.
    127+    virtual Optional<uint256> getBlockHash(int height) = 0;
    


    ryanofsky commented at 7:58 pm on February 6, 2020:

    In commit “Move getBlockHash from Chain::Lock interface to simple Chain” (51addd56eccda4dd91843920ae7a929af4ba8d60)

    Would prefer not to have to add this method. It just seems inherently racy to be looking up a block based on height with no lock held. #17954 avoids the need to add this by changing more wallet code to use hashes instead of heights

  106. in src/wallet/rpcdump.cpp:794 in 51addd56ec outdated
    789@@ -790,7 +790,8 @@ UniValue dumpwallet(const JSONRPCRequest& request)
    790     file << strprintf("# Wallet dump created by Bitcoin %s\n", CLIENT_BUILD);
    791     file << strprintf("# * Created on %s\n", FormatISO8601DateTime(GetTime()));
    792     int tip_height = pwallet->GetLastBlockHeight();
    793-    file << strprintf("# * Best block at time of backup was %i (%s),\n", tip_height, locked_chain->getBlockHash(tip_height).ToString());
    794+    Optional<uint256> block_hash = pwallet->chain().getBlockHash(tip_height);
    795+    file << strprintf("# * Best block at time of backup was %i (%s),\n", tip_height, block_hash ? (*block_hash).ToString() : "(missing block hash)");
    


    ryanofsky commented at 8:01 pm on February 6, 2020:

    In commit “Move getBlockHash from Chain::Lock interface to simple Chain” (51addd56eccda4dd91843920ae7a929af4ba8d60)

    Could write block_hash-> instead of (*block_hash). #17954 also takes a different approach of using wallet last processed block hash, which I think is more consistent

  107. in src/wallet/rpcwallet.cpp:1637 in 51addd56ec outdated
    1633@@ -1634,8 +1634,8 @@ static UniValue listsinceblock(const JSONRPCRequest& request)
    1634         --*altheight;
    1635     }
    1636 
    1637-    int lastheight = tip_height + 1 - target_confirms;
    1638-    uint256 lastblock = lastheight >= 0 ? locked_chain->getBlockHash(lastheight) : uint256();
    1639+    Optional<uint256> lastblock_hash = pwallet->chain().getBlockHash(tip_height + 1 - target_confirms);
    


    ryanofsky commented at 8:05 pm on February 6, 2020:

    In commit “Move getBlockHash from Chain::Lock interface to simple Chain” (51addd56eccda4dd91843920ae7a929af4ba8d60)

    I don’t think this is going to do the right thing after Chain::Lock is removed from the code. This is going to return last black hash relative to the node tip, which can vary while this code is running, instead of relative to the wallet last block processed tip, which should be fixed. #17954 uses a more consistent approach ignoring the node tip.

  108. in src/wallet/rpcwallet.cpp:3571 in 51addd56ec outdated
    3569+            if (stop_height >= 0) {
    3570+                Optional<uint256> block_hash = pwallet->chain().getBlockHash(stop_height);
    3571+                if (block_hash) {
    3572+                    stop_block = *block_hash;
    3573+                } else {
    3574+                    throw JSONRPCError(RPC_MISC_ERROR, "stop_block has been reorged out of main chain");
    


    ryanofsky commented at 8:07 pm on February 6, 2020:

    In commit “Move getBlockHash from Chain::Lock interface to simple Chain” (51addd56eccda4dd91843920ae7a929af4ba8d60)

    IMO would be better to just continuing using the height value passed, rather than throwing an error if the node tip changes in the background. Implementing this actually makes things simpler since it doesn’t require these extra checks. This is done in #17954

  109. in src/wallet/wallet.cpp:1677 in 51addd56ec outdated
    1674             // handle updated tip hash
    1675             const uint256 prev_tip_hash = tip_hash;
    1676-            tip_hash = locked_chain->getBlockHash(*tip_height);
    1677+            Optional<uint256> optional_tip_hash = chain().getBlockHash(*tip_height);
    1678+            if (!optional_tip_hash) {
    1679+                break;
    


    ryanofsky commented at 8:39 pm on February 6, 2020:

    In commit “Move getBlockHash from Chain::Lock interface to simple Chain” (51addd56eccda4dd91843920ae7a929af4ba8d60)

    It’s not right to completely stop scanning the entire chain just because the tip block happened to be reorged away, and returning success as if the scan completed. I’m pretty sure rescan code is already somewhat broken and returning success in cases where it should return failure, but this change could making it a lot more fragile by essentially skipping the entire scan. #17954 does better here by just trying to rescan to the wallet last block processed instead of the node tip

  110. in src/interfaces/chain.cpp:206 in 998597bfc4 outdated
    243+        CBlockIndex* block = ::ChainActive()[height];
    244+        if (block != nullptr) {
    245+            return block->GetBlockTime();
    246+        }
    247+        return nullopt;
    248+    }
    


    ryanofsky commented at 8:42 pm on February 6, 2020:

    In commit “Move getBlockTime from Chain::Lock interface to simple Chain” (998597bfc4ca8fd0e205f261e79509fa999565a0)

    Again would prefer not to add this method, because it seems inherently racy to look up block data by height when no lock is held and the active chain can change at any time. #17954 avoids the need for this by using findBlock and looking up blocks by hash. It makes all the getBlockTime calls sites in this commit more robust when there are reorgs

  111. in src/wallet/wallet.cpp:3417 in 998597bfc4 outdated
    3411@@ -3412,8 +3412,10 @@ void CWallet::GetKeyBirthTimes(interfaces::Chain::Lock& locked_chain, std::map<C
    3412     }
    3413 
    3414     // Extract block timestamps for those keys
    3415-    for (const auto& entry : mapKeyFirstBlock)
    3416-        mapKeyBirth[entry.first] = locked_chain.getBlockTime(entry.second) - TIMESTAMP_WINDOW; // block times can be 2h off
    3417+    for (const auto& entry : mapKeyFirstBlock) {
    3418+        Optional<int64_t> block_time = chain().getBlockTime(entry.second);
    3419+        mapKeyBirth[entry.first] = !block_time ? 0 : *block_time - TIMESTAMP_WINDOW; // block times can be 2h off
    


    ryanofsky commented at 8:47 pm on February 6, 2020:

    In commit “Move getBlockTime from Chain::Lock interface to simple Chain” (998597bfc4ca8fd0e205f261e79509fa999565a0)

    #17954 handles this differently by looking up blocks by hash and not setting 0 times when there is a reorg

  112. in src/wallet/rpcdump.cpp:1375 in b61867bd11 outdated
    1368@@ -1369,8 +1369,11 @@ UniValue importmulti(const JSONRPCRequest& mainRequest)
    1369         EnsureWalletIsUnlocked(pwallet);
    1370 
    1371         // Verify all timestamps are present before importing any keys.
    1372-        const Optional<int> tip_height = pwallet->chain().getHeight();
    1373-        now = locked_chain->getBlockMedianTimePast(*tip_height);
    1374+        Optional<int> tip_height = pwallet->chain().getHeight();
    1375+        if (tip_height) {
    1376+            const Optional<int64_t> tip_mtp = pwallet->chain().getBlockMedianTimePast(*tip_height);
    1377+            now = tip_mtp ? *tip_mtp : 0;
    


    ryanofsky commented at 8:53 pm on February 6, 2020:

    In commit “Move getBlockMedianTimePast from Chain::Lock interface to simple Chain” (b61867bd11bf797555c486223b6d4ce7dd290152)

    This seems like it could result in rescanning the entire chain if there’s a reorg between get height and get time calls. It’s be really unlikely, but this just seems unnecessarily fragile. #17954 avoids this using block hashes

  113. in src/interfaces/chain.h:113 in afa43f9560 outdated
    120+    virtual CBlockLocator getTipLocator() = 0;
    121+
    122+    //! Return height of the highest block on chain in common with the locator,
    123+    //! which will either be the original block used to create the locator,
    124+    //! or one of its ancestors.
    125+    virtual Optional<int> findLocatorFork(const CBlockLocator& locator) = 0;
    


    ryanofsky commented at 8:58 pm on February 6, 2020:

    In commit “Move multiple methods from Chain::Lock interface to simple Chain” (afa43f9560b57ae2b45b725ba80bd68f191bd690)

    Would prefer not to add these methods returning only heights instead of hashes. With the locks removed in the next commit “Remove locked_chain from CWallet”, the heights returned by these methods will not uniquely identify blocks and code will be inherently racy. #17954 does a better job here eliminiating these or changing them to return hashes

  114. ryanofsky approved
  115. ryanofsky commented at 9:25 pm on February 6, 2020: member

    Code review semi-ACK ac05c8b8119e4f0a055a26a98d69e4b6cbfdd2c1

    I finally reviewed this whole PR and would be ok with merging it in its current form. But my preference would be to rebase this PR on top of #17954 and merge it after. Reasons:

    1. This PR seems to add a lot of little race conditions, looking up blocks by height instead of hash when the chain state is not locked. I don’t think any of these are likely to ever trigger in practice or cause serious problems if they did trigger, but it leaves the code in an awkward state to have these.

    2. Ordering #17954 before this PR would reduce churn in Chain interface methods. It would avoid adding temporary methods like getBlockTime that we are just going to delete later. (For the same reason it also might be nice to merge #17443 before this, to avoid needing to add Chain::checkFinalTx which that PR would get rid of.)

    3. Ordering #17954 before this PR would reduce churn in wallet code and make the changes there easier to review. #17954 is structured to update complicated functions like listsinceblock and CWallet::ScanForWalletTransactions once in individual commits. By contrast this PR updates the same functions multiple times in different commits, which I think can make it hard to see how the changes fit together. If #17954 is ordered first, most of these changes should be unnecessary.

  116. laanwj added the label Waiting for author on Feb 26, 2020
  117. laanwj commented at 4:40 pm on February 26, 2020: member
    This needs rebase and addressing @ryanofsky’s comments.
  118. ryanofsky commented at 5:07 pm on February 27, 2020: member

    This needs rebase and addressing @ryanofsky’s comments.

    This is basically just blocked on #17954.

    Antoine has a rebased version of this PR on top of #17954 and #17443, which both simplify this PR in a working branch https://github.com/bitcoin/bitcoin/compare/master...ariard:2020-02-wallet-standalone

    The relevant commits in that branch make a smaller version of this PR:

    • 23c478e0515571ef8291a337d9c351d487b059e3 Move getBlockHeight from Chain::Lock interface to simple Chain
    • 7a17b679aad171b67426d74b445cec3ac16c0043 Move getBlockHash from Chain::Lock interface to simple Chain
    • 08c90335510c95f845c840407058699385abf93c Move multiple methods from Chain::Lock interface to simple Chain
    • d2fa1e1ad3bad97436142262964707ab87f395f1 Remove locked_chain from CWallet, its RPCs and tests

    (Later commits on that branch have to do with implementing rescan logic outside of the wallet so it is cleaner and can allow txindex, blockfilterindex, and multiple wallet rescans to all happen in parallel without each wallet and index having to loop and reread the same blocks. The nonwallet parts are in another branch: https://github.com/bitcoin/bitcoin/compare/master...ariard:2019-08-rescan-index-refactor)

  119. ariard commented at 9:12 pm on February 28, 2020: member
    @laanwj Thanks for the push-up, we agreed with Russ to get #17954 first, almost of all of his comments are related in way #17954 do it better if rebased on top. Also it would let me remove few commits. Updated PR description.
  120. laanwj removed this from the "Blockers" column in a project

  121. MarcoFalke referenced this in commit 4702cadca9 on Apr 14, 2020
  122. sidhujag referenced this in commit c4eb63fbfe on Apr 14, 2020
  123. ryanofsky commented at 5:28 pm on April 15, 2020: member
    @ariard, it would be great to rebase this now that #17954 is merged. It should be simpler now, but good to merge early in the release cycle since it changes wallet locking in a big way
  124. ariard force-pushed on Apr 17, 2020
  125. ariard commented at 1:09 am on April 17, 2020: member

    @ryanofsky Thanks for the push-up after #17954, it simplifies this PR so much. Let me know if you think I should split/aggregate commits.

    Note, I removed findPruned and findFork as they were useless after your PR.

  126. DrahtBot removed the label Needs rebase on Apr 17, 2020
  127. MarcoFalke commented at 11:54 am on April 17, 2020: member

    https://travis-ci.org/github/bitcoin/bitcoin/jobs/676012190#L7078

     02020-04-17T04:31:03.393017Z POTENTIAL DEADLOCK DETECTED
     1
     22020-04-17T04:31:03.393067Z Previous lock order was:
     3
     42020-04-17T04:31:03.393091Z  (1) cs_wallet wallet/wallet.cpp:1667 (in thread )
     5
     62020-04-17T04:31:03.393138Z  (2) cs_main interfaces/chain.cpp:218 (in thread )
     7
     82020-04-17T04:31:03.393211Z Current lock order is:
     9
    102020-04-17T04:31:03.393233Z  (2) ::cs_main wallet/test/wallet_tests.cpp:464 (in thread )
    11
    122020-04-17T04:31:03.393275Z  (1) wallet->cs_wallet wallet/test/wallet_tests.cpp:464 (in thread )
    13
    14Assertion failed: detected inconsistent lock order at sync.cpp:124, details in debug log.
    
  128. MarcoFalke removed the label Waiting for author on Apr 17, 2020
  129. DrahtBot added the label Needs rebase on Apr 19, 2020
  130. sidhujag referenced this in commit 36bfea40fa on Apr 19, 2020
  131. ryanofsky referenced this in commit f9cea7231d on Apr 21, 2020
  132. in src/wallet/wallet.cpp:3914 in ef8c6ca607 outdated
    3982+    // be pending on the validation-side until lock release. It's likely to have
    3983+    // block processing duplicata (if rescan block range overlaps with notification one)
    3984+    // but we guarantee at least than wallet state is correct after notifications delivery.
    3985+    // This is temporary until rescan and notifications delivery are unified under same
    3986+    // interface.
    3987+    walletInstance->m_chain_notifications_handler = walletInstance->chain().handleNotifications(walletInstance);
    


    ryanofsky commented at 8:15 pm on April 21, 2020:

    In commit “[wallet] Remove locked_chain from CWallet, its RPCs and tests” (ef8c6ca60767cac589d98ca57ee33179608ccda8)

    Note I wrote a test to detect the race condition this early handleNotifications call prevents: 25651aad58b1f6e543f1ad565d821de46268e724 (branch). Most of the test works on master so I made a separate PR #18727 for it. There’s just a few lines in 25651aad58b1f6e543f1ad565d821de46268e724 not in that PR, helping the test work with this PR

  133. in src/interfaces/chain.h:86 in b4eb6aee0d outdated
    129@@ -135,6 +130,11 @@ class Chain
    130     //! unlocked when the returned interface is freed.
    131     virtual std::unique_ptr<Lock> lock(bool try_lock = false) = 0;
    132 
    133+    //! Get current chain height, not including genesis block (returns 0 if
    134+    //! chain only contains genesis block, nullopt if chain does not contain
    135+    //! any blocks)
    136+    virtual Optional<int> getHeight() = 0;
    


    ryanofsky commented at 8:35 pm on April 21, 2020:

    In commit “[wallet] Move getHeight from Chain::Lock interface to simple Chain” (b4eb6aee0d259c3586894694a67928759503c823)

    Note: After this commit there’s just a single Chain::getHeight() call remaining in CreateWalletFromFile, which #15719 then removes. I expect this PR will get merged before #15719, so just making a note about that.


    ariard commented at 0:26 am on April 23, 2020:
    Okay looking through #15719, this solve the issue by doing wallet rescan on the server-side so you don’t need to get current tip.
  134. in src/interfaces/wallet.cpp:323 in b4eb6aee0d outdated
    334@@ -335,7 +335,7 @@ class WalletImpl : public Wallet
    335         LOCK(m_wallet->cs_wallet);
    336         auto mi = m_wallet->mapWallet.find(txid);
    337         if (mi != m_wallet->mapWallet.end()) {
    338-            num_blocks = locked_chain->getHeight().get_value_or(-1);
    339+            num_blocks = m_wallet->GetLastBlockHeight();
    


    ryanofsky commented at 8:49 pm on April 21, 2020:

    In commit “[wallet] Move getHeight from Chain::Lock interface to simple Chain” (b4eb6aee0d259c3586894694a67928759503c823)

    Note: This change should just cause a very minor change behavior. Even more minor than other num_blocks changes previously discussed. The GUI isn’t using the num_blocks value for polling, just for display in transaction html status


    ariard commented at 0:29 am on April 23, 2020:
    m_last_block_processed_height is already initialized at -1, so behavior change is in case wallet isn’t at tip right ? If yes, it should be more accurate.
  135. in src/interfaces/chain.h:91 in 58f0deef83 outdated
    129@@ -135,6 +130,11 @@ class Chain
    130     //! any blocks)
    131     virtual Optional<int> getHeight() = 0;
    132 
    133+    //! Get block height above genesis block. Returns 0 for genesis block,
    134+    //! 1 for following block, and so on. Returns nullopt for a block not
    135+    //! included in the current chain.
    136+    virtual Optional<int> getBlockHeight(const uint256& hash) = 0;
    


    ryanofsky commented at 8:59 pm on April 21, 2020:

    In commit “[wallet] Move getBlockHeight from Chain::Lock interface to simple Chain” (58f0deef83d82b8f132bab80999e60af6adde55f)

    Feel free to ignore this suggestion, but instead of adding a new getBlockHeight method, code could call the existing Chain::findBlock method, and the returned FoundBlock could have a new inChain(bool& in_chain) output slot. This should save a little code, maybe make the in-chain restriction more obvious, reduce the number of Chain methods, and increase usefulness of other Chain methods which return FoundBlock


    ariard commented at 0:58 am on April 23, 2020:
    Okay going to let it as it is, given it would need an extension of FoundBlock, and this being gauged by other reviewers. Let’s keep diff straight.
  136. in src/interfaces/chain.h:94 in c6fa030ca8 outdated
    131@@ -135,6 +132,9 @@ class Chain
    132     //! included in the current chain.
    133     virtual Optional<int> getBlockHeight(const uint256& hash) = 0;
    134 
    135+    //! Get block hash. Height must be valid or this function will abort.
    136+    virtual uint256 getBlockHash(int height) = 0;
    


    ryanofsky commented at 9:11 pm on April 21, 2020:

    In commit “[wallet] Move getBlockHash from Chain::Lock interface to simple Chain” (c6fa030ca8a282a7906260186e93d8c8902aa5a4)

    Note: After this commit, the only remaining Chain::getBlockHash() calls are in CreateWalletFromFile and are removed in #15719. I expect this PR will get merged before #15719, so just making a note about this.


    ariard commented at 0:38 am on April 23, 2020:
    Yes I have also #17484, #17443 which remove more interface methods to skim in after.
  137. in src/interfaces/chain.cpp:230 in b4eb6aee0d outdated
    224@@ -234,6 +225,15 @@ class ChainImpl : public Chain
    225         std::unique_ptr<Chain::Lock> result = std::move(lock); // Temporary to avoid CWG 1579
    226         return result;
    227     }
    228+    Optional<int> getHeight() override
    229+    {
    230+        WAIT_LOCK(cs_main, lock);
    


    ryanofsky commented at 9:17 pm on April 21, 2020:

    In commit “[wallet] Move getHeight from Chain::Lock interface to simple Chain” (b4eb6aee0d259c3586894694a67928759503c823)

    This is fine, but in this commit and other commits throughout this PR could simplify WAIT_LOCK(cs_main, lock) to just LOCK(cs_main) since lock object is unused


    ariard commented at 0:45 am on April 23, 2020:
    Right, only difference between them is what is logged for debug, attributing a name for the lock for WAIT_LOCK(cs_main, lock)? They both declare DebugLock<decltype>. Updating
  138. in src/interfaces/chain.h:98 in 22d0fb127e outdated
    104@@ -135,6 +105,28 @@ class Chain
    105     //! Get block hash. Height must be valid or this function will abort.
    106     virtual uint256 getBlockHash(int height) = 0;
    107 
    108+    //! Check that the block is available on disk (i.e. has not been
    109+    //! pruned), and contains transactions.
    110+    virtual bool haveBlockOnDisk(int height) = 0;
    


    ryanofsky commented at 9:25 pm on April 21, 2020:

    In commit “[wallet] Move methods from Chain::Lock interface to simple Chain” (22d0fb127e99d6cdcaa003785a7278f58038b465)

    Note: After this commit, the only remaining Chain::haveBlockOnDisk, findFirstBlockWithTimeAndHeight, getTipLocator, and findLocatorFork calls are in CreateWalletFromFile and are removed in #15719. I expect this PR will get merged before #15719, so just making a note about this.

    Similarly, the Chain::checkFinalTx method can go away after #17443, which removes all those calls

  139. ryanofsky approved
  140. ryanofsky commented at 9:36 pm on April 21, 2020: member
    Code review ACK ef8c6ca60767cac589d98ca57ee33179608ccda8. Looks great! This PR now just has safer changes and I think is easier to review. I left a bunch of notes and maybe two minor suggestions, which you should feel free to ignore. This does need rebase but otherwise looks good to merge
  141. ariard force-pushed on Apr 23, 2020
  142. ariard commented at 1:01 am on April 23, 2020: member
    Rebased at d6d6632, with some Russ suggestions. Test failure was due to deadlock in test framework introduced by a new LOCK2 and not cached at last rebae.
  143. MarcoFalke commented at 10:56 am on April 23, 2020: member
    Tests are still failing
  144. ariard force-pushed on Apr 23, 2020
  145. ariard force-pushed on Apr 23, 2020
  146. DrahtBot removed the label Needs rebase on Apr 23, 2020
  147. promag commented at 10:29 am on April 27, 2020: member

    Code review ACK 0716c9cb68de74cd6e3f791dbe2aec103f70e292. Still running some tests.

    So loading an old wallet (that needs rescan) still freezes the node until the wallet catches up. Maybe it could try to register as late as possible? Like before the last rescanned block?

  148. MarcoFalke commented at 3:50 pm on April 27, 2020: member
    needs rebase
  149. fjahr commented at 4:32 pm on April 27, 2020: member

    re-ACK 0716c9cb68de74cd6e3f791dbe2aec103f70e292

    Did another full review of the code and built and ran tests locally.

  150. ryanofsky referenced this in commit 2142bf14fc on Apr 27, 2020
  151. ryanofsky referenced this in commit 1dcbe64463 on Apr 27, 2020
  152. kallewoof commented at 4:56 am on April 28, 2020: member
    Typo in commit 4509c05eab82dda405b4265838a61ed7ca302bf7 message: chain acceschain access.
  153. in src/wallet/test/wallet_tests.cpp:243 in ac543129b3 outdated
    239@@ -240,6 +240,10 @@ BOOST_FIXTURE_TEST_CASE(importwallet_rescan, TestChain100Setup)
    240         std::shared_ptr<CWallet> wallet = std::make_shared<CWallet>(chain.get(), WalletLocation(), WalletDatabase::CreateDummy());
    241         LOCK(wallet->cs_wallet);
    242         wallet->SetupLegacyScriptPubKeyMan();
    243+        {
    


    kallewoof commented at 4:59 am on April 28, 2020:
    This is added here and then removed 2 commits later, and looks like a rebase error. I would love to have a comment pointing out that this is temporary to fix tests in between commits (assuming that’s what it is). If you end up rebasing, adding would be great.

    ariard commented at 6:23 am on April 28, 2020:
    Yeah was a rebase error due to lock being added in between in another PR. Fixed, thanks!
  154. kallewoof commented at 5:13 am on April 28, 2020: member
    Code review ACK 0716c9cb68de74cd6e3f791dbe2aec103f70e292
  155. ariard force-pushed on Apr 28, 2020
  156. ariard commented at 6:23 am on April 28, 2020: member

    Rebased 49ecec4. Thanks a lot @promag @fjahr @kallewoof for reviews! If you can re-ACK there should be no change apart of dropping locks added in importdescripotrs in the meanwhile.

    So loading an old wallet (that needs rescan) still freezes the node until the wallet catches up. Maybe it could try to register as late as possible? Like before the last rescanned block?

    It’s planned in future works, see #15719 by Russ. I’ve also a branch doing it in an alternative way.

  157. ryanofsky approved
  158. ryanofsky commented at 1:35 pm on April 28, 2020: member

    Code review ACK 49ecec42ea81e9cbf3970416ead79e18863dea87. Changes since last review: rebase, switching from WAIT_LOCK to LOCK, dropping assert in getBlockHash (still segfaults but this is temporary #16426 (review)), switched to consistent lock order in ListCoins test

    This has a few recent ACKs so I think may be ready to merge or very close

  159. ryanofsky commented at 1:41 pm on April 28, 2020: member

    re: #16426#pullrequestreview-400824121

    So loading an old wallet (that needs rescan) still freezes the node until the wallet catches up. @promag is this comment saying the PR is introducing a new freeze which is not in master, or is this talking about an existing freeze? Either way I think #15719 may address it, but it would be good to know what the exact issue is (steps to reproduce and observed behavior)

  160. ryanofsky referenced this in commit 66a4d4b2ec on Apr 28, 2020
  161. jonatack commented at 10:21 pm on April 28, 2020: member
    Finishing up review of Signet PR, I’ll try to review this tomorrow @ariard
  162. in src/interfaces/chain.cpp:171 in 27b1af965e outdated
    226@@ -234,6 +227,12 @@ class ChainImpl : public Chain
    227         }
    228         return nullopt;
    229     }
    230+    uint256 getBlockHash(int height) override
    231+    {
    232+        LOCK(::cs_main);
    233+        CBlockIndex* block = ::ChainActive()[height];
    234+        return block->GetBlockHash();
    


    MarcoFalke commented at 2:37 pm on April 29, 2020:

    why remove the assert and make it undefined behavior?

    27b1af965e


    ryanofsky commented at 6:05 pm on April 29, 2020:

    re: #16426 (review)

    why remove the assert and make it undefined behavior?

    27b1af9

    I think it came from a suggestion I made #16426 (review) that wasn’t clear about a desire not to crash :smile:. But this is only called one place where it’s extremely unlikely to trigger (there’d have to be a very specific thread pre-emption at the same time as a reorg at the same time as loading a wallet)

    https://github.com/ariard/bitcoin/blob/49ecec42ea81e9cbf3970416ead79e18863dea87/src/wallet/wallet.cpp#L3927-L3929

    Also this method should be removed with #15719


    MarcoFalke commented at 7:22 pm on April 29, 2020:

    But this is only called one place where it’s extremely unlikely to trigger

    Sounds like an excellent use-case for assert instead of extremely unlikely undefined behavior. Especially given that the assert was there in the fist place.

    I don’t see why removing the assert can simply wait until #15719


    jonatack commented at 7:32 pm on April 29, 2020:
    Agree with @MarcoFalke here.

    ryanofsky commented at 8:15 pm on April 29, 2020:

    re: #16426 (review)

    I don’t see why removing the assert can simply wait until #15719

    Of course. My suggestion was never to remove the assert by segfaulting instead. Originally the suggestion was to return Optional #16426 (review). Just pointing out what’s done here isn’t too important because it’s basically impossible to hit this condition and there’s already a PR deleting the method


    ariard commented at 6:28 pm on April 30, 2020:
    By removing cs_main lock taking you may have now a block nullptr in the case where between getHeight/findFirstBlockWithTimeAndHeight and getBlockHash calls you have a thread taking cs_main lock, a reorg happening while at same time wallet loading. It’s really unlikely but glad #15719 is cleaning this as a follow-up.
  163. in src/wallet/test/wallet_tests.cpp:593 in 7c73c4888f outdated
    585@@ -586,6 +586,11 @@ BOOST_FIXTURE_TEST_CASE(wallet_disableprivkeys, TestChain100Setup)
    586     auto chain = interfaces::MakeChain(node);
    587     std::shared_ptr<CWallet> wallet = std::make_shared<CWallet>(chain.get(), WalletLocation(), WalletDatabase::CreateDummy());
    588     wallet->SetupLegacyScriptPubKeyMan();
    589+    {
    590+        auto locked_chain = chain->lock();
    591+        LOCK(wallet->cs_wallet);
    592+        wallet->SetLastBlockProcessed(::ChainActive().Height(), ::ChainActive().Tip()->GetBlockHash());
    593+    }
    


    MarcoFalke commented at 2:46 pm on April 29, 2020:
    Why is this needed? 7c73c4888f

    ryanofsky commented at 6:06 pm on April 29, 2020:

    re: #16426 (review)

    Why is this needed? 7c73c48

    It does seem not to be needed anymore. Test passes for me with it commented out.

  164. in src/interfaces/wallet.cpp:71 in 9229faa752 outdated
    69     result.depth_in_main_chain = wtx.GetDepthInMainChain();
    70     result.time_received = wtx.nTimeReceived;
    71     result.lock_time = wtx.tx->nLockTime;
    72-    result.is_final = locked_chain.checkFinalTx(*wtx.tx);
    73-    result.is_trusted = wtx.IsTrusted(locked_chain);
    74+    result.is_final = wallet.chain().checkFinalTx(*wtx.tx);
    


    MarcoFalke commented at 2:49 pm on April 29, 2020:
    what is the risk of this getting out-of-sync with result.block_height or other members like result.is_trusted? 9229faa752

    ryanofsky commented at 6:05 pm on April 29, 2020:

    re: #16426 (review)

    what is the risk of this getting out-of-sync with result.block_height or other members like result.is_trusted? 9229faa

    I think only risk is gui briefly displaying transaction as final in the transaction HTML description before wallet processing catches up and other fields in the HTML description reflect that. This is probably a minor concern but #17443 should fix it


    MarcoFalke commented at 2:39 pm on April 30, 2020:
    Oh, I wasn’t aware this is only used for the gui
  165. MarcoFalke commented at 2:57 pm on April 29, 2020: member

    Approach ACK 49ecec42ea 🛒

    Signature:

     0-----BEGIN PGP SIGNED MESSAGE-----
     1Hash: SHA512
     2
     3Approach ACK 49ecec42ea 🛒
     4-----BEGIN PGP SIGNATURE-----
     5
     6iQGzBAEBCgAdFiEE+rVPoUahrI9sLGYTzit1aX5ppUgFAlwqrYAACgkQzit1aX5p
     7pUgcNgwAp3x7yEnzet/P74Z6x4BR0H3RIBb2n+oW1iOR8i1kkCEMviMketbAha8S
     8vziSxJJ45p7uQBEvetTRb3jlErpB6/tNkp7PAEVep204RTpQX03IlIrfVNg6s6Am
     9I5YF/m3DYJ2HQ5lyiTwk0+mQ5wgOKPYszYHRrxcV+x/qfxTwNN9WZnRp6TbEpiuV
    10s3B87BMz8olNUCjawcG4Y0jVahetsgH90e2wFUD0I1ERWfktHSNQ9CQaBOVWFU9L
    11nY1KtYxKE0KHPnctojN1tOeltcUUa5jQB+ezks6FACOKOs53yzG28QXzviiWD5gx
    12EThUTcn6DB/H4i7Z3QriaYsF7DCECvO+WDaHBZY+kMoCbpP6c3kCkPPD9axv9kZY
    13kA+gyIlzTqdJ7H2w2s/wsGffjiviYaowXd2R4bEuIqTlAdzt6xy+7hjMCbb5kjou
    14fB1r8eg3o6O2IM1mbWkZdH0tNK/itkxitMdvOICDk6bPAIk1zlEw6jQKa51hr61l
    15aUnAG1/b
    16=RG4E
    17-----END PGP SIGNATURE-----
    

    Timestamp of file with hash 9b0f6482a573980d5bcd1e554e17ef16e67861b355c26c79f7accca02eb8219b -

  166. ryanofsky referenced this in commit 7918c1b019 on Apr 29, 2020
  167. MarcoFalke referenced this in commit 0f204dd3f2 on Apr 29, 2020
  168. fjahr commented at 7:40 pm on April 29, 2020: member
    Re-ACK 49ecec42ea81e9cbf3970416ead79e18863dea87 but agree with #16426 (review) and #16426 (review) as worthwhile updates before merging.
  169. jonatack commented at 7:48 pm on April 29, 2020: member
    Handy to have #18727 merged in. Concept ACK, agree with @fjahr above, will re-look after rebase.
  170. MarcoFalke commented at 11:00 pm on April 29, 2020: member
    Tests fail. Needs rebase
  171. kallewoof commented at 7:18 am on April 30, 2020: member

    Re-ACK based on the diff below (derived by rebasing the old commit on master and the git diffing this against the resulting state):

     0diff --git a/src/interfaces/chain.h b/src/interfaces/chain.h
     1index ec256d7b2..e33fe54ac 100644
     2--- a/src/interfaces/chain.h
     3+++ b/src/interfaces/chain.h
     4@@ -72,7 +72,7 @@ public:
     5 //!   wallet cache it, fee estimation being driven by node mempool, wallet
     6 //!   should be the consumer.
     7 //!
     8-//! * The `guesVerificationProgress`, `getBlockHeight`, `getBlockHash`, etc
     9+//! * The `guessVerificationProgress`, `getBlockHeight`, `getBlockHash`, etc
    10 //!   methods can go away if rescan logic is moved on the node side, and wallet
    11 //!   only register rescan request.
    12 class Chain
    13diff --git a/src/wallet/rpcdump.cpp b/src/wallet/rpcdump.cpp
    14index 7e46cb0c9..c863d2253 100644
    15--- a/src/wallet/rpcdump.cpp
    16+++ b/src/wallet/rpcdump.cpp
    17@@ -1664,7 +1664,6 @@ UniValue importdescriptors(const JSONRPCRequest& main_request) {
    18     bool rescan = false;
    19     UniValue response(UniValue::VARR);
    20     {
    21-        auto locked_chain = pwallet->chain().lock();
    22         LOCK(pwallet->cs_wallet);
    23         EnsureWalletIsUnlocked(pwallet);
    24
    25@@ -1693,7 +1692,6 @@ UniValue importdescriptors(const JSONRPCRequest& main_request) {
    26     if (rescan) {
    27         int64_t scanned_time = pwallet->RescanFromTime(lowest_timestamp, reserver, true /* update */);
    28         {
    29-            auto locked_chain = pwallet->chain().lock();
    30             LOCK(pwallet->cs_wallet);
    31             pwallet->ReacceptWalletTransactions();
    32         }
    
  172. ryanofsky commented at 2:11 pm on April 30, 2020: member

    Note: after #18727 the last commit here will cause https://github.com/bitcoin/bitcoin/blob/a66ba6d029b3948dec0cf092bb844f70c16b07dc/src/wallet/test/wallet_tests.cpp#L729 to fail.

    But I previously posted a version of that test which works with this PR in 25651aad58b1f6e543f1ad565d821de46268e724 (branch) in #16426 (review), so there should be no new code to write if you paste the test code from that commit:

    https://github.com/ryanofsky/bitcoin/blob/25651aad58b1f6e543f1ad565d821de46268e724/src/wallet/test/wallet_tests.cpp#L669-L771

  173. ryanofsky commented at 2:36 pm on April 30, 2020: member
    Rebased 49ecec42ea81e9cbf3970416ead79e18863dea87 -> c524664df16057763eaf2e5805af9bf8975ac3f8 (pr/rev.1 -> pr/rev.2, compare). No conflicts and no changes other than adding back getBlockHash assert to resolve #16426 (review) and adding test code which was previously posted #16426 (review) so the new test from #18727 passes
  174. MarcoFalke commented at 2:39 pm on April 30, 2020: member
    Thanks @ryanofsky
  175. [wallet] Move getHeight from Chain::Lock interface to simple Chain
    Instead of calling getHeight, we rely on CWallet::m_last_block
    processed_height where it's possible.
    b855592d83
  176. [wallet] Move getBlockHeight from Chain::Lock interface to simple Chain
    Add HaveChain to assert chain access for wallet-tool in LoadToWallet.
    de13363a47
  177. [wallet] Move getBlockHash from Chain::Lock interface to simple Chain 0a76287387
  178. [wallet] Move methods from Chain::Lock interface to simple Chain
    Remove findPruned and findFork, no more used after 17954.
    841178820d
  179. [wallet] Remove locked_chain from CWallet, its RPCs and tests
    This change is intended to make the bitcoin node and its rpc, network
    and gui interfaces more responsive while the wallet is in use. Currently
    because the node's cs_main mutex is always locked before the wallet's
    cs_wallet mutex (to prevent deadlocks), cs_main currently stays locked
    while the wallet does relatively slow things like creating and listing
    transactions.
    
    This commit only remmove chain lock tacking in wallet code, and invert
    lock order from cs_main, cs_wallet to cs_wallet, cs_main.
    must happen at once to avoid any deadlock. Previous commit were only
    removing Chain::Lock methods to Chain interface and enforcing they
    take cs_main.
    
    Remove LockChain method from CWallet and Chain::Lock interface.
    6a72f26968
  180. ariard referenced this in commit 86ec6f7ce6 on Apr 30, 2020
  181. ariard force-pushed on Apr 30, 2020
  182. ariard force-pushed on Apr 30, 2020
  183. ariard commented at 7:03 pm on April 30, 2020: member
    Whaao thanks @ryanofsky for rebase! Took your branch at 6a72f26 while fixing @MarcoFalke ’s #16426 (review).
  184. laanwj added this to the "Blockers" column in a project

  185. ryanofsky approved
  186. ryanofsky commented at 8:37 pm on April 30, 2020: member
    Code review ACK 6a72f26968cf931c985d8d4797b6264274cabd06. Only difference compared to the rebase I posted is reverting unneeded SetLastBlockProcessed change in wallet_disableprivkeys test
  187. MarcoFalke commented at 10:43 pm on April 30, 2020: member

    re-ACK 6a72f26968 🔏

    Signature:

     0-----BEGIN PGP SIGNED MESSAGE-----
     1Hash: SHA512
     2
     3re-ACK 6a72f26968 🔏
     4-----BEGIN PGP SIGNATURE-----
     5
     6iQGzBAEBCgAdFiEE+rVPoUahrI9sLGYTzit1aX5ppUgFAlwqrYAACgkQzit1aX5p
     7pUhtVAwAprBpnjBnEpN+ACT69Dfo+rp5Gz1Ew6sRxvJr88Zkc/GSfZ9rCTlcrKIH
     8TXRD5CR4ba8Cj9zQSF4wo/LmlqwqJlgEZnhil5qRi+Cpo/nqJEoJaajmFQBjDOJW
     97E68hxIxb9r9YF1G81gx71dLSHwUgx8RZp3POdZXInUigXk+XJJRwIlUckheMgXH
    10vF+als8TE+Fwv2gNyu03ggEAbNtFBi20yB/xLCApBcI6Ec3HDuj76JjhnqYQEetF
    11nxowDiw4EfUqwyD/XUek/OKQaXelynRoMhC4Xr+n1DNaWePoNune3qD/AGi2IF0v
    12VYleEGI01uPSryujDjo+o6HwWliKv7ooYjqSq4MAO387rE+hpVh368hfLqTwaPps
    13AeH12GMpgeALphIsqKzhd62PdeDUHxApvglj1zUQsuhSukx5ojX8oGNokWfA0slj
    14E+QKUT67kbM+2/s4kIJC3PLeMd97iYZfSanWrKTByjoecn42xf6hn15dldSsiqGM
    15KV4l6ZuH
    16=FnnV
    17-----END PGP SIGNATURE-----
    

    Timestamp of file with hash e9607093f00416bfc4093550ccd6a4d573cc579bde37edf8d5b8a119a0b88ea9 -

  188. fjahr commented at 10:58 pm on April 30, 2020: member

    re-ACK 6a72f26968cf931c985d8d4797b6264274cabd06

    Reviewed latest changes, built and ran tests locally.

  189. ariard commented at 11:08 pm on April 30, 2020: member
    There is a stale test on some disk I/O issue, other are valid. If you can relaunch.
  190. pierreN referenced this in commit 3eaf73ad8b on May 1, 2020
  191. MarcoFalke merged this on May 1, 2020
  192. MarcoFalke closed this on May 1, 2020

  193. jonatack commented at 1:00 pm on May 1, 2020: member
    Congrats on the merge @ariard! Sorry, I was still reviewing the BIP324 PR.
  194. MarcoFalke commented at 1:23 pm on May 1, 2020: member

    Review still welcome :)

    This had three explicit ACKs, one implicit re-ACK and a bunch of Concept ACKs. So beside being ready for merge, and also in light of the rebase hell this went through in the past, with the risk of (silent) merge conflicts putting it back in there, I went ahead and merged it.

  195. ryanofsky commented at 1:46 pm on May 1, 2020: member

    Review still welcome :)

    Testing is useful too because all the wallet code that was originally written assuming cs_main was locked and state couldn’t change in the background is now in an a more asynchronous world.

    I’m also still curious about @promag’s comment in #16426#pullrequestreview-400824121 and possible effects on the GUI. (Unclear if the comment is saying there’s a new GUI freeze.)

  196. jonatack commented at 7:54 pm on May 1, 2020: member
    All good, I agree!
  197. fanquake removed this from the "Blockers" column in a project

  198. sidhujag referenced this in commit d7c1f85877 on May 2, 2020
  199. promag commented at 11:59 pm on May 3, 2020: member

    I’m also still curious about @promag’s comment in #16426 (review) and possible effects on the GUI. @ryanofsky missed this, AFAICT this doesn’t add new issues to the GUI.

  200. domob1812 referenced this in commit 97cd04ee3b on May 4, 2020
  201. ryanofsky referenced this in commit f8d600c3e4 on Jun 1, 2020
  202. ryanofsky referenced this in commit c0ea9e57d2 on Jun 5, 2020
  203. ryanofsky referenced this in commit 175a412cc6 on Jun 5, 2020
  204. ryanofsky referenced this in commit 47b3b7a4f8 on Jun 5, 2020
  205. ryanofsky referenced this in commit 72a825d3ef on Jun 5, 2020
  206. ryanofsky referenced this in commit 0c9dea569a on Jul 1, 2020
  207. ryanofsky referenced this in commit 87bc7618cf on Jul 1, 2020
  208. ryanofsky referenced this in commit 8125ed13fc on Jul 10, 2020
  209. ryanofsky referenced this in commit 91ac0388e8 on Jul 10, 2020
  210. ryanofsky referenced this in commit ae5d16ac7a on Jul 11, 2020
  211. ryanofsky referenced this in commit feb4c159e2 on Jul 11, 2020
  212. ryanofsky referenced this in commit 80715b397f on Jul 13, 2020
  213. ryanofsky referenced this in commit eae4592ac4 on Jul 14, 2020
  214. ryanofsky referenced this in commit 3e54d36b17 on Jul 21, 2020
  215. ryanofsky referenced this in commit ba3cdcc8a0 on Aug 7, 2020
  216. ryanofsky referenced this in commit b4b381b422 on Aug 7, 2020
  217. ryanofsky referenced this in commit fd96326d31 on Aug 7, 2020
  218. ryanofsky referenced this in commit 53fae31e49 on Aug 7, 2020
  219. luke-jr referenced this in commit 74de8d57b2 on Aug 15, 2020
  220. ryanofsky referenced this in commit ad8d69fd6e on Aug 17, 2020
  221. ryanofsky referenced this in commit 097168853e on Aug 17, 2020
  222. ryanofsky referenced this in commit 3013f972bc on Sep 2, 2020
  223. ryanofsky referenced this in commit efab0d88b2 on Sep 2, 2020
  224. hebasto commented at 9:25 pm on September 19, 2020: member
    FWIW, this PR causes the issue #19049.
  225. ryanofsky referenced this in commit 4e7906269c on Sep 25, 2020
  226. ryanofsky referenced this in commit 97e683d1a1 on Sep 25, 2020
  227. ryanofsky referenced this in commit 70ed802e90 on Sep 25, 2020
  228. ryanofsky referenced this in commit 35205452b5 on Sep 25, 2020
  229. jasonbcox referenced this in commit f9f575bb01 on Oct 15, 2020
  230. deadalnix referenced this in commit 87c5aec9b3 on Oct 15, 2020
  231. jasonbcox referenced this in commit 890765dfd7 on Oct 15, 2020
  232. jasonbcox referenced this in commit 731b33f5de on Oct 15, 2020
  233. jasonbcox referenced this in commit 1cb2090b01 on Oct 15, 2020
  234. janus referenced this in commit c0add1ba79 on Nov 15, 2020
  235. ryanofsky referenced this in commit d0abe4e56d on Dec 8, 2020
  236. ryanofsky referenced this in commit 3fbbb9a640 on Dec 8, 2020
  237. MarcoFalke referenced this in commit e98d1d6740 on Dec 8, 2020
  238. fanquake referenced this in commit 736eb4d808 on Dec 11, 2020
  239. sidhujag referenced this in commit 9d8bc3e82d on Dec 11, 2020
  240. Fabcien referenced this in commit a8ce4f8ed8 on Feb 9, 2021
  241. ryanofsky referenced this in commit 9e1abdbdfb on Apr 11, 2021
  242. ryanofsky referenced this in commit 59cd6f8e6a on Apr 11, 2021
  243. ryanofsky referenced this in commit 08d2529787 on May 19, 2021
  244. ryanofsky referenced this in commit 9d7c3881f3 on May 19, 2021
  245. ryanofsky referenced this in commit 53e94024b8 on Jun 14, 2021
  246. ryanofsky referenced this in commit 96c2246bdf on Jun 14, 2021
  247. ryanofsky referenced this in commit 554c3edef8 on Jun 14, 2021
  248. ryanofsky referenced this in commit 90bf033d71 on Jun 15, 2021
  249. ryanofsky referenced this in commit 7bbec573f6 on Aug 19, 2021
  250. ryanofsky referenced this in commit 2ef0b64f42 on Aug 19, 2021
  251. ryanofsky referenced this in commit 8c4d9fe797 on Sep 3, 2021
  252. ryanofsky referenced this in commit 128132b0a8 on Sep 3, 2021
  253. ryanofsky referenced this in commit bab7a0a52e on Oct 5, 2021
  254. ryanofsky referenced this in commit 0d5d3370e0 on Oct 5, 2021
  255. ryanofsky referenced this in commit 11c22a6cbb on Oct 13, 2021
  256. ryanofsky referenced this in commit 9facba601c on Oct 13, 2021
  257. ryanofsky referenced this in commit 9273bdeee1 on Nov 29, 2021
  258. ryanofsky referenced this in commit d6b8f93fb7 on Nov 29, 2021
  259. ryanofsky referenced this in commit 6245cdadc9 on Nov 29, 2021
  260. DrahtBot locked this on Feb 15, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-01 10:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me