test: Prevent “Duplicate-wallet filename specified” #14919

pull AkioNak wants to merge 1 commits into bitcoin:master from AkioNak:confirm_unloadwallet_done changing 2 files +16 −8
  1. AkioNak commented at 2:36 am on December 11, 2018: contributor

    wallet_multiwallet.py --usecli may sometimes fails with “Duplicate -wallet filename specified” unloadwallet() RPC command just notify the unload intention and it may delay a little bit until actual unloading.

    This patch check the completion of unloading by using listwallets().

    This fixes #14917

  2. fanquake added the label Tests on Dec 11, 2018
  3. promag commented at 8:53 am on December 11, 2018: member
    Not sure if this is the right fix, IIRC the wallet is removed from the list first and then the actual unload happens.
  4. kallewoof commented at 11:21 am on December 11, 2018: member

    The relevant code is

    https://github.com/bitcoin/bitcoin/blob/5f23460c7e316fe7c944680f3feff07ebb867f70/src/wallet/wallet.cpp#L3879-L3883

    The wallet itself has locks to prevent it from being loaded while already loaded, so it seems like an OK fix to me.

  5. promag commented at 12:04 pm on December 11, 2018: member

    @kallewoof that’s what happening, it hits that error.

    It’s always the case that after unloadwallet X listwallets doesn’t return X but that wallet can be unloading. This change only reduces the race to the actual duplicate condition.

    I see 4 solutions:

    1. add an option to unloadwallet to force waiting 👍 👍
    2. use this change but also change listwallets to actually verify if the wallet is not unloaded 👎
    3. make loadwallet aware the wallet is unloading and wait for it 👍
    4. change the test to repeat the loadwallet until it succeeds 👍 👍 👍 👍

    Note that there are 2 wallet “registries”: https://github.com/bitcoin/bitcoin/blob/5f23460c7e316fe7c944680f3feff07ebb867f70/src/wallet/wallet.cpp#L69-L73 https://github.com/bitcoin/bitcoin/blob/5f23460c7e316fe7c944680f3feff07ebb867f70/src/wallet/db.cpp#L75-L84

  6. AkioNak commented at 2:48 pm on December 11, 2018: contributor

    @promag @kallewoof Thank you for the discussion.

    change the test to repeat the loadwallet until it succeeds 👍 👍 👍 👍

    I will try this.

  7. promag commented at 3:36 pm on December 12, 2018: member

    @AkioNak you could add an option to wait_until, untested:

     0diff --git a/test/functional/test_framework/util.py b/test/functional/test_framework/util.py
     1index d0a78d8df..4af7a51ff 100644
     2--- a/test/functional/test_framework/util.py
     3+++ b/test/functional/test_framework/util.py
     4@@ -201,20 +201,24 @@ def str_to_b64str(string):
     5 def satoshi_round(amount):
     6     return Decimal(amount).quantize(Decimal('0.00000001'), rounding=ROUND_DOWN)
     7
     8-def wait_until(predicate, *, attempts=float('inf'), timeout=float('inf'), lock=None):
     9+def wait_until(predicate, *, attempts=float('inf'), timeout=float('inf'), lock=None, ignore_expections=False):
    10     if attempts == float('inf') and timeout == float('inf'):
    11         timeout = 60
    12     attempt = 0
    13     time_end = time.time() + timeout
    14
    15     while attempt < attempts and time.time() < time_end:
    16-        if lock:
    17-            with lock:
    18+        try:
    19+            if lock:
    20+                with lock:
    21+                    if predicate():
    22+                        return
    23+            else:
    24                 if predicate():
    25                     return
    26-        else:
    27-            if predicate():
    28-                return
    29+        except JSONRPCException as e:
    30+            if not ignore_expections:
    31+                raise e
    32         attempt += 1
    33         time.sleep(0.05)
    

    And then:

    0wait_until(lambda: self.nodes[0].loadwallet(wallet_name), ignore_exceptions=True)
    

    BTW, now I’m more fond of

    make loadwallet aware the wallet is unloading and wait for it @ryanofsky what are your thoughts?

  8. ryanofsky commented at 4:42 pm on December 12, 2018: member

    make loadwallet aware the wallet is unloading and wait for it

    I think a better approach would be to make unloadwallet aware that a wallet is unloading and wait for it. It’s surprising to me that this would be an asynchronous operation.

  9. promag commented at 6:46 pm on December 12, 2018: member
    @ryanofsky the idea was to not block especially if the wallet is being used or a rescan is going on. In that case the client could timeout and then the problem would be the same — attempt to load would raise Duplicate -wallet filename specified error. But maybe we should ignore these edge cases and make it synchronous or even add an option?
  10. MarcoFalke commented at 6:54 pm on December 12, 2018: member
    Agree with @ryanofsky. Seems unnecessary to make this call async. If the wallet is in use (e.g. rescan) you should be aware of it and not call unloadwallet. So shouldn’t unloadwallet throw an error in that case?
  11. AkioNak force-pushed on Dec 13, 2018
  12. test: Prevent "Duplicate-wallet filename specified"
    wallet_multiwallet.py --usecli may sometimes fails with
    "Duplicate -wallet filename specified"
    Unloadwallet RPC command just notify the unload intention
    and it may delay a little bit until actual unloading.
    
    This patch make loadwallet to retry if JSONRPCException occurs.
    52b380b5bd
  13. AkioNak force-pushed on Dec 13, 2018
  14. laanwj commented at 1:48 pm on December 13, 2018: member
    Closing in favor of #14941
  15. laanwj closed this on Dec 13, 2018

  16. kallewoof commented at 10:23 pm on December 13, 2018: member
    @laanwj Please do not close PR:s unless there is inactivity or the author is clearly not cooperating. The authors are perfectly capable of doing this themselves.
  17. laanwj commented at 10:27 pm on December 13, 2018: member
    @kallewoof I don’t think there’s a point of having multiple competing PRs open, this spreads discussion around, reopening is always a possibility if discussion goes the other way
  18. kallewoof commented at 11:22 pm on December 13, 2018: member
    I understand your desire to have a clean list of PRs, but there really is no rush on the matter. This PR literally had pushes to it hours before you closed it. If the author judges that the replacement is objectively better they will close their PR on their own, and if they refuse, I see no problem with you making a judgement call like this one.
  19. laanwj commented at 8:58 am on December 14, 2018: member

    I’m not sure how you’re interpreting it, but for me, closing the PR is only an indication that it’s currently not considered for merging. It doesn’t delete the discussion, doesn’t even prevent replying! it doesn’t delete the code changes, etc.

    That’s a typical maintainer task. at least for me it has nothing to do with ‘urgency’ or capability or non-capability of the author to do it themselves.

    But anyhow if you insist I’ll re-open.

  20. laanwj reopened this on Dec 14, 2018

  21. kallewoof commented at 9:24 am on December 14, 2018: member
    Removing my comment as it strays off topic from the PR. I’ve sent it to @laanwj in private.
  22. promag commented at 10:01 am on December 14, 2018: member
    @kallewoof I don’t think it makes sense to change the test tree to later revert. There is already agreement to change unloadwallet and @laanwj just closed this in light of that - being closed doesn’t mean is locked and dead.
  23. laanwj commented at 10:05 am on December 14, 2018: member

    Yes, so I think there’s a misunderstanding here: Thanks @AkioNak for trying to fix the test!

    However your PR made us realize that the current unloadwallet behavior is broken in the first place and that the unload shouldn’t be asynchronous, and the test failure is pointing out an actual issue. It likely means that the test doesn’t need to change.

  24. AkioNak commented at 11:17 am on December 14, 2018: contributor
    @kallewoof @promag @ryanofsky @MarcoFalke @laanwj Thank you so much. Now I understand that we need to fix the behaviour of unloadwallet rather than changing the test. So, I close this PR myself.
  25. AkioNak closed this on Dec 14, 2018

  26. promag commented at 11:29 am on December 14, 2018: member
    Thank you @AkioNak for identifying the problem so quick.
  27. laanwj referenced this in commit 91dbba8857 on Jan 10, 2019
  28. AkioNak deleted the branch on Jan 11, 2019
  29. laanwj referenced this in commit 1b6fc30530 on Jan 15, 2019
  30. PastaPastaPasta referenced this in commit 39308f017a on Sep 8, 2021
  31. vijaydasmp referenced this in commit 70a1f2c79b on Sep 13, 2021
  32. DrahtBot locked this on Dec 16, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-17 15:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me