wallet: don’t duplicate change output if already exist #27601

pull furszy wants to merge 8 commits into bitcoin:master from furszy:2023_wallet_double_change_output changing 8 files +188 −100
  1. furszy commented at 9:03 pm on May 8, 2023: member

    If the transaction includes an output that goes to the custom change address (CoinControl::destChange), the transaction creation process should avoid creating duplicate outputs to the same address.

    Instead, it should reuse the existing output to prevent revealing which address is the change address, which could compromise privacy.

    This will also be useful to make other wallet processes less confusing and error-prone. For instance, within the bumpfee functionality, we manually extract the change recipient from the transaction before calling ‘CreateTransaction’ to ensure that the change output is not duplicated.

    Side note: This is something that will be using for the #26732 rework.

  2. DrahtBot commented at 9:03 pm on May 8, 2023: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage

    For detailed information about the code coverage, see the test coverage report.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Concept ACK luke-jr, murchandamus
    Approach ACK S3RK

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #28453 (wallet: Receive silent payment transactions by achow101)
    • #28372 (fuzz: coinselection, improve min_viable_change/change_output_size by brunoerg)
    • #28366 (Fix waste calculation in SelectionResult by murchandamus)
    • #28246 (wallet: Use CTxDestination in CRecipient instead of just scriptPubKey by achow101)
    • #28201 (Silent Payments: sending by josibake)
    • #28122 (Silent Payments: Implement BIP352 by josibake)
    • #27827 (Silent Payments: send and receive by josibake)
    • #26152 (Bump unconfirmed ancestor transactions to target feerate by murchandamus)
    • #25273 (wallet: Pass through transaction locktime and preset input sequences and scripts to CreateTransaction by achow101)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. DrahtBot added the label Wallet on May 8, 2023
  4. DrahtBot added the label CI failed on May 8, 2023
  5. DrahtBot removed the label CI failed on May 9, 2023
  6. in src/wallet/spend.cpp:878 in cc82f6f65a outdated
    873@@ -884,6 +874,19 @@ static util::Result<CreatedTransactionResult> CreateTransactionInternal(
    874         // change keypool ran out, but change is required.
    875         CHECK_NONFATAL(IsValidDestination(dest) != scriptChange.empty());
    876     }
    877+
    878+    // Gather total out and whether SFFO is enabled or not
    


    murchandamus commented at 7:42 pm on June 14, 2023:
    0    // Gather sum of recipient output amounts and count number of outputs to subtract fees from
    

    furszy commented at 2:43 pm on June 30, 2023:
    done
  7. in src/wallet/spend.cpp:1009 in 6499c4c007 outdated
     997-            nChangePosInOut = rng_fast.randrange(txNew.vout.size() + 1);
     998-        } else if ((unsigned int)nChangePosInOut > txNew.vout.size()) {
     999-            return util::Error{_("Transaction change output index out of range")};
    1000+        if (existent_change_out_index) {
    1001+            // If there is an output with the change address already, increase that one instead of creating a new output
    1002+            txNew.vout.at(*existent_change_out_index).nValue += change_amount;
    


    murchandamus commented at 7:54 pm on June 14, 2023:
    I would expect GetChange(…) to deduct the fee for the change, but if you already have a change output, wouldn’t that mean that you account for the change fee twice? You might need to add the change_fee back in the case where you just increase an existing change output.

    furszy commented at 9:42 pm on June 29, 2023:

    It depends on how the output was crafted. If it was created manually, it will not account for the fee. And also, it will not account for it if it was created in a previous createTransaction call with SFFO enabled.

    I think that is fine to “overpay” the fee at this line because later in the process (50 lines below) we check if the final fee is above the needed one, automatically increasing the change by the surplus if needed.


    S3RK commented at 7:26 am on July 3, 2023:

    Isn’t the fee for all outputs accounted for in not_input_fees? It seems to me this will always overpay and then will be corrected when we detect surplus fee.

    Maybe a clearer way to handle this would be to pass chnage_fee=0 to GetChange if existent_change_out_index=true. I’d prefer to minimise number of cases when we need to correct fees at the later stage of transaction building.


    furszy commented at 1:49 pm on July 3, 2023:

    Isn’t the fee for all outputs accounted for in not_input_fees? It seems to me this will always overpay and then will be corrected when we detect surplus fee.

    Maybe a clearer way to handle this would be to pass chnage_fee=0 to GetChange if existent_change_out_index=true. I’d prefer to minimise number of cases when we need to correct fees at the later stage of transaction building.

    Good point. Could go further and set the coin control m_change_fee and min_viable_change to 0 so coin selection produces results closer to the target (and don’t drop any sats). And also, another improvement idea would be to set the change_cost to 0 as well. But.. that would need further modifications, so probably for a follow-up, because the waste calculation is not contemplating this scenario.


    murchandamus commented at 2:19 pm on September 12, 2023:

    No, the not_input_fees only includes the fee for the recipient outputs. The fee for the change output is added conditionally after determining whether there will be change.

    Maybe I’m misunderstanding the intention here, but it seems to me that we treat the existing change output like any other recipient. If we get here by calling RBF, wouldn’t it make sense to start a bit higher up in the call stack?

    Perhaps we should provide the inputs of the original transaction as the “preset inputs”, remove the change output altogether from the output list if we can identify it, then redo coin selection and create a new change output to the provided custom change address or a new change address if none has been set manually?

  8. in test/functional/wallet_fundrawtransaction.py:387 in 7ec42848aa outdated
    374+                assert_equal(out['scriptPubKey']['address'], change_addr)
    375+            else:
    376+                # Non-change output must remain the same
    377+                assert_equal(out['value'], outputs[external_addr])
    378+                assert_equal(out['scriptPubKey']['address'], external_addr)
    379+
    


    murchandamus commented at 7:57 pm on June 14, 2023:
    Perhaps you should add a test that checks how much fees you’ve paid for the change outputs. I suspect you might pay two change fees when increasing an existing change, unless that gets cleaned up in some fashion. A test that rules it out would be neat.

    furszy commented at 2:43 pm on June 30, 2023:

    Yeah, it gets cleaned at the end of the tx creation process (check #27601 (review)).

    But new tests never hurt, so just added it inside 8f49902. Thanks for pushing me to do it.

  9. murchandamus commented at 7:57 pm on June 14, 2023: contributor
    Concept ACK
  10. luke-jr commented at 5:01 pm on June 23, 2023: member
    Concept ACK
  11. furszy force-pushed on Jun 30, 2023
  12. furszy commented at 2:53 pm on June 30, 2023: member

    Updated per feedback, thanks @Xekyo.

    Added test coverage verifying that the same fee is paid when the wallet creates a change output automatically vs when the user provides the change output and the wallet just use it as recipient for the fee surplus.

  13. furszy force-pushed on Jun 30, 2023
  14. DrahtBot added the label CI failed on Jun 30, 2023
  15. furszy force-pushed on Jun 30, 2023
  16. furszy force-pushed on Jul 1, 2023
  17. S3RK commented at 7:29 am on July 3, 2023: contributor
    How would it work if coinselection produced result without change output?
  18. furszy commented at 2:09 pm on July 3, 2023: member

    How would it work if coinselection produced result without change output?

    There is a test exercising the behavior inside a8ac2ad8 and also an explanation there.

    But essentially, the process must adhere to the outputs specified by the user. If the change destination option in ‘fundrawtransaction’ matches one of the output scripts, the wallet is instructed to ‘increase the specified output only if there is any change’ and must not discard outputs predefined by the user under any circumstances

  19. maflcko commented at 5:38 pm on July 4, 2023: member

    From CI https://cirrus-ci.com/task/4675435070488576?logs=ci#L3365:

    0                                     File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/wallet_fundrawtransaction.py", line 337, in test_double_change
    1                                       assert_equal(wallet.gettransaction(desc_tx['vin'][0]['txid'])['amount'], Decimal(50))  # assert input value
    2                                     File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/util.py", line 57, in assert_equal
    3                                       raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
    4                                   AssertionError: not(0E-8 == 50)
    
  20. furszy force-pushed on Jul 13, 2023
  21. DrahtBot removed the label CI failed on Jul 13, 2023
  22. furszy force-pushed on Jul 14, 2023
  23. furszy commented at 2:36 pm on July 14, 2023: member

    Updated. Added required changes to make coin selection, change and fee calculations take into consideration the presence of an existing change output. Avoiding cases where we end up having to increase the change output at the end of the process by the fee surplus.

    Changes:

    • Cleaned two change related members from the CoinSelectedParams struct that are not used inside the coin selection process. They were only intermediate results used to calculate other values.
    • Excluded the change cost from the waste score when the transaction already accounts for it.
    • In transactions with an existing change output, the spending process will no longer compute all the change related coin selection params. By setting them to 0, we instruct coin selection to produce results closer to the target.
  24. in src/wallet/spend.cpp:928 in 9dc5bbe042 outdated
    924     // For creating the change output now, we use the effective feerate.
    925     // For spending the change output in the future, we use the discard feerate for now.
    926     // So cost of change = (change output size * effective feerate) + (size of spending change output * discard feerate)
    927     coin_selection_params.m_change_fee = coin_selection_params.m_effective_feerate.GetFee(coin_selection_params.change_output_size);
    928-    coin_selection_params.m_cost_of_change = coin_selection_params.m_discard_feerate.GetFee(change_spend_size) + coin_selection_params.m_change_fee;
    929+    coin_selection_params.m_cost_of_change = discard_feerate.GetFee(change_spend_size) + coin_selection_params.m_change_fee;
    


    S3RK commented at 7:15 am on July 17, 2023:

    in 9dc5bbe042f4d1febde8753a8c56da35eedd9d3e

    I know this preserves current behaviour, but don’t we want to use long_term_fee_rate to determine cost of change? @murchandamus


    murchandamus commented at 2:43 pm on September 12, 2023:
    I don’t know why the implementer of the waste metric in Bitcoin Core chose to use discard_feerate instead of LTFRE, but it does use discard_feerate throughout, so this would be consistent.
  25. in src/wallet/spend.cpp:828 in ea57562b4a outdated
    818@@ -819,6 +819,41 @@ static void DiscourageFeeSniping(CMutableTransaction& tx, FastRandomContext& rng
    819     }
    820 }
    821 
    822+static void ComputeChangeParams(CoinSelectionParams& coin_selection_params, CWallet& wallet, const CScript& scriptChange, CAmount recipients_sum, size_t recipients_size)
    


    S3RK commented at 7:22 am on July 17, 2023:

    in ea57562b4ab4071fd1a716f09e13d19ff68e99f1

    I see how it’s useful for the next commit, but the contract of the method is not really clear without reading the code.


    furszy commented at 1:25 pm on July 17, 2023:

    Guess that you are referring to the last two args?

    I think that it isn’t much of a problem because it is an internal static helper function.

    They are needed for Knapsack min change target calculation. What maybe could do is to pass them as one: mean_amount = std::floor(recipients_sum / recipients_size) and document the function properly.

    And also, in a follow-up, we could go further and move the else branch of the scriptChange block of code inside ComputeChangeParams too. That part is only needed when there isn’t an existent change output.


    S3RK commented at 6:39 am on July 18, 2023:
    Let’s document which part of CoinSelectionParams is modified and which is not.

    furszy commented at 1:11 pm on July 18, 2023:

    Let’s document which part of CoinSelectionParams is modified and which is not. @S3RK. CoinSelectionParams wasn’t modified. Only change related code was moved from CreateTransactionInternal to ComputeChangeParams.


    S3RK commented at 7:03 am on July 19, 2023:
    I mean ComputeChangeParams computes some of CoinSelectionParams, but it’s not immediately obvious which ones without reading the code of the function

    furszy commented at 1:28 pm on July 19, 2023:

    I mean ComputeChangeParams computes some of CoinSelectionParams, but it’s not immediately obvious which ones without reading the code of the function

    Ok. Added docs. The function calculates all the CoinSelectionParams fields related to ‘change’. No exceptions.

  26. S3RK commented at 7:32 am on July 17, 2023: contributor

    Approach ACK

    I think setting change related params to 0 in case where we reuse existing output is the correct way to go.

  27. furszy force-pushed on Jul 19, 2023
  28. furszy commented at 1:35 pm on July 19, 2023: member

    Updated per feedback. Thanks.

    Added docs for the ComputeChangeParams new function.

  29. achow101 commented at 7:40 pm on July 27, 2023: member

    While I agree that doing something smart like this in the context of fee bumping is fine, CreateTransaction is a generic function for the wallet’s transaction building so changing the behavior here will have an effect on other uses as well, so I don’t think this is the right approach. In general, we shouldn’t assume what the change is when it is ambiguous - conceivably a user could be reusing addresses or otherwise actually wants a particular change address that is also one of the outputs. Doing this is allowed now, and I don’t think we should change the meaning of the change address argument.

    Instead, it should reuse the existing output to prevent revealing which address is the change address, which could compromise privacy.

    No, reusing the output reveals the change since the amount changes.

  30. murchandamus commented at 7:48 pm on July 27, 2023: contributor

    Perhaps this is a good way forward: only merge the new change output and the existing output on the original transaction if either a) Bitcoin Core organically detects that the original output was the change and it is also provided as the custom change address again, or b) if the user explicitly instructs the wallet to treat the original transaction’s output as the change per #26467 and provides the same address as custom change again. Otherwise, if the output script associated with the custom change address matches an existing output on the original transaction, it should fail and request more precise instructions from the user.

    Other than that, I agree that a surveillant seeing the same output script in the original and replacement transaction but with different amounts would have a strong hint that it’s the change. Two outputs to the same output script would also be suspicious, especially if the tx fingerprint matches Bitcoin Core, since it does not permit duplicate outputs to the same output script usually.

  31. furszy commented at 9:19 pm on July 27, 2023: member

    In general, we shouldn’t assume what the change is when it is ambiguous - conceivably a user could be reusing addresses or otherwise actually wants a particular change address that is also one of the outputs. Doing this is allowed now, and I don’t think we should change the meaning of the change address argument.

    That is not really how the wallet behaves for regular outputs. Not sure why it should behave differently for change outputs only.

    Right now, the user cannot duplicate outputs in any of the wallet related RPC commands. They are all blocking dup output scripts (Check ConstructTransaction, inside AddOutputs). Only the change output is auto-magically duplicated by the wallet without the user consent.

    The only way for users to duplicate outputs destinations is to use the createrawtransaction command. Which I think that is good, the wallet RPC related commands should be “safe” to use (they shouldn’t duplicate outputs automatically nor do stuff that can leak information), and then the raw creation commands should be rules free for people to do whatever they want.

    Instead, it should reuse the existing output to prevent revealing which address is the change address, which could compromise privacy.

    No, reusing the output reveals the change since the amount changes.

    Depends on the context. If this is used to bump certain transaction, the outcome is practically the same for both. Either increasing the single change output or creating another output with the same script compromises privacy (more if we are the only software doing it).

    But, for transactions being funded (not published), it is worst to create a tx with two outputs going to the same script rather than increasing the amount of one of them.


    I think that if your concern is about allowing the user to have a second output going to the same change address, they could also do it with this as well. They just need to craft a raw transaction with two outputs going to the change address. The wallet will take one of them as change, and increase it.

    Still, will think more about this. I’m feeling that you surely have a point (other than the ones written above) but I’m not seeing it yet. It feels that it is related to a “use existing change output if exist” option instead of automatically using one of the outputs when it matches to the user provided change destination.

    But.. this last thing also sound weird. The user provided the outputs, and also set a custom change destination that matches one of them. So.. it should be more than fine to interpret it as “use this output for change”.

  32. DrahtBot added the label CI failed on Aug 27, 2023
  33. furszy force-pushed on Aug 28, 2023
  34. furszy commented at 1:38 pm on August 28, 2023: member
    Rebased due a silent merge conflict.
  35. wallet: refactor, move recipients sum loop after change script creation
    this will be used in the follow-up commit to detect whether one of the
    recipients matches the change script or not.
    e05d14e51f
  36. wallet: FundTransaction, don't add change as another recipient if exists
    If the transaction includes an output that goes to the custom change
    address (CoinControl::destChange), the transaction creation process
    should avoid creating duplicate outputs to the same address.
    
    Instead, it should reuse the existing output to prevent revealing
    which address is the change address, which could compromise privacy.
    
    This will also be useful to make other wallet processes less
    confusing and error-prone. For instance, within the bumpfee
    functionality, we are extracting the change recipient from the
    transaction before calling 'CreateTransaction' to ensure that the
    change output is not duplicated.
    a834234680
  37. test: fundrawtxn, add coverage for double change output creation
    Coverage for fundrawtxn:
    
    1) Should not create a second change output if there is
       one already going to that address.
    
    2) The wallet changeless predilection should never discard
       a manually set output by the user.
       e.g. the user manually crafted a tx that already contain a
       change output with certain value. This output shouldn't get
       lost as it was predefined by the user. Only should be used
       as recipient of the "inputs - outputs" surplus during
       'fundrawtransaction'.
    84a37f8a5b
  38. refactor: remove change_spend_size field from CoinSelectionParams
    The field is an intermediate result used to calculate other values
    and not read anywhere inside coin selection.
    2aff70073a
  39. refactor: remove discard_feerate field from CoinSelectionParams
    The field is an intermediate result used to calculate other values
    and not read anywhere inside coin selection.
    09899ff8e4
  40. coinselection: option to exclude change cost from waste score
    Useful when the transaction already accounted for the change cost.
    Which happens when the user, or wallet process, set the coin control
    custom change address equal to one of the existing output addresses.
    
    In other words, when the user selects one of the transaction existing
    outputs to be increased by the difference between inputs - output.
    
    This will be connected to the transaction creation process in the
    following-up commit.
    83e85c058a
  41. refactor: wallet spend, decouple change params initialization
    No behavior change.
    
    The initialization of all the CoinSelectionParams fields related
    to 'change' were moved to its own standalone function.
    50c5e25207
  42. wallet: compute change params only when we might produce change output
    Instructing coin selection to not account for the 'change' costs.
    
    For example:
    
    * The minimum viable change will be 0, which means that the surplus
      between inputs and outputs will never be sent to fees.
    
    * With change_fee=0:
      - The target value being sought will be the exact target (and not
         target + change_fee).
      - 'GetChange' will not deduce the change_fee from the value
         calculation.
    
    * The waste score calculation will exclude the change cost.
    349a8f0401
  43. furszy force-pushed on Aug 30, 2023
  44. DrahtBot removed the label CI failed on Aug 31, 2023
  45. achow101 commented at 6:41 pm on September 5, 2023: member

    That is not really how the wallet behaves for regular outputs. Not sure why it should behave differently for change outputs only.

    Then it should just fail instead of merging the outputs. It should tell the user that they’ve done something wrong, as we do in the RPCs. I agree with @murchandamus’s suggestion.

    The only way for users to duplicate outputs destinations is to use the createrawtransaction command.

    No, that forbids duplicates too.

    But.. this last thing also sound weird. The user provided the outputs, and also set a custom change destination that matches one of them. So.. it should be more than fine to interpret it as “use this output for change”.

    I think the issue is mainly that this is not expected typical user behavior. We don’t expect users to have an address as both a normal output and as the change address, in the same way that we don’t expect users to have the same address twice. Because this is unexpected of our users, we shouldn’t make assumptions about why they may be doing this and should instead tell them that they’ve done something unexpected.

  46. furszy commented at 3:52 pm on September 6, 2023: member

    Thanks achow for the feedback :).

    That is not really how the wallet behaves for regular outputs. Not sure why it should behave differently for change outputs only.

    Then it should just fail instead of merging the outputs. It should tell the user that they’ve done something wrong, as we do in the RPCs. I agree with @murchandamus’s suggestion.

    Ok, great. We are almost on the same page. There is one point about Murch’s suggestion that makes some noise in my head.

    Describing the general process first:

    When the user manually specifies a custom change address/script, the transaction creation and transaction funding processes are instructed to follow one of the following option:

    1. If there is an existing output matching the user provided change destination: the process will send the change amount to the existing output.
    2. If there is no output matching the user provided change destination: the process will create a new output under the provided destination and send the change amount to it.

    Murch’s suggestion:

    The only difference between the general process description and what murch suggested is the following one; he said to only perform (1) when the wallet organically detect the custom change address provided by the user as change (which makes the existing output matching this address also change). In other words, he said to only perform (1) when the wallet’s IsChange() returns true for the provided destination.

    Now, the discussion point.

    What is the point behind forbidding this action? Does it have any advantage over the current status?

    If the concerns are related to sending the change amount to an address not owned by the wallet; that is already possible today. Simply by specifying a custom change address, the user can send the change amount to any destination. This is part of the contract behind the utilization of the custom change address option.

    My perspective is that what this PR introduces isn’t radically different from our current process. The only difference arises when one of the user’s manually specified outputs matches the user’s manually specified change address. In such cases, we will increase the existing output instead of creating a second output going to the same address.

    I mean, I feel that what we are introducing here is a natural behavior rather than something unexpected. As a user, if I provide a custom change address that relates to one of the outputs that I provided, it suggests my intention to send the change to the existing output. I don’t think that anyone would expect another behavior.

    But, having that said, I agree with you that we should explicitly state this behavior on the RPC custom change address option description. So there are no doubts about it.

    Would you agree with this rationale, or do you still have second thoughts about it?

  47. S3RK commented at 6:47 am on September 7, 2023: contributor

    We don’t expect users to have an address as both a normal output and as the change address, in the same way that we don’t expect users to have the same address twice.

    I think there is a difference in these two cases.

    a) When user provided the same address twice and we show an error, then the user can easily achieve the intended goal by just summing the amounts. Having two UTXOs don’t give an advantage over a one with combined amount as spending conditions are the same anyway, so there is no point.

    b) If user provided the same address for output and change and we show an error. What the user should do? Pick a different change address? But new change address will have different spending conditions, so this is not an equivalent substitute. And even then, it’s more expensive as it forces an extra output. I could be wrong, but there is no way for the user to specify they want to send change to an existing output

  48. Sjors commented at 6:07 pm on September 7, 2023: member

    For the send RPC at least (have to think about the others) I think that if a user sets the same address in outputs and for change_address that should be considered an error. They should either remove it from outputs, ~or use the change_position argument.~

    But the above doesn’t answer what should happen if someone does createrawtransaction followed by fundrawtransaction.

  49. achow101 commented at 6:13 pm on September 7, 2023: member

    I mean, I feel that what we are introducing here is a natural behavior rather than something unexpected. As a user, if I provide a custom change address that relates to one of the outputs that I provided, it suggests my intention to send the change to the existing output. I don’t think that anyone would expect another behavior.

    I disagree. I think it’s a mistake if that happens. If the user wants to use something for change, then they shouldn’t be giving it some value first, because it’s change, not an recipient.

    On second thought, I think we should actually just always error, regardless of whether the wallet thinks the address is change.

    If user provided the same address for output and change and we show an error. What the user should do? Pick a different change address?

    Either pick a different change address, or remove the output.

    I could be wrong, but there is no way for the user to specify they want to send change to an existing output

    There isn’t, but it also doesn’t make sense to me for someone to want to do that. Why specify an existing amount instead of just dropping it and letting the software add one if it has to? Maybe coin selection finds a solution that doesn’t have change and the output doesn’t actually need to exist.

  50. S3RK commented at 6:43 am on September 11, 2023: contributor

    There isn’t, but it also doesn’t make sense to me for someone to want to do that. Why specify an existing amount instead of just dropping it and letting the software add one if it has to? Maybe coin selection finds a solution that doesn’t have change and the output doesn’t actually need to exist.

    I think this is the crux of the discussion here, and we should agree on the use-case before talking about implementation.

    Specifying one of the existing outputs as change would be useful when a user wants to pay someone and send to themselves (at least X sats) in the same tx. I can think of following examples:

    • Consolidating UTXOs and adding a low priority payment to it.
    • Moving funds between wallets and paying someone.

    What do you think? From your experience, does it sound like realistic use-case?

  51. DrahtBot added the label CI failed on Sep 11, 2023
  52. in test/functional/wallet_fundrawtransaction.py:331 in 349a8f0401
    326+        change_pos = funded_tx['changepos']
    327+        assert change_pos != -1
    328+
    329+        # Instead of creating a new change value, the wallet should have increased the existing one
    330+        desc_tx = wallet.decoderawtransaction(funded_tx['hex'])
    331+        assert_equal(wallet.gettransaction(desc_tx['vin'][0]['txid'])['amount'], Decimal(50))  # assert input value
    


    maflcko commented at 6:30 am on September 12, 2023:
     0 test  2023-09-11T08:16:14.393000Z TestFramework (ERROR): Assertion failed 
     1                                   Traceback (most recent call last):
     2                                     File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 131, in main
     3                                       self.run_test()
     4                                     File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/wallet_fundrawtransaction.py", line 127, in run_test
     5                                       self.test_double_change()
     6                                     File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/wallet_fundrawtransaction.py", line 331, in test_double_change
     7                                       assert_equal(wallet.gettransaction(desc_tx['vin'][0]['txid'])['amount'], Decimal(50))  # assert input value
     8                                     File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/util.py", line 57, in assert_equal
     9                                       raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
    10                                   AssertionError: not(-5.00000000 == 50)
    
  53. murchandamus commented at 2:05 pm on September 12, 2023: contributor

    Specifying one of the existing outputs as change would be useful when a user wants to pay someone and send to themselves (at least X sats) in the same tx. I can think of following examples:

    * Consolidating UTXOs and adding a low priority payment to it.
    
    * Moving funds between wallets and paying someone.
    

    What do you think? From your experience, does it sound like realistic use-case?

    Yeah those sound like things that I would consider doing if I were operating a high-volume wallet, e.g. in a multi-wallet setup the receive-only wallet could send change to the cold-wallet when topping up the send-only withdrawal wallet.

  54. in test/functional/wallet_fundrawtransaction.py:317 in 84a37f8a5b outdated
    308@@ -308,6 +309,82 @@ def test_change_type(self):
    309         dec_tx = self.nodes[2].decoderawtransaction(rawtx['hex'])
    310         assert_equal('witness_v0_keyhash', dec_tx['vout'][rawtx['changepos']]['scriptPubKey']['type'])
    311 
    312+    def test_double_change(self):
    313+        self.log.info("Test fundrawtxn funding a tx with an existent change output and the same custom change addr")
    314+        wallet = self.nodes[0]
    315+        wallet.syncwithvalidationinterfacequeue()  # up-to-date wallet
    316+
    317+        # 1) fundrawtxn shouldn't create a second change output if there is one already going to that address
    


    murchandamus commented at 2:33 pm on September 12, 2023:

    Regarding the commitment message and test in 84a37f8a5bc52131d341a6ae9eb2d5532a010027 (“test: fundrawtxn, add coverage for double change output creation”): If there is an output with a destination address and a manually set amount, it should not be considered change but a recipient. If there is an output that was calculated automatically and goes to the change address, it can be treated as ephemeral. Is there a way for us to reliably distinguish those two scenarios?

    Either way, if there is a recipient with an explicitly set amount and the same address is provided as a custom change destination, that sounds like a conflicting instruction to me.

  55. in src/wallet/coinselection.cpp:452 in 83e85c058a outdated
    448@@ -449,7 +449,7 @@ void OutputGroupTypeMap::Push(const OutputGroup& group, OutputType type, bool in
    449     }
    450 }
    451 
    452-CAmount GetSelectionWaste(const std::set<std::shared_ptr<COutput>>& inputs, CAmount change_cost, CAmount target, bool use_effective_value)
    453+CAmount GetSelectionWaste(const std::set<std::shared_ptr<COutput>>& inputs, std::optional<CAmount> change_cost, CAmount target, bool use_effective_value)
    


    murchandamus commented at 2:47 pm on September 12, 2023:

    The majority of 83e85c058ab7b63adcd9ead3ac80c70d64cf31eb “coinselection: option to exclude change cost from waste score” feels like a layer violation. If we don’t create change, that should be obvious from the excess being smaller than min_viable_change + change_fee. Let’s please not use change_cost as a dual-intent variable. change_cost/cost_of_change is the amount that it would cost to create and spend a change if there is one. If you want to communicate the intent of not creating change explicitly, please use a separate designated variable that only communicates that and only that.

    If we can identify the change output, we should remove it before coin selection, redo coin selection with the existing inputs as preset inputs and then create a new change output. The change output should absolutely be part of the waste calculation, since the replacement transaction at a higher feerate may have multiple possible input sets, some of which might be changeless, which then should be preferred by the scoring accordingly.

    Generally, if a custom change output address is provided, I would read that as “if there will be change, please send it to this”. If the user wants a specific output, they should add it as a recipient. If the user wants a specific output of at least some amount without creating change, they should send a higher amount and subtract the fee from that output, or use a sendall with preset inputs. So I don’t see how there should ever be a case of “the user explicitly wanted a specific change”—that would be the description of a recipient output.

  56. in src/wallet/test/coinselector_tests.cpp:1074 in 83e85c058a outdated
    1070@@ -1071,7 +1071,7 @@ BOOST_AUTO_TEST_CASE(check_max_weight)
    1071         rand,
    1072         /*change_output_size=*/34,
    1073         /*min_change_target=*/CENT,
    1074-        /*effective_feerate=*/CFeeRate(0),
    1075+        /*effective_feerate=*/CFeeRate(1), // tiny feerate so the waste score isn't always 0.
    


    murchandamus commented at 2:50 pm on September 12, 2023:

    I would like us to get away from predominantly using values that the wallet would never encounter in the normal operation in our tests. Unless we are testing edge cases, using values that are out-of-scope for the normal operation just confuses. Please use at least 1,000 for the effective feerate.

    Also see: https://github.com/bitcoin/bitcoin/issues/27754

  57. in src/wallet/spend.cpp:952 in 349a8f0401
    945@@ -946,8 +946,10 @@ static util::Result<CreatedTransactionResult> CreateTransactionInternal(
    946         return util::Error{strprintf(_("Fee estimation failed. Fallbackfee is disabled. Wait a few blocks or enable %s."), "-fallbackfee")};
    947     }
    948 
    949-    // Set the 'change' related coin selection params
    950-    ComputeChangeParams(coin_selection_params, wallet, scriptChange, recipients_sum, vecSend.size());
    951+    // Only compute 'change' params when we might produce a change output
    952+    if (!existent_change_out_index) {
    953+        ComputeChangeParams(coin_selection_params, wallet, scriptChange, recipients_sum, vecSend.size());
    954+    }
    


    murchandamus commented at 3:45 pm on September 12, 2023:

    No, please don’t do this. This is hacky and muddles meaning of variables.

    The parameters of creating change are independent of whether change is created or not. In fact, we use some of these value in some of the coinselection algorithms to determine whether we have a changeless solution as well as in transaction building to determine the size of the change output. By fiddling with these values, it feels like we are coding against the implementation of the GetChange(…) function rather than against the function definition. Please use a new designated variable to explicitly track that transaction building should not create a change output if we absolutely must have that.

    I would much prefer an approach that sidesteps this entire issue by starting the creation of an RBF transaction higher in the stack where we can remove a pre-existing identifiable change output from the draft transaction before just kicking off an entire transaction building process with some preset inputs.


    murchandamus commented at 7:48 pm on September 12, 2023:
    This is also related to #28366 (last two commits only, rest is #26152).
  58. murchandamus commented at 3:52 pm on September 12, 2023: contributor
    Concept ACK, but I dislike the approach:
  59. DrahtBot added the label Needs rebase on Sep 14, 2023
  60. DrahtBot commented at 9:22 pm on September 14, 2023: contributor

    🐙 This pull request conflicts with the target branch and needs rebase.

  61. maflcko commented at 7:19 am on October 18, 2023: member
    Maybe mark as draft while this is stale?
  62. furszy marked this as a draft on Oct 21, 2023
  63. DrahtBot commented at 10:43 am on January 5, 2024: contributor

    There hasn’t been much activity lately and the patch still needs rebase. What is the status here?

    • Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
    • Is it no longer relevant? ➡️ Please close.
    • Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.
  64. maflcko commented at 8:10 am on March 11, 2024: member
    Are you still working on this?
  65. furszy commented at 10:28 pm on March 11, 2024: member

    Are you still working on this?

    Probably next month. But no problem on closing it for now. Will re-open it or create a new one whenever I back to it.

  66. furszy closed this on Mar 11, 2024


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-06-29 07:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me