test: Fee estimation functional test cleanups #23075

pull darosior wants to merge 7 commits into bitcoin:master from darosior:fee_est_test_cleanup changing 1 files +88 −112
  1. darosior commented at 2:42 pm on September 23, 2021: member

    Some cleanups that i noticed would be desirable while working on #23074 and #22539, which are intentionally not based on it. Mainly simplifications and a slight speedup.

    • Use a single tx to create the 2**9 coins instead of creating 2**8 2-outputs transactions
    • Use a single P2SH script
    • Avoid the use of non-standard transactions
    • Misc style nits (happy to take more)
  2. laanwj added the label Tests on Sep 23, 2021
  3. DrahtBot commented at 10:13 pm on September 23, 2021: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #13990 (Allow fee estimation to work with lower fees by ajtowns)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  4. theStack commented at 10:33 pm on October 4, 2021: member

    Concept ACK

    After #22539 is merged, I’d suggest to also replace bare asserts with assert_equal()/assert_greater_than() helpers in the newly introduced subtest sanity_check_rbf_estimates (plus other fixups proposed in #22539).

  5. DrahtBot added the label Needs rebase on Oct 7, 2021
  6. darosior force-pushed on Oct 15, 2021
  7. darosior commented at 10:56 am on October 15, 2021: member
    Rebased and addressed @theStack ’s feedback from both here and #22539. But i forgot #22539 (review), will address but i think it can start to be reviewed in the meantime.
  8. DrahtBot removed the label Needs rebase on Oct 15, 2021
  9. glozow commented at 10:06 am on October 26, 2021: member
    Concept ACK
  10. DrahtBot added the label Needs rebase on Nov 9, 2021
  11. darosior force-pushed on Nov 15, 2021
  12. darosior commented at 0:09 am on November 15, 2021: member
    Rebased. Gave a look to replacing the use of the P2SH anyonecanspend by Miniwallet and it’s actually a pretty invasive refactoring of the test.
  13. DrahtBot removed the label Needs rebase on Nov 15, 2021
  14. DrahtBot added the label Needs rebase on Nov 16, 2021
  15. DariusParvin commented at 10:22 pm on December 1, 2021: contributor
    Concept ACK. In a future PR, I think the initial_split function could be moved to the MiniWallet since other tests could benefit from it too.
  16. qa: remove a redundant condition in fee estimation test
    Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
    19dd91a9be
  17. darosior force-pushed on Dec 9, 2021
  18. darosior commented at 12:27 pm on December 9, 2021: member
    Rebased.
  19. DrahtBot removed the label Needs rebase on Dec 9, 2021
  20. in test/functional/feature_fee_estimation.py:146 in 7b077e21f8 outdated
    139@@ -158,8 +140,10 @@ def send_tx(node, utxo, feerate):
    140     fee = tx_size * feerate
    141 
    142     tx = CTransaction()
    143-    tx.vin = [CTxIn(COutPoint(int(utxo["txid"], 16), utxo["vout"]), SCRIPT_SIG[utxo["vout"]])]
    144-    tx.vout = [CTxOut(int(utxo["amount"] * COIN) - fee, P2SH_1)]
    145+    tx.vin = [
    146+        CTxIn(COutPoint(int(utxo["txid"], 16), utxo["vout"]), REDEEM_SCRIPT)
    147+    ]
    148+    tx.vout = [CTxOut(int(utxo["amount"] * COIN) - fee, P2SH)]
    


    glozow commented at 1:13 pm on December 9, 2021:
    This change is in “split coins in a single tx in fee estimation test” but should be in “use a single p2sh script in fee estimation test” instead, yes?
  21. in test/functional/feature_fee_estimation.py:141 in 7ccd775dcc outdated
    142-    tx.vout = [CTxOut(int(utxo["amount"] * COIN) - fee, P2SH)]
    143-    txid = node.sendrawtransaction(tx.serialize().hex())
    144+    tx.vout = [CTxOut(int(utxo["amount"] * COIN), P2SH)]
    145+
    146+    # vbytes == bytes as we are using legacy transactions
    147+    fee = len(tx.serialize()) * feerate
    


    glozow commented at 1:24 pm on December 9, 2021:

    in 7ccd775dcc5f363c9f35409e9170b1e07bfdc753:

    Why not just use tx.get_vsize() ?

  22. glozow commented at 1:38 pm on December 9, 2021: member
    overall looks good, nice cleanup. I think there’s a commit diff reorg needed - feature_fee_estimation.py doesn’t pass in 9d6a88e7252ab64370c7391348c3f452003da97e because it doesn’t fully remove instances of SCRIPT_SIG
  23. glozow commented at 1:39 pm on December 9, 2021: member
    Also seems like you tried to co-author @theStack in 7ccd775dcc5f363c9f35409e9170b1e07bfdc753, but it needs to be on the 1st line maybe?
  24. fanquake commented at 1:41 pm on December 9, 2021: member

    Also seems like you tried to co-author

    The issue is that Co-Atuhored-By: is spelt wrong.

  25. qa: use a single p2sh script in fee estimation test
    Using 2 different scripts is unnecessary complication
    
    Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
    cc204b8be7
  26. qa: split coins in a single tx in fee estimation test
    This simplifies the code, and slightly speeds up the test.
    
    Running `./test/functional/test_runner.py -j15 $(printf 'feature_fee_estimation %.0s' {1..15})`
    on master 3 times gives:
    
    - Before:
        ALL                       | ✓ Passed  | 788 s (accumulated)
        ALL                       | ✓ Passed  | 818 s (accumulated)
        ALL                       | ✓ Passed  | 873 s (accumulated)
    
    - After:
        ALL                       | ✓ Passed  | 763 s (accumulated)
        ALL                       | ✓ Passed  | 798 s (accumulated)
        ALL                       | ✓ Passed  | 731 s (accumulated)
    
    Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
    1fc03155e5
  27. qa: pass scriptsig directly to txins constructor in fee estimation test
    Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
    eae52dd6ab
  28. qa: don't mine non standard txs in fee estimation test
    We don't need dust outputs anymore.
    
    Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
    15f5fd62af
  29. qa: fee estimation with RBF test cleanups
    Followups to #22539
    
    Co-Authored-By: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
    e50213967b
  30. qa: replace assert with test framework assertion helpers in fee estimation test 60ae1161a4
  31. darosior force-pushed on Dec 9, 2021
  32. darosior commented at 2:36 pm on December 9, 2021: member
    Thanks you both! Done.
  33. in test/functional/feature_fee_estimation.py:227 in 60ae1161a4
    228+        self.confutxo = [
    229+            {"txid": txid, "vout": i, "amount": splitted_amount}
    230+            for i in range(utxo_count)
    231+        ]
    232         while len(node.getrawmempool()) > 0:
    233             self.generate(node, 1, sync_fun=self.no_op)
    


    pg156 commented at 2:44 pm on December 30, 2021:

    Is the node “state” here expected to exactly the same before and after changes in https://github.com/bitcoin/bitcoin/pull/23075/commits/1fc03155e53f4f0a7f0a2529e55e802f49260b23? If so, how to test?

    E.g. this is what I tried to compare:

    • before

    node.getbalance() returns Decimal('1200.00000000') len(node.listunspent()) returns 24

    • after

    node.getbalance() returns Decimal('1150.00000000') len(node.listunspent()) returns 23


    darosior commented at 12:53 pm on January 6, 2022:

    Is the node “state” here expected to exactly the same before and after changes in 1fc0315?

    It’s not. At least, it was not my intention.

  34. in test/functional/feature_fee_estimation.py:141 in 60ae1161a4
    148+    tx.vout = [CTxOut(int(utxo["amount"] * COIN), P2SH)]
    149+
    150+    # vbytes == bytes as we are using legacy transactions
    151+    fee = tx.get_vsize() * feerate
    152+    tx.vout[0].nValue -= fee
    153 
    


    pg156 commented at 3:15 pm on December 30, 2021:
    @darosior said “since we use legacy txs in this test it’s indeed much cleaner” in #22539 (review). Does this mean for non-legacy txs, we can no longer use the same logic fee = tx.get_vsize() * feerate to calculate fee?

    darosior commented at 12:55 pm on January 6, 2022:
    #22539 (review) was refering to using the length of the serialized transaction. This works only for legacy transactions as there is no witness data discount (hence 1 byte == 1 vbyte).
  35. pg156 commented at 3:30 pm on December 30, 2021: none

    Thank you for the pr. As a newcomer, I find it very educational to review.

    I ACK all commits up to https://github.com/bitcoin/bitcoin/pull/23075/commits/60ae1161a4c415cc73f47df95598f3688e8d34df (except https://github.com/bitcoin/bitcoin/pull/23075/commits/1fc03155e53f4f0a7f0a2529e55e802f49260b23, where I have a question more for my own learning than actually questioning the PR). I built and ran the test successfully. I agree after the changes, the behavior is kept the same and the code is shorter and easier to reason.

  36. in test/functional/feature_fee_estimation.py:154 in 15f5fd62af outdated
    145@@ -146,12 +146,11 @@ def send_tx(node, utxo, feerate):
    146 class EstimateFeeTest(BitcoinTestFramework):
    147     def set_test_params(self):
    148         self.num_nodes = 3
    149-        # mine non-standard txs (e.g. txs with "dust" outputs)
    150         # Force fSendTrickle to true (via whitelist.noban)
    151         self.extra_args = [
    152-            ["-acceptnonstdtxn", "-whitelist=noban@127.0.0.1"],
    153-            ["-acceptnonstdtxn", "-whitelist=noban@127.0.0.1", "-blockmaxweight=68000"],
    154-            ["-acceptnonstdtxn", "-whitelist=noban@127.0.0.1", "-blockmaxweight=32000"],
    


    glozow commented at 10:24 am on January 4, 2022:
    yus :rocket: big fan of removing -acceptnonstdtxn
  37. glozow commented at 11:07 am on January 4, 2022: member

    utACK 60ae1161a4c415cc73f47df95598f3688e8d34df

    Light code re-review, commits all pass now. Thanks for addressing comments, sorry for the delay!

  38. MarcoFalke renamed this:
    refactoring: Fee estimation functional test cleanups
    test: Fee estimation functional test cleanups
    on Jan 6, 2022
  39. MarcoFalke merged this on Jan 6, 2022
  40. MarcoFalke closed this on Jan 6, 2022

  41. in test/functional/feature_fee_estimation.py:40 in 60ae1161a4
    39-REDEEM_SCRIPT_2 = CScript([OP_2, OP_DROP])
    40-P2SH_1 = script_to_p2sh_script(REDEEM_SCRIPT_1)
    41-P2SH_2 = script_to_p2sh_script(REDEEM_SCRIPT_2)
    42+SCRIPT = CScript([OP_1, OP_DROP])
    43+P2SH = script_to_p2sh_script(SCRIPT)
    44+REDEEM_SCRIPT = CScript([OP_TRUE, SCRIPT])
    


    MarcoFalke commented at 12:31 pm on January 6, 2022:

    I think the previous wording was correct. Redeem script refers to the thing that is hashed (serialized), see also BIP 16:

    serialized script - also referred to as the redeemScript

  42. MarcoFalke commented at 12:31 pm on January 6, 2022: member
    LGTM
  43. sidhujag referenced this in commit c30e11fc27 on Jan 6, 2022
  44. DrahtBot locked this on Jan 6, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-05 19:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me