Run specified functional tests with all matching flags #23097

pull meshcollider wants to merge 1 commits into bitcoin:master from meshcollider:202109_tests_matchall changing 1 files +3 −2

meshcollider commented at 1:11 AM on September 26, 2021: contributor
Functional tests which use flags like --descriptors or --legacy-wallet won't run if only the base script is given to test_runner.py because it doesn't match any script in the list exactly. It would be easier if it would just run both options.

For example, instead of:
```
test_runner.py 'wallet_basic.py --legacy-wallet' 'wallet_basic.py --descriptors' 
```
We can now just run:
```
test_runner.py wallet_basic 
```
Also useful for --usecli, the IPv4/IPv6/nonloopback rpc_bind.py variations, etc.
Run functional tests with all possible flags b8909b0746
meshcollider added the label Tests on Sep 26, 2021
achow101 commented at 4:43 AM on September 26, 2021: member

Alternatively there's #20892 which will run both --legacy and --descriptors if neither are specified, and it works outside of test_runner.py
meshcollider commented at 5:32 AM on September 26, 2021: contributor

Thanks @achow101, wasn't aware of that PR. This is just a quick fix because its kinda inconvenient, and also works with the other test flags.

in test/functional/test_runner.py:406 in b8909b0746

 400 | @@ -401,8 +401,9 @@ def main():
 401 |          for test in tests:
 402 |              script = test.split("/")[-1]
 403 |              script = script + ".py" if ".py" not in script else script
 404 | -            if script in ALL_SCRIPTS:
 405 | -                test_list.append(script)
 406 | +            matching_scripts = [s for s in ALL_SCRIPTS if s.startswith(script)]
 407 | +            if matching_scripts:
 408 | +                test_list.extend(matching_scripts)

MarcoFalke commented at 7:59 AM on September 27, 2021:

            test_list.extend([s for s in ALL_SCRIPTS if s.startswith(script)])

meshcollider commented at 8:22 AM on September 27, 2021:

That change prevents the error that no test matches though?

MarcoFalke commented at 8:27 AM on September 27, 2021:

Right. Sorry, missed that there is an else.

MarcoFalke approved
MarcoFalke commented at 7:59 AM on September 27, 2021: member

review ACK b8909b074603a05a1a255461ca78d8a013dd3044

This also helps with --mineblock, --usecli, ...
laanwj commented at 1:24 PM on September 28, 2021: member

I think this makes sense, I also considered making this change. I see no drawbacks, you can still have the same level of specificity by giving the whole name.

#20892 can be considered separately.

Code review ACK b8909b074603a05a1a255461ca78d8a013dd3044
laanwj referenced this in commit efa227f5df on Sep 28, 2021
DrahtBot added the label Needs rebase on Sep 28, 2021
DrahtBot commented at 1:46 PM on September 28, 2021: member



🐙 This pull request conflicts with the target branch and needs rebase.

<sub>Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".</sub>
MarcoFalke closed this on Sep 28, 2021
sidhujag referenced this in commit 14751ca250 on Sep 28, 2021
meshcollider deleted the branch on Sep 30, 2021
PastaPastaPasta referenced this in commit f963695e7a on Mar 13, 2022
DrahtBot locked this on Oct 30, 2022

Contributors

Labels