[Tests] Require exact match in assert_start_raises_init_eror (jnewbery) #12718

1. 
   MarcoFalke commented at 5:03 pm on March 18, 2018: member

   Extracted from #12379, because the changes are important on their own.

   This allows for exact testing, since the match can be specified with a strict regex. Internal details (such as exact formatting of the error message) can still be fuzzed away by regex wildcards.

2. [Tests] Move assert_start_raises_init_error method to TestNode 0ec08a672d
3. MarcoFalke added the label Tests on Mar 18, 2018
4. [Tests] Require exact match in assert_start_raises_init_eror() 58122736b5
5. MarcoFalke force-pushed on Mar 18, 2018
6. MarcoFalke force-pushed on Mar 18, 2018
7. 
   laanwj commented at 8:31 am on March 19, 2018: member

   I did a similar thing in #12302 initially and no one agreed :/
8. 
   MarcoFalke commented at 4:28 pm on March 19, 2018: member

   Hmm, I didn’t notice the initial version of #12302, otherwise it would have been a Concept ACK from me
9. 
   laanwj commented at 4:58 pm on March 19, 2018: member

   Yeah doesn’t matter, utACK
10. MarcoFalke requested review from jnewbery on Mar 19, 2018
11. 
    jnewbery commented at 6:54 pm on March 19, 2018: member

    Tested ACK 58122736b53390a2013630e95ff760800af160e7

    Should be tested on other platforms in case stderr is different. See #12379 (comment) for example

    I did a similar thing in #12302 initially and no one agreed :/

    I think everyone agreed, but I argued that we should do the minimal fix required to get the v0.16 rc building and passing the tests.

12. 
    MarcoFalke commented at 6:56 pm on March 19, 2018: member

    @Sjors Do they still fail for you?
13. qa: Allow for partial_match when checking init error

    This allows the tests to pass on different platforms

    fae137454a
14. in test/functional/test_framework/test_node.py:193 in fad4973241 outdated

    187@@ -188,8 +188,10 @@ def assert_start_raises_init_error(self, extra_args=None, expected_msg=None, *ar
    188                 if expected_msg is not None:
    189                     log_stderr.seek(0)
    190                     stderr = log_stderr.read().decode('utf-8')
    191-                    if re.fullmatch(expected_msg + '\n', stderr) is None:
    192-                        raise AssertionError('Expected message "{}" does not match stderr:\n"{}"'.format(expected_msg, stderr))
    193+                    if partial_match and re.search(expected_msg, stderr) is None:
    194+                        raise AssertionError('Expected message "{}" does not parially match stderr:\n"{}"'.format(expected_msg, stderr))
    195+                    elif re.fullmatch(expected_msg + '\n', stderr) is None:
    

    ---

    ---

    

    jnewbery commented at 7:41 pm on March 19, 2018:

    This is wrong. Should be elif not partial_match and re.fullmatch(...)

    or nest the whole thing one level deeper:

    0if partial match:
    1    if re.search(...) is None:
    2        #assert
    3else:
    4    if re.fullmatch(...) is None:
    5        #assert
    

15. MarcoFalke force-pushed on Mar 19, 2018
16. 
    MarcoFalke commented at 8:01 pm on March 19, 2018: member

    Added a commit to make the tests pass on windows
17. 
    jnewbery commented at 8:06 pm on March 19, 2018: member

    Tested ACK fae137454adc50a4e1d448ed7219a8f5344486c9. Works for me.
18. 
    Sjors commented at 3:00 pm on March 21, 2018: member

    @MarcoFalke on MacOS 10.13.3. wallet_encryption.py still fails with a timeout. wallet_multiwallet.py and wallet_multiwallet.py --usecli now pass.

    Running the full suite now; will update this comment if something new fails.

19. 
    laanwj commented at 9:30 am on March 22, 2018: member

    Tested ACK fae1374

    @MarcoFalke on MacOS 10.13.3. wallet_encryption.py still fails with a timeout.

    Huh - we don’t use any remotely complex regexes, so I wonder how this can cause timeout.

20. laanwj merged this on Mar 22, 2018
21. laanwj closed this on Mar 22, 2018

    ---

22. laanwj referenced this in commit 185d48473e on Mar 22, 2018
23. MarcoFalke deleted the branch on Mar 22, 2018
24. PastaPastaPasta referenced this in commit 62ece1f1ff on Sep 27, 2020
25. PastaPastaPasta referenced this in commit 7a880f29df on Oct 22, 2020
26. random-zebra referenced this in commit 3f94147b3a on Aug 1, 2021
27. random-zebra referenced this in commit 61a098a775 on Aug 5, 2021
28. DrahtBot locked this on Sep 8, 2021

Contributors
MarcoFalke laanwj jnewbery Sjors

Labels
Tests