TestFramework: TestShell.reset() will always fail #31131

1. 
   edilmedeiros commented at 2:38 pm on October 22, 2024: contributor

   TestShell documentation mentions a TestShell.reset() method.

   When experimenting with it, it will always fail with:

   0Traceback (most recent call last):
   1  File "/Users/jose.edil/2-development/bitcoin/vinteum/foss-program/infra-signet-server/signet-server.py", line 169, in <module>
   2    TestShell().reset()
   3  File "/Users/jose.edil/2-development/bitcoin/bitcoin-core-build/test/functional/test_framework/test_shell.py", line 64, in reset
   4    super().__init__()
   5TypeError: BitcoinTestFramework.__init__() missing 1 required positional argument: 'test_file'
   

   Indeed, inspecting the source code, we see that TestShell is a wrapper class for BitcoinTestFramework. TestShell.reset() will call super().__init__() which refers to BitcoinTestFramework.__init__().

   But we only have def __init__(self, test_file) -> None: which will require a mandatory test_file parameter. (https://github.com/bitcoin/bitcoin/blob/28ce159bc327e6dfec34077ff2e379b23a95db65/test/functional/test_framework/test_framework.py#L95C5-L95C43).

   Not sure if TestShell.reset() should change to receive this parameter and pass along to the __init__() function it calls internally or any other workaround. In my specific case, I can’t see what I can pass as test_file.

2. edilmedeiros renamed this:
   [TestFramework] TestShell.reset() will always fail
   TestFramework: TestShell.reset() will always fail
   on Oct 22, 2024
3. 
   brunoerg commented at 9:40 am on October 23, 2024: contributor

   Thanks for it! I think this is similar to #30714 (which fixed the initialization). cc: @theStack
4. 
   edilmedeiros commented at 3:31 pm on October 23, 2024: contributor

   Looks similar, indeed, but I think #30714 did not covered TestShell().setup().
5. 
   theStack commented at 6:15 pm on October 23, 2024: contributor

   It seems that the TestShell instructions generally don’t work anymore since we switched to CMake. For me they fail already earlier at the TestShell instantiation, as the generated config.ini file is searched in a path that still assumes in-tree builds:

    0$ git rev-parse HEAD
    1ffe4261cb0669b1e1a926638e0498ae5b63f3599
    2$ python3
    3Python 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] on linux
    4Type "help", "copyright", "credits" or "license" for more information.
    5>>> import sys
    6>>> sys.path.insert(0, "./test/functional")
    7>>> from test_framework.test_shell import TestShell
    8>>> test = TestShell().setup(num_nodes=2, setup_clean_chain=True)
    9tests_directory = /home/fas/bitcoin/test/functional
   10Traceback (most recent call last):
   11  File "<stdin>", line 1, in <module>
   12  File "/home/fas/bitcoin/./test/functional/test_framework/test_shell.py", line 79, in __new__
   13    TestShell.instance = TestShell.__TestShell(tests_directory / "testshell_dummy.py")
   14  File "/home/fas/bitcoin/./test/functional/test_framework/test_framework.py", line 106, in __init__
   15    self.parse_args(test_file)
   16  File "/home/fas/bitcoin/./test/functional/test_framework/test_framework.py", line 210, in parse_args
   17    config.read_file(open(self.options.configfile))
   18FileNotFoundError: [Errno 2] No such file or directory: '/home/fas/bitcoin/test/config.ini'
   

   I assume that this was not noticed for some time as users probably had left-over files around from previous autotools builds (or, no-one just tried it after the switch to CMake, which was merged after the last TestShell fix #30714). A first naive try to fix this by adding the out-of-tree path “./build/test/functional” to sys.path instead of the in-tree one didn’t help either.

Contributors
edilmedeiros brunoerg theStack