Test Framework - test_framework.test_node.FailedToStartError: No RPC credentials #30818

issue somethingbeta openend this issue on September 4, 2024
  1. somethingbeta commented at 6:31 pm on September 4, 2024: none

    Is there an existing issue for this?

    • I have searched the existing issues

    Current behaviour

    On execution bitcoind fails to start and logs Failed to start error. Have pasted logs into the logs section.

    It looks as though there is some rpc config required for this to run, but have not found a way to specify whats required. When i hard code a username/password in utils.py .. fails with Connection Refused.

    Example Command python functional/feature_rbf.py --loglevel=DEBUG --tracerpc

    Config.ini [environment] PACKAGE_NAME=Bitcoin Core PACKAGE_BUGREPORT=https://github.com/bitcoin/bitcoin/issues SRCDIR=/usr/src/bitcoin BUILDDIR=/usr/src/bitcoin EXEEXT=.exe RPCAUTH=/usr/src/bitcoin/share/rpcauth/rpcauth.py

    [components] ENABLE_WALLET=true USE_SQLITE=true USE_BDB=true ENABLE_CLI=true ENABLE_BITCOIN_UTIL=true ENABLE_WALLET_TOOL=true ENABLE_BITCOIND=true ENABLE_FUZZ_BINARY=true ENABLE_ZMQ=true #ENABLE_EXTERNAL_SIGNER=true #ENABLE_USDT_TRACEPOINTS=true `

    Bitcoin.conf (autogenerated) `

    regtest=1 [regtest] port=12201 rpcport=17201 rpcservertimeout=99000 rpcdoccheck=1 fallbackfee=0.0002 server=1 keypool=1 discover=0 dnsseed=0 fixedseeds=0 listenonion=0 peertimeout=999999999 printtoconsole=0 upnp=0 natpmp=0 shrinkdebugfile=0 deprecatedrpc=create_bdb unsafesqlitesync=1 connect=0 bind=127.0.0.1 `

    Expected behaviour

    The test should run to completion.

    Steps to reproduce

    Build 27.* on Ubuntu WLS

    Execute python test with example command. python functional/feature_rbf.py –loglevel=DEBUG –tracerpc

    Relevant log output

    ` 2024-09-04T18:07:10.779000Z TestFramework (INFO): PRNG seed is: 4851958720779390906 2024-09-04T18:07:10.779000Z TestFramework (DEBUG): Setting up network thread 2024-09-04T18:07:10.780000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_u9zpt_2g 2024-09-04T18:07:10.780000Z TestFramework (DEBUG): Copy cache directory /usr/src/bitcoin/test/cache/node0 to node 0 2024-09-04T18:07:10.781000Z TestFramework (DEBUG): Copy cache directory /usr/src/bitcoin/test/cache/node0 to node 1 2024-09-04T18:07:10.783000Z TestFramework.node0 (DEBUG): [’/usr/src/bitcoin/src/bitcoind.exe’, ‘-datadir=/tmp/bitcoin_func_test_u9zpt_2g/node0’, ‘-logtimemicros’, ‘-debug’, ‘-debugexclude=libevent’, ‘-debugexclude=leveldb’, ‘-debugexclude=rand’, ‘-uacomment=testnode0’, ‘-logthreadnames’, ‘-logsourcelocations’, ‘-loglevel=trace’, ‘-v2transport=0’] 2024-09-04T18:07:10.783000Z TestFramework.node0 (DEBUG): [’-maxorphantx=1000’, ‘-limitancestorcount=50’, ‘-limitancestorsize=101’, ‘-limitdescendantcount=200’, ‘-limitdescendantsize=101’] 2024-09-04T18:07:10.783000Z TestFramework.node0 (DEBUG): bitcoind started, waiting for RPC to come up 2024-09-04T18:07:10.784000Z TestFramework.node1 (DEBUG): [’/usr/src/bitcoin/src/bitcoind.exe’, ‘-datadir=/tmp/bitcoin_func_test_u9zpt_2g/node1’, ‘-logtimemicros’, ‘-debug’, ‘-debugexclude=libevent’, ‘-debugexclude=leveldb’, ‘-debugexclude=rand’, ‘-uacomment=testnode1’, ‘-logthreadnames’, ‘-logsourcelocations’, ‘-loglevel=trace’, ‘-v2transport=0’] 2024-09-04T18:07:10.784000Z TestFramework.node1 (DEBUG): [] 2024-09-04T18:07:10.784000Z TestFramework.node1 (DEBUG): bitcoind started, waiting for RPC to come up 2024-09-04T18:07:10.784000Z TestFramework.utils (DEBUG): /tmp/bitcoin_func_test_u9zpt_2g/node0/bitcoin.conf 2024-09-04T18:07:10.784000Z TestFramework.node0 (DEBUG): Value Error No RPC credentials …. 2024-09-04T18:07:14.552000Z TestFramework.utils (DEBUG): /tmp/bitcoin_func_test_u9zpt_2g/node0/bitcoin.conf 2024-09-04T18:07:14.552000Z TestFramework.node0 (DEBUG): Value Error No RPC credentials 2024-09-04T18:07:14.803000Z TestFramework.node0 (DEBUG): Stopping node 2024-09-04T18:07:14.804000Z TestFramework (ERROR): Assertion failed Traceback (most recent call last): File “/usr/src/bitcoin/test/functional/test_framework/test_framework.py”, line 556, in start_nodes node.wait_for_rpc_connection() File “/usr/src/bitcoin/test/functional/test_framework/test_node.py”, line 260, in wait_for_rpc_connection raise FailedToStartError(self._node_msg( test_framework.test_node.FailedToStartError: [node 0] bitcoind exited with status 1 during initialization. Error: Specified data directory “/tmp/bitcoin_func_test_u9zpt_2g/node0” does not exist.

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File “/usr/src/bitcoin/test/functional/test_framework/test_framework.py”, line 130, in main self.setup() File “/usr/src/bitcoin/test/functional/test_framework/test_framework.py”, line 297, in setup self.setup_network() File “/usr/src/bitcoin/test/functional/test_framework/test_framework.py”, line 391, in setup_network self.setup_nodes() File “/usr/src/bitcoin/test/functional/test_framework/test_framework.py”, line 413, in setup_nodes self.start_nodes() File “/usr/src/bitcoin/test/functional/test_framework/test_framework.py”, line 559, in start_nodes self.stop_nodes() File “/usr/src/bitcoin/test/functional/test_framework/test_framework.py”, line 574, in stop_nodes node.stop_node(wait=wait, wait_until_stopped=False) File “/usr/src/bitcoin/test/functional/test_framework/test_node.py”, line 386, in stop_node self.stop(wait=wait) File “/usr/src/bitcoin/test/functional/test_framework/test_node.py”, line 207, in getattr assert self.rpc_connected and self.rpc is not None, self._node_msg(“Error: no RPC connection”) AssertionError: [node 0] Error: no RPC connection

    2024-09-04T18:07:14.806000Z TestFramework (DEBUG): Closing down network thread 2024-09-04T18:07:14.857000Z TestFramework (INFO): Stopping nodes 2024-09-04T18:07:14.857000Z TestFramework.node0 (DEBUG): Stopping node Traceback (most recent call last): File “/usr/src/bitcoin/test/functional/feature_rbf.py”, line 731, in ReplaceByFeeTest().main() File “/usr/src/bitcoin/test/functional/test_framework/test_framework.py”, line 154, in main exit_code = self.shutdown() File “/usr/src/bitcoin/test/functional/test_framework/test_framework.py”, line 313, in shutdown self.stop_nodes() File “/usr/src/bitcoin/test/functional/test_framework/test_framework.py”, line 574, in stop_nodes node.stop_node(wait=wait, wait_until_stopped=False) File “/usr/src/bitcoin/test/functional/test_framework/test_node.py”, line 386, in stop_node self.stop(wait=wait) File “/usr/src/bitcoin/test/functional/test_framework/test_node.py”, line 207, in getattr assert self.rpc_connected and self.rpc is not None, self._node_msg(“Error: no RPC connection”)

    AssertionError: [node 0] Error: no RPC connection

    [node 1] Cleaning up leftover process [node 0] Cleaning up leftover process `

    How did you obtain Bitcoin Core

    Compiled from source

    What version of Bitcoin Core are you using?

    v27.1.0

    Operating system and version

    Ubuntu 22.04.4 LTS

    Machine specifications

    Windows 11 Home 8 Core i7 9700 32 GB Ram

  2. maflcko added the label Windows on Sep 5, 2024
  3. maflcko added the label Linux/Unix on Sep 5, 2024
  4. maflcko added the label Build system on Sep 5, 2024
  5. maflcko added the label Questions and Help on Sep 5, 2024
  6. maflcko commented at 8:07 am on September 5, 2024: member

    Error: Specified data directory “/tmp/bitcoin_func_test_u9zpt_2g/node0” does not exist.

    This seems to be the first error, so it would be good to debug this first. Can you check whether /usr/src/bitcoin/test/cache/node0 exists and whether the following is successful:

    2024-09-04T18:07:10.780000Z TestFramework (DEBUG): Copy cache directory /usr/src/bitcoin/test/cache/node0 to node 0

    Given that you claim that bitcoin.conf exists (presumably in this dir), the error message that the directory does not exist seems odd.

    Generally, it may be a bit difficult to get the Windows functional tests run on Linux, which is probably also the reason why the corresponding CI task only runs the Windows unit tests on Linux: https://github.com/bitcoin/bitcoin/runs/29691284283

  7. maflcko commented at 8:09 am on September 5, 2024: member

    Even if you get it to run, Wine will print errors to stderr, which the functional test framework does not like, so they will have to be silenced one way or another.

     0# test/functional/feature_rbf.py
 12024-09-05T08:02:41.987000Z TestFramework (INFO): PRNG seed is: 7350764182512023160
 22024-09-05T08:02:41.988000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_bdledk26
 32024-09-05T08:02:43.869000Z TestFramework (INFO): Running test simple doublespend...
 42024-09-05T08:02:43.889000Z TestFramework (INFO): Running test doublespend chain...
 52024-09-05T08:02:45.400000Z TestFramework (INFO): Running test doublespend tree...
 62024-09-05T08:02:50.121000Z TestFramework (INFO): Running test replacement feeperkb...
 72024-09-05T08:02:51.283000Z TestFramework (INFO): Running test spends of conflicting outputs...
 82024-09-05T08:02:51.397000Z TestFramework (INFO): Running test new unconfirmed inputs...
 92024-09-05T08:02:51.464000Z TestFramework (INFO): Running test too many replacements...
102024-09-05T08:02:52.743000Z TestFramework (INFO): Running test too many replacements using default mempool params...
112024-09-05T08:02:57.833000Z TestFramework (INFO): Running test opt-in...
122024-09-05T08:02:57.984000Z TestFramework (INFO): Running test RPC...
132024-09-05T08:02:58.291000Z TestFramework (INFO): Running test prioritised transactions...
142024-09-05T08:02:58.393000Z TestFramework (INFO): Running test no inherited signaling...
152024-09-05T08:02:58.427000Z TestFramework (INFO): Check that the child tx can still be replaced (via a tx that also replaces the parent)
162024-09-05T08:02:58.446000Z TestFramework (INFO): Running test replacement relay fee...
172024-09-05T08:02:58.461000Z TestFramework (INFO): Running test full replace by fee...
182024-09-05T08:02:58.704000Z TestFramework (ERROR): Assertion failed
19Traceback (most recent call last):
20  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/test_framework.py", line 132, in main
21    self.run_test()
22  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/feature_rbf.py", line 88, in run_test
23    self.test_fullrbf()
24  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/feature_rbf.py", line 707, in test_fullrbf
25    self.restart_node(0, extra_args=["-mempoolfullrbf=1"])
26  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/test_framework.py", line 595, in restart_node
27    self.stop_node(i)
28  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/test_framework.py", line 581, in stop_node
29    self.nodes[i].stop_node(expected_stderr, wait=wait)
30  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/test_node.py", line 409, in stop_node
31    self.wait_until_stopped(expected_stderr=expected_stderr)
32  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/test_node.py", line 444, in wait_until_stopped
33    self.wait_until(lambda: self.is_node_stopped(**kwargs), timeout=timeout)
34  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/test_node.py", line 842, in wait_until
35    return wait_until_helper_internal(test_function, timeout=timeout, timeout_factor=self.timeout_factor)
36           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
37  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/util.py", line 289, in wait_until_helper_internal
38    if predicate():
39       ^^^^^^^^^^^
40  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/test_node.py", line 444, in <lambda>
41    self.wait_until(lambda: self.is_node_stopped(**kwargs), timeout=timeout)
42                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
43  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/test_node.py", line 429, in is_node_stopped
44    raise AssertionError("Unexpected stderr {} != {}".format(stderr, expected_stderr))
45AssertionError: Unexpected stderr 004c:err:winediag:nodrv_CreateWindow Application tried to create a window, but no driver could be loaded.
46004c:err:winediag:nodrv_CreateWindow L"The explorer process failed to start."
47004c:err:systray:initialize_systray Could not create tray window != 
482024-09-05T08:02:58.756000Z TestFramework (INFO): Stopping nodes
49Traceback (most recent call last):
50  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/feature_rbf.py", line 733, in <module>
51    ReplaceByFeeTest(__file__).main()
52  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/test_framework.py", line 155, in main
53    exit_code = self.shutdown()
54                ^^^^^^^^^^^^^^^
55  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/test_framework.py", line 318, in shutdown
56    self.stop_nodes()
57  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/test_framework.py", line 587, in stop_nodes
58    node.stop_node(wait=wait, wait_until_stopped=False)
59  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/test_node.py", line 395, in stop_node
60    self.stop(wait=wait)
61  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/coverage.py", line 50, in __call__
62    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
63                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
64  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/authproxy.py", line 127, in __call__
65    response, status = self._request('POST', self.__url.path, postdata.encode('utf-8'))
66                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
67  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-w64-mingw32/test/functional/test_framework/authproxy.py", line 105, in _request
68    self.__conn.request(method, path, postdata, headers)
69  File "/usr/lib/python3.11/http/client.py", line 1282, in request
70    self._send_request(method, url, body, headers, encode_chunked)
71  File "/usr/lib/python3.11/http/client.py", line 1328, in _send_request
72    self.endheaders(body, encode_chunked=encode_chunked)
73  File "/usr/lib/python3.11/http/client.py", line 1277, in endheaders
74    self._send_output(message_body, encode_chunked=encode_chunked)
75  File "/usr/lib/python3.11/http/client.py", line 1037, in _send_output
76    self.send(msg)
77  File "/usr/lib/python3.11/http/client.py", line 975, in send
78    self.connect()
79  File "/usr/lib/python3.11/http/client.py", line 941, in connect
80    self.sock = self._create_connection(
81                ^^^^^^^^^^^^^^^^^^^^^^^^
82  File "/usr/lib/python3.11/socket.py", line 851, in create_connection
83    raise exceptions[0]
84  File "/usr/lib/python3.11/socket.py", line 836, in create_connection
85    sock.connect(sa)
86ConnectionRefusedError: [Errno 111] Connection refused
87[node 1] Cleaning up leftover process
88[node 0] Cleaning up leftover process
  8. maflcko removed the label Questions and Help on Sep 5, 2024
  9. maflcko removed the label Build system on Sep 5, 2024
  10. maflcko added the label Tests on Sep 5, 2024
  11. maflcko commented at 8:17 am on September 5, 2024: member

    Running the Windows tests on Wine in Linux is obviously better than nothing, but it would be better to run them directly on Windows. However, I am not sure what the easiest way to do this would be. For now, editing the config ini to adjust the paths is required.

  12. somethingbeta commented at 9:22 am on September 5, 2024: none

    Error: Specified data directory “/tmp/bitcoin_func_test_u9zpt_2g/node0” does not exist.

    This seems to be the first error, so it would be good to debug this first. Can you check whether /usr/src/bitcoin/test/cache/node0 exists and whether the following is successful:

    2024-09-04T18:07:10.780000Z TestFramework (DEBUG): Copy cache directory /usr/src/bitcoin/test/cache/node0 to node 0

    Given that you claim that bitcoin.conf exists (presumably in this dir), the error message that the directory does not exist seems odd.

    Generally, it may be a bit difficult to get the Windows functional tests run on Linux, which is probably also the reason why the corresponding CI task only runs the Windows unit tests on Linux: https://github.com/bitcoin/bitcoin/runs/29691284283

    yes the directory exists, for some reason only get the does not exist. The contents of the bitcoin.conf are from the one in the respective dir.

    Inlcuded the log output for the copy, that succeeds for both nodes :

    2024-09-05T09:17:31.312000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_t2z9w4gc 2024-09-05T09:17:31.312000Z TestFramework (DEBUG): Copy cache directory /usr/src/bitcoin/test/cache/node0 to node 0 2024-09-05T09:17:31.315000Z TestFramework (DEBUG): – Result of Copytree /tmp/bitcoin_func_test_t2z9w4gc/node0 2024-09-05T09:17:31.315000Z TestFramework (DEBUG): [‘stderr’, ‘bitcoin.conf’, ‘stdout’]

    2024-09-05T09:17:31.315000Z TestFramework (DEBUG): Copy cache directory /usr/src/bitcoin/test/cache/node0 to node 1 2024-09-05T09:17:31.320000Z TestFramework (DEBUG): – Result of Copytree /tmp/bitcoin_func_test_t2z9w4gc/node1 2024-09-05T09:17:31.320000Z TestFramework (DEBUG): [‘stderr’, ‘bitcoin.conf’, ‘stdout’]

  13. somethingbeta commented at 9:27 am on September 5, 2024: none

    Running the Windows tests on Wine in Linux is obviously better than nothing, but it would be better to run them directly on Windows. However, I am not sure what the easiest way to do this would be. For now, editing the config ini to adjust the paths is required.

    When you say edit the config.ini paths, what would i be editing here please?

  14. maflcko commented at 9:30 am on September 5, 2024: member

    For WSL, can you share exact steps to reproduce, please? I guess they are https://github.com/bitcoin/bitcoin/blob/28.x/doc/build-windows.md ?

    Running the Windows tests on Wine in Linux is obviously better than nothing, but it would be better to run them directly on Windows. However, I am not sure what the easiest way to do this would be. For now, editing the config ini to adjust the paths is required.

    When you say edit the config.ini paths, what would i be editing here please?

    You can try to manually edit the paths in the config ini file to point to the “source dir” and “build dir”. The source dir is the folder that contains the src cpp files (e.g. 'src', 'rpc', 'client.cpp'). The build dir is the folder that cointains the exe files (e.g. bitcoind.exe). They may be identical, depending on your build. Inside WSL this is not needed. However, outside WSL (in Windows) this should be required.

  15. somethingbeta commented at 9:40 am on September 5, 2024: none

    For WSL, can you share exact steps to reproduce, please? I guess they are https://github.com/bitcoin/bitcoin/blob/28.x/doc/build-windows.md ?

    Running the Windows tests on Wine in Linux is obviously better than nothing, but it would be better to run them directly on Windows. However, I am not sure what the easiest way to do this would be. For now, editing the config ini to adjust the paths is required.

    When you say edit the config.ini paths, what would i be editing here please?

    You can try to manually edit the paths in the config ini file to point to the “source dir” and “build dir”. The source dir is the folder that contains the src cpp files (e.g. 'src', 'rpc', 'client.cpp'). The build dir is the folder that cointains the exe files (e.g. bitcoind.exe). They may be identical, depending on your build. Inside WSL this is not needed. However, outside WSL (in Windows) this should be required.

    Used the 27.x branch ..followed the steps based on https://github.com/bitcoin/bitcoin/blob/27.x/doc/build-windows.md except the “make deploy”

    then i just navigate to the path, and invoke the test

    all this is on WSL including the invocation of the test. I don’t leave that env.

  16. maflcko commented at 9:48 am on September 5, 2024: member

    Ok, thanks for confirming.

    I don’t have WSL, so I can’t test this. However running the functional tests inside WSL was never supported, IIRC.

    Even if you work around your initial issue, the test framework and individual tests have to modified for the Wine environment.

    So my recommendation would be to run the tests on Windows directly. However that isn’t possible out-of-the-box either and requires modifying the config ini file.

  17. somethingbeta commented at 9:58 am on September 5, 2024: none

    Ok, thanks for confirming.

    I don’t have WSL, so I can’t test this. However running the functional tests inside WSL was never supported, IIRC.

    Even if you work around your initial issue, the test framework and individual tests have to modified for the Wine environment.

    So my recommendation would be to run the tests on Windows directly. However that isn’t possible out-of-the-box either and requires modifying the config ini file.

    Ok good to know re support for running the tests in WSL.

    Let me have a play getting these to run on windows directly.

  18. maflcko commented at 10:47 am on September 5, 2024: member

    I think it would be fine to add support for functional tests in WSL (or document how to do it), but I am not sure how easy it is.

    I presume adding support for tests directly on Windows is easier (or adding documentation on how to do it). However, I don’t use WSL, nor Windows, so I can’t tell for sure.


somethingbeta maflcko

Labels
Windows Linux/Unix Tests

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-09-28 22:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me