test: functional test failures under `--usecli` #32264

issue fanquake opened this issue on April 14, 2025

fanquake commented at 9:36 AM on April 14, 2025: member

Ubuntu 24.04.2 LTS Python 3.12.3 master @ 817edfb21e4fa96289d69d4e1dbabb6b9ef9d8f5

cmake -B build
cmake --build build
./build/test/functional/test_runner.py --usecli

feature_assumeutxo.py                                                | ✖ Failed  | 11 s
feature_bip68_sequence.py                                            | ✖ Failed  | 58 s
feature_fastprune.py                                                 | ✖ Failed  | 1 s
feature_fee_estimation.py                                            | ✖ Failed  | 1 s
feature_nulldummy.py                                                 | ✖ Failed  | 1 s
feature_reindex_readonly.py                                          | ✖ Failed  | 0 s
feature_signet.py                                                    | ✖ Failed  | 1 s
feature_taproot.py                                                   | ✖ Failed  | 6 s
mempool_package_rbf.py                                               | ✖ Failed  | 22 s
mining_mainnet.py                                                    | ✖ Failed  | 1 s
p2p_net_deadlock.py --v1transport                                    | ✖ Failed  | 1 s
p2p_net_deadlock.py --v2transport                                    | ✖ Failed  | 1 s
p2p_tx_download.py                                                   | ✖ Failed  | 34 s
rpc_deprecated.py                                                    | ✖ Failed  | 1 s
rpc_estimatefee.py                                                   | ✖ Failed  | 1 s
rpc_generate.py                                                      | ✖ Failed  | 1 s
rpc_getblockfrompeer.py                                              | ✖ Failed  | 1 s
rpc_getdescriptorinfo.py                                             | ✖ Failed  | 1 s
rpc_invalid_address_message.py                                       | ✖ Failed  | 2 s
rpc_packages.py                                                      | ✖ Failed  | 5 s
rpc_scanblocks.py                                                    | ✖ Failed  | 1 s
rpc_scantxoutset.py                                                  | ✖ Failed  | 5 s
rpc_whitelist.py                                                     | ✖ Failed  | 1 s
wallet_bumpfee.py --descriptors                                      | ✖ Failed  | 10 s
wallet_descriptor.py --descriptors                                   | ✖ Failed  | 7 s
wallet_encryption.py --descriptors                                   | ✖ Failed  | 6 s
wallet_fundrawtransaction.py --descriptors                           | ✖ Failed  | 12 s
wallet_labels.py --descriptors                                       | ✖ Failed  | 2 s
wallet_miniscript.py --descriptors                                   | ✖ Failed  | 2 s
wallet_send.py --descriptors                                         | ✖ Failed  | 1 s

147/318 - rpc_deprecated.py failed, Duration: 1 s

stdout:
2025-04-14T09:29:49.903000Z TestFramework (INFO): PRNG seed is: 5746030027211002162
2025-04-14T09:29:49.903000Z TestFramework (INFO): Initializing test directory /tmp/test_runner_₿_🏃_20250414_092629/rpc_deprecated_169
2025-04-14T09:29:50.177000Z TestFramework (INFO): Tests for deprecated RPC methods (if any)
2025-04-14T09:29:50.178000Z TestFramework (INFO): Tests for deprecated wallet-related RPC methods (if any)
2025-04-14T09:29:50.178000Z TestFramework (INFO): Test settxfee RPC deprecation
2025-04-14T09:29:50.203000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
  File "/root/ci_scratch/test/functional/test_framework/test_framework.py", line 182, in main
    self.run_test()
  File "/root/ci_scratch/build/test/functional/rpc_deprecated.py", line 37, in run_test
    assert_raises_rpc_error(-32, 'settxfee is deprecated and will be fully removed in v31.0.', self.nodes[0].rpc.settxfee, 0.01)
                                                                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'settxfee'

131/318 - rpc_whitelist.py failed, Duration: 1 s

stdout:
2025-04-14T09:29:42.390000Z TestFramework (INFO): PRNG seed is: 3309837604605811676
2025-04-14T09:29:42.390000Z TestFramework (INFO): Initializing test directory /tmp/test_runner_₿_🏃_20250414_092629/rpc_whitelist_186
2025-04-14T09:29:43.102000Z TestFramework (INFO): [user1]: Testing a non permitted permission (getnetworkinfo)
2025-04-14T09:29:43.102000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
  File "/root/ci_scratch/test/functional/test_framework/test_framework.py", line 182, in main
    self.run_test()
  File "/root/ci_scratch/build/test/functional/rpc_whitelist.py", line 78, in run_test
    assert_equal(403, rpccall(self.nodes[0], user, permission).status)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/ci_scratch/build/test/functional/rpc_whitelist.py", line 20, in rpccall
    conn = http.client.HTTPConnection(url.hostname, url.port)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/http/client.py", line 880, in __init__
    (self.host, self.port) = self._get_hostport(host, port)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/http/client.py", line 926, in _get_hostport
    i = host.rfind(':')
        ^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'rfind'

90/318 - feature_bip68_sequence.py failed, Duration: 58 s

stdout:
2025-04-14T09:28:25.778000Z TestFramework (INFO): PRNG seed is: 2393500354933353065
2025-04-14T09:28:25.779000Z TestFramework (INFO): Initializing test directory /tmp/test_runner_₿_🏃_20250414_092629/feature_bip68_sequence_271
2025-04-14T09:28:26.365000Z TestFramework (INFO): Running test disable flag
2025-04-14T09:28:26.430000Z TestFramework (INFO): Running test sequence-lock-confirmed-inputs
2025-04-14T09:29:22.254000Z TestFramework (INFO): Running test sequence-lock-unconfirmed-inputs
2025-04-14T09:29:23.869000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/root/ci_scratch/test/functional/test_framework/test_framework.py", line 182, in main
    self.run_test()
  File "/root/ci_scratch/build/test/functional/feature_bip68_sequence.py", line 73, in run_test
    self.test_sequence_lock_unconfirmed_inputs()
  File "/root/ci_scratch/build/test/functional/feature_bip68_sequence.py", line 336, in test_sequence_lock_unconfirmed_inputs
    assert_equal(None if i == 1 else 'inconclusive', self.nodes[0].submitblock(block.serialize().hex()))
  File "/root/ci_scratch/test/functional/test_framework/util.py", line 77, in assert_equal
    raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not(None == )

maflcko commented at 9:50 AM on April 14, 2025: member

File "/root/ci_scratch/build/test/functional/rpc_deprecated.py", line 37, in run_test assert_raises_rpc_error(-32, 'settxfee is deprecated and will be fully removed in v31.0.', self.nodes[0].rpc.settxfee, 0.01) ^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'settxfee'

This one seems obvious: The rpc member field should almost never be used. Directly calling the RPC field via the getattr is better.

File "/root/ci_scratch/build/test/functional/feature_bip68_sequence.py", line 336, in test_sequence_lock_unconfirmed_inputs assert_equal(None if i == 1 else 'inconclusive', self.nodes[0].submitblock(block.serialize().hex())) File "/root/ci_scratch/test/functional/test_framework/util.py", line 77, in assert_equal raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args)) AssertionError: not(None == )

This affects a few RPCs, where json-none is an empty string for the cli, see also #18476.

For the other issues, I haven't looked, but there is self.supports_cli = False.
maflcko added the label Tests on Apr 14, 2025
mzumsande commented at 9:55 PM on April 15, 2025: contributor

I'm looking into these currently, supports_cli = False is always possible of course but maybe we can fix the source of the failure instead in many cases?!

This affects a few RPCs, where json-none is an empty string for the cli, see also #18476.

My suggestion would be to have the test framework return None back to the the test if the cli is used and returns an empty string. That fixes a handful of the failing tests.

[Edit]: #32286 (merged) fixed this problem, #32290 (open) attempts to fix the remaining ones.
achow101 closed this on Apr 16, 2025
achow101 referenced this in commit eb6b1003c1 on Apr 16, 2025
achow101 reopened this on Apr 16, 2025
fanquake closed this on Jul 3, 2025
fanquake referenced this in commit 6251949443 on Jul 3, 2025

Contributors

Labels

Linked (view graph)

#32286 test: Handle empty string returned by CLI as None in RPC tests #32290 test: allow all functional tests to be run or skipped with --usecli