get_socket_inodes() calls os.listdir() and then iterates on the results using os.readlink(). However a file may disappear from the directory after os.listdir() and before os.readlink() resulting in a FileNotFoundError exception.
It is expected that this may happen for bitcoind which is running and could open or close files or sockets at any time. Thus ignore the FileNotFoundError exception.
`get_socket_inodes()` calls `os.listdir()` and then iterates on the
results using `os.readlink()`. However a file may disappear from the
directory after `os.listdir()` and before `os.readlink()` resulting in a
`FileNotFoundError` exception.
It is expected that this may happen for `bitcoind` which is running and
could open or close files or sockets at any time. Thus ignore the
`FileNotFoundError` exception.
<!--e57a25ab6845829454e8d69fc972939a-->
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
<!--006a51241073e994b41acfe9ec718e94-->
For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31614.
<!--021abf342d371248e50ceaed478a90ca-->
See the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.
A failure like this observed in #31349 in CI job https://github.com/bitcoin/bitcoin/actions/runs/12633511389/job/35241370470?pr=31349:
2025-01-07T09:20:07.629000Z TestFramework (INFO): Bind test for ['[::1]']
2025-01-07T09:20:08.155000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
File "/home/runner/work/_temp/test/functional/test_framework/test_framework.py", line 135, in main
self.run_test()
File "/home/runner/work/_temp/ci/scratch/build-x86_64-pc-linux-gnu/test/functional/rpc_bind.py", line 99, in run_test
self._run_loopback_tests()
File "/home/runner/work/_temp/ci/scratch/build-x86_64-pc-linux-gnu/test/functional/rpc_bind.py", line 126, in _run_loopback_tests
self.run_bind_test(['[::1]'], '[::1]', ['[::1]'],
File "/home/runner/work/_temp/ci/scratch/build-x86_64-pc-linux-gnu/test/functional/rpc_bind.py", line 45, in run_bind_test
assert_equal(set(get_bind_addrs(pid)), set(expected))
^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/_temp/test/functional/test_framework/netutil.py", line 84, in get_bind_addrs
inodes = get_socket_inodes(pid)
^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/_temp/test/functional/test_framework/netutil.py", line 40, in get_socket_inodes
target = os.readlink(os.path.join(base, item))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/proc/18148/fd/19'
crACK
utACK b2e9fdc00f5c40c241a37739f7b73b74c2181e39
I looked around for a way to iterate on a cached list using os.scandir, but one doesn't seems to exist. So this makes sense to me.
ACK b2e9fdc
The issue is that the process of getting the list of files in the directory and then reading each one is not atomic. This can lead to race conditions. The proposed solution doesn’t fully fix the problem but avoids its consequences, preventing the test from failing when it shouldn’t.
Following @theuni’s idea, I tried exploring other solutions using scandir:
for item in os.scandir(base):
if item.is_symlink():
target = os.readlink(os.path.join(base, item))
if target.startswith('socket:'):
inodes.append(int(target[8:-1]))
return inodes
This approach also works and and satisfy the tests test/functional/feature_bind_extra.py where the suggested change is called, but since is_symlink and readlink do not happen atomically, the issue still exists. That said, the proposed solution is simple and effective.
ACK b2e9fdc00f5c40c241a37739f7b73b74c2181e39
Since there is no trivial platform-independent way in Python to take a snapshot of the filesystem to my knowledge, suppressing exceptions stemming from file descriptors being closed right under our noses seems like the best solution.
Modified test reported to have failed on CI to provoke the issue.
diff --git a/test/functional/rpc_bind.py b/test/functional/rpc_bind.py
index 69afd45b9a..033612a9eb 100755
--- a/test/functional/rpc_bind.py
+++ b/test/functional/rpc_bind.py
@@ -40,10 +40,14 @@ class RPCBindTest(BitcoinTestFramework):
base_args += ['-rpcallowip=' + x for x in allow_ips]
binds = ['-rpcbind='+addr for addr in addresses]
self.nodes[0].rpchost = connect_to
- self.start_node(0, base_args + binds)
- pid = self.nodes[0].process.pid
- assert_equal(set(get_bind_addrs(pid)), set(expected))
- self.stop_nodes()
+ for i in range(1, 100):
+ print(f"\nOuter {i}")
+ self.nodes[0].start(base_args + binds)
+ pid = self.nodes[0].process.pid
+ while set(get_bind_addrs(pid)) != set(expected):
+ print(".", end='')
+ self.nodes[0].wait_for_rpc_connection()
+ self.stop_nodes()
def run_invalid_bind_test(self, allow_ips, addresses):
'''
Code very often succeeds, but fails within a minute when running without the change from this PR.
<details><summary>Output sample without PR changes</summary>
â‚¿ build/test/functional/rpc_bind.py
2025-02-07T21:00:40.605000Z TestFramework (INFO): PRNG seed is: 6821738029737765731
2025-02-07T21:00:40.606000Z TestFramework (INFO): Initializing test directory /tmp/nix-shell-92894-0/nix-shell-159962-0/bitcoin_func_test_jdmurpof
2025-02-07T21:00:40.606000Z TestFramework (INFO): Check for ipv6
2025-02-07T21:00:40.606000Z TestFramework (INFO): Check for non-loopback interface
2025-02-07T21:00:40.606000Z TestFramework (INFO): Bind test for []
Outer 1
.............................
Outer 2
..............................
Outer 3
.............................
Outer 4
.............................
Outer 5
.........................
Outer 6
.............................
Outer 7
............................2025-02-07T21:00:43.678000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
File "/home/hodlinator/b2/test/functional/test_framework/test_framework.py", line 135, in main
self.run_test()
File "/home/hodlinator/b2/build/test/functional/rpc_bind.py", line 103, in run_test
self._run_loopback_tests()
File "/home/hodlinator/b2/build/test/functional/rpc_bind.py", line 124, in _run_loopback_tests
self.run_bind_test(None, '127.0.0.1', [],
File "/home/hodlinator/b2/build/test/functional/rpc_bind.py", line 47, in run_bind_test
while set(get_bind_addrs(pid)) != set(expected):
^^^^^^^^^^^^^^^^^^^
File "/home/hodlinator/b2/test/functional/test_framework/netutil.py", line 84, in get_bind_addrs
inodes = get_socket_inodes(pid)
^^^^^^^^^^^^^^^^^^^^^^
File "/home/hodlinator/b2/test/functional/test_framework/netutil.py", line 40, in get_socket_inodes
target = os.readlink(os.path.join(base, item))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/proc/290417/fd/13'
</details>
utACK b2e9fdc00f5c40c241a37739f7b73b74c2181e39
ACK b2e9fdc00f5c40c241a37739f7b73b74c2181e39