Test interface_ipc.py fails with Duplicate ID error when libmultiprocess is installed system-wide #33417

1. 
   zaidmstrr commented at 2:37 pm on September 17, 2025: contributor

   When I tried to run the interface_ipc.py functional test it failed with the error:

   02025-09-17T08:33:50.642653Z TestFramework (INFO): PRNG seed is: 5837581670543711780
   12025-09-17T08:33:50.643144Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_qg32uhlp
   2terminate called after throwing an instance of 'kj::ExceptionImpl'
   3  what():  mp/proxy.capnp:0: failed: Duplicate ID [@0xcc316e3f71a040fb](/bitcoin-bitcoin/contributor/0xcc316e3f71a040fb/).
   4Aborted (core dumped)
   

   After further investigating the issue I found that when I have the libmultiprocess library installed system wide this test is failing, and removing the library from the system will make the test run as normal. The main reason for this is the load_capnp_modules function, which imports certain files and libraries. Here is the breakdown for the issue:

   1. If you have already installed capnp globally on your system. The function will detect it and add the whole path of the /include directory of your system. Which contains all other libraries of your system including capnp and mp (if installed system-wide) library.

   2. This line of code mp_dir = src_dir / "ipc" / "libmultiprocess" / "include" includes the path of libmultiprocess directory inside the Bitcoin source tree.
   3. Now, when the capnp.load() function parses the imports, it will see multiple declarations of the same file. The first implementation was included from the /include path (as this is also the default path for the libmultiprocess library when you install system-wide) and secondly from the Bitcoin internal source tree.

   How to reproduce

   To reproduce the error you must have capnp and libmultiprocess installed system wide.

   Possible Solution

   One possible solution is to track when we have libmultiprocess installed system-wide or not. If it’s installed system-wide, then don’t add the one from the Bitcoin source tree.

2. 
   fanquake commented at 2:39 pm on September 17, 2025: member

   cc @Sjors @ryanofsky
3. 
   ryanofsky commented at 3:43 pm on September 17, 2025: contributor

   I wonder if changing order of import directories to put the local include path first would fix this:

   0-        imports = [str(capnp_dir), str(src_dir), str(mp_dir)]
   1+        imports = [str(mp_dir), str(capnp_dir), str(src_dir)]
   

   So far I’m not able to reproduce it on nixos since there’s no global include directory. But I think the problem might be the line capnp.load(str(mp_dir / "mp" / "proxy.capnp"), imports=imports) loading the local file first and then later import "/mp/proxy.capnp" lines importing the system file, causing the duplicate id error because two different files with the same id are loaded

4. ryanofsky referenced this in commit e9c52272eb on Sep 17, 2025
5. 
   ryanofsky commented at 4:41 pm on September 17, 2025: contributor

   I was able to reproduce the error by modifying the test to point at an external libmultiprocess installation and I think #33420 should fix it
6. 
   zaidmstrr commented at 4:41 pm on September 17, 2025: contributor

   I tested with your suggested change imports = [str(mp_dir), str(capnp_dir), str(src_dir)] and this fixes the issue. As now mp_dir was in the first place of the imports list so when capnp.load() parses the imports it finds proxy.capnp early, so no further lookups in the global directory were taken.
7. fanquake referenced this in commit 74fa028da1 on Sep 18, 2025
8. fanquake referenced this in commit 56fb74ee89 on Sep 18, 2025
9. zaidmstrr closed this on Sep 18, 2025

   ---

Contributors
zaidmstrr fanquake ryanofsky