rpc: preserve global xpubs and proprietary fields in joinpsbts #35516

pull thomasbuilds wants to merge 2 commits into bitcoin:master from thomasbuilds:fix-joinpsbts-drop-globals changing 2 files +44 −20
  1. thomasbuilds commented at 7:12 PM on June 11, 2026: contributor

    joinpsbts collects the global xpubs of all the joined PSBTs into merged_psbt, but returns a separately constructed shuffled_psbt into which only the inputs, outputs, and unknown fields are copied. The collected PSBT_GLOBAL_XPUB records are silently dropped, and PSBT_GLOBAL_PROPRIETARY records are not collected at all.

    The xpub collection was added in #17034, which was written against a joinpsbts that still returned merged_psbt, but was merged after #16512 had introduced the shuffled_psbt rebuild, so the collected xpubs have never reached the result.

    Shuffle the inputs and outputs of merged_psbt in place instead of rebuilding a new PSBT, so that all global data is preserved, and union the global proprietary records in the merge loop, matching the combinepsbt behavior from #34893.

  2. DrahtBot added the label RPC/REST/ZMQ on Jun 11, 2026
  3. DrahtBot commented at 7:12 PM on June 11, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35516.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    Type Reviewers
    ACK winterrdog, jpk68, achow101
    Stale ACK g-maxxx, Bicaru20

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    No conflicts as of last run.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  4. fanquake commented at 12:35 PM on June 12, 2026: member
  5. achow101 commented at 9:49 PM on June 12, 2026: member

    Concept ACK

  6. DrahtBot requested review from achow101 on Jul 4, 2026
  7. in test/functional/rpc_psbt.py:1096 in faa2500a88 outdated
    1088 | @@ -1087,6 +1089,26 @@ def test_psbt_input_keys(psbt_input, keys):
    1089 |                  break
    1090 |          assert shuffled
    1091 |  
    1092 | +        # Check that joining preserves global xpub and proprietary records
    1093 | +        def global_xpub_key(extended_pubkey):
    1094 | +            xpub_data, xpub_version = base58_to_byte(extended_pubkey)
    1095 | +            return bytes([PSBT_GLOBAL_XPUB]) + bytes([xpub_version]) + xpub_data
    1096 | +        xpub_key1 = global_xpub_key("tpubD6NzVbkrYhZ4XgiXtGrdW5XDAPFCL9h7we1vwNCpn8tGbBcgfVYjXyhWo4E1xkh56hjod1RhGjxbaTLV3X4FyWuejifB9jusQ46QzG87VKp")
    


    winterrdog commented at 6:26 PM on July 5, 2026:

    nit: i think leaving a blank line after the global_xpub_key function's return would be more consistent with the surrounding style and a bit easier to read (just like you did here in a related PR - #35665)

  8. in test/functional/rpc_psbt.py:1110 in faa2500a88 outdated
    1105 | +        psbt2_obj = PSBT.from_base64(psbt2)
    1106 | +        psbt2_obj.g.map[xpub_key2] = xpub_value
    1107 | +        joined_globals = PSBT.from_base64(self.nodes[0].joinpsbts([psbt1_obj.to_base64(), psbt2_obj.to_base64()]))
    1108 | +        assert_equal(joined_globals.g.map[xpub_key1], xpub_value)
    1109 | +        assert_equal(joined_globals.g.map[xpub_key2], xpub_value)
    1110 | +        assert_equal(joined_globals.g.map[global_prop_key], global_prop_value)
    


    winterrdog commented at 8:31 AM on July 7, 2026:

    the PR description says joinpsbts now unions m_proprietary "matching the combinepsbt behavior from #34893," which is a set insert() i.e. first PSBT wins if the same key shows up twice or more. right now, each global key in the test is set on psbt1_obj or psbt2_obj, but never both at once. that confirms nothing gets dropped when the 2 PSBTs do not overlap -- but it does not tell us what happens if both PSBTs used the same proprietary key with different values (e.g. psbt1_obj sets it to 0x11... and psbt2_obj sets it to 0x22...). that collision case, and which value should survive, is not covered by any existing test.

    this can happen in practice: joinpsbts is meant for batching independently-created PSBTs, and two PSBTs from the same wallet or software could easily carry the same proprietary key (e.g. a session ID or some internal bookkeeping field) with different values each time

    would it make sense to add a small case like this, right after the existing xpub/proprietary check ?

    diff --git a/test/functional/rpc_psbt.py b/test/functional/rpc_psbt.py
    index 54cabc5844..cb1a494ebf 100755
    --- a/test/functional/rpc_psbt.py
    +++ b/test/functional/rpc_psbt.py
    @@ -1109,6 +1109,19 @@ class PSBTTest(BitcoinTestFramework):
             assert_equal(joined_globals.g.map[xpub_key2], xpub_value)
             assert_equal(joined_globals.g.map[global_prop_key], global_prop_value)
    
    +        # same proprietary key in both PSBTs, different values: first PSBT SHOULD win
    +        collide_key = bytes([PSBT_GLOBAL_PROPRIETARY]) + b"\x02\x03\x04\x00"
    +        psbt_first_obj = PSBT.from_base64(psbt1)
    +        psbt_first_obj.g.map[collide_key] = b"\x11\x11\x11\x11"
    +
    +        psbt_second_obj = PSBT.from_base64(psbt2)
    +        psbt_second_obj.g.map[collide_key] = b"\x22\x22\x22\x22"
    +
    +        joined_collision = PSBT.from_base64(
    +            self.nodes[0].joinpsbts([psbt_first_obj.to_base64(), psbt_second_obj.to_base64()])
    +        )
    +        assert_equal(joined_collision.g.map[collide_key], b"\x11\x11\x11\x11") # first PSBT's value wins
    +
             # Newly created PSBT needs UTXOs and updating
             addr = self.nodes[1].getnewaddress("", "p2sh-segwit")
             utxo = self.create_outpoints(self.nodes[0], outputs=[{addr: 7}])[0]
    

    at core, this proposed test just asks: when 2 PSBTs disagree on the same key, which one wins?

    this is just a nice-to-have, not a blocker. what's your take on this ?


    thomasbuilds commented at 9:30 AM on July 7, 2026:

    Yes thanks

  9. thomasbuilds force-pushed on Jul 7, 2026
  10. winterrdog commented at 9:52 PM on July 10, 2026: contributor

    ACK 461d83abca1cc68b197cb0c8d9711b4176d4a269

    always nice to see all metadata make it safely to its destination. preserving all PSBT global data is required because most wallets and other bitcoin software may heavily depend on specific data in the global map being defined and accurate

  11. g-maxxx commented at 5:28 AM on July 16, 2026: none

    ACK 461d83a

  12. in test/functional/rpc_psbt.py:1120 in 461d83abca outdated
    1115 | +        psbt_first_obj = PSBT.from_base64(psbt1)
    1116 | +        psbt_first_obj.g.map[collide_key] = b"\x11\x11\x11\x11"
    1117 | +        psbt_second_obj = PSBT.from_base64(psbt2)
    1118 | +        psbt_second_obj.g.map[collide_key] = b"\x22\x22\x22\x22"
    1119 | +        joined_collision = PSBT.from_base64(self.nodes[0].joinpsbts([psbt_first_obj.to_base64(), psbt_second_obj.to_base64()]))
    1120 | +        assert_equal(joined_collision.g.map[collide_key], b"\x11\x11\x11\x11")
    


    Bicaru20 commented at 10:30 AM on July 23, 2026:

    If I am not missing something, maybe we could shorten the test. This way we check at the same time that joining preserves global xpub and proprietary records and that when having the same proprietary key in both PSBTs the first PSBT's value wins.

    I think it is clear enough to combine both teast cases into one, but I am fine as leaving it as is if you think it is clearer this way.

            psbt1_obj = PSBT.from_base64(psbt1)
            psbt1_obj.g.map[xpub_key1] = xpub_value
            psbt1_obj.g.map[global_prop_key] = b"\x11\x11\x11\x11"
            psbt2_obj = PSBT.from_base64(psbt2)
            psbt2_obj.g.map[xpub_key2] = xpub_value
            psbt2_obj.g.map[global_prop_key] = b"\x22\x22\x22\x22"
            joined_globals = PSBT.from_base64(self.nodes[0].joinpsbts([psbt1_obj.to_base64(), psbt2_obj.to_base64()]))
            assert_equal(joined_globals.g.map[xpub_key1], xpub_value)
            assert_equal(joined_globals.g.map[xpub_key2], xpub_value)
            # Same proprietary key in both PSBTs with different values: the first PSBT's value wins
            assert_equal(joined_globals.g.map[global_prop_key], b"\x11\x11\x11\x11")
    
  13. Bicaru20 commented at 10:36 AM on July 23, 2026: contributor

    ACK 461d83abca I really like the approach taken. This way in joinpsbt the code is simpler and easy to follow and we do not creat an extra psbt to shuffle the inputs and outputs. Also this way preserve the global data.

    Left a code suggestion. Feel free to ignore it if you prefer the current approach.

  14. achow101 referenced this in commit 367b2202a4 on Aug 19, 2026
  15. rpc: preserve global xpubs and proprietary fields in joinpsbts
    joinpsbts collects the global xpubs of all the joined PSBTs into
    merged_psbt, but returns a separately constructed shuffled_psbt into
    which only the inputs, outputs, and unknown fields are copied. The
    collected PSBT_GLOBAL_XPUB records are silently dropped, and
    PSBT_GLOBAL_PROPRIETARY records are not collected at all.
    
    The xpub collection was added in #17034, which was written against a
    joinpsbts that still returned merged_psbt, but was merged after #16512
    had introduced the shuffled_psbt rebuild, so the collected xpubs have
    never reached the result.
    
    Shuffle the inputs and outputs of merged_psbt in place instead of
    rebuilding a new PSBT, so that all global data is preserved, and union
    the global proprietary records in the merge loop, matching the
    combinepsbt behavior from #34893.
    011094b282
  16. achow101 commented at 6:09 PM on August 19, 2026: member

    Please rebase

  17. test: check joinpsbts preserves global xpubs and proprietary fields 436921eb46
  18. thomasbuilds force-pushed on Aug 19, 2026
  19. winterrdog commented at 6:33 PM on August 19, 2026: contributor

    @thomasbuilds

    Is this still on, now that a rebase has been done ?

  20. thomasbuilds commented at 6:42 PM on August 19, 2026: contributor

    Yes check the diff please L1192-L1198

  21. winterrdog commented at 11:42 PM on August 19, 2026: contributor

    tACK 436921eb469a1c4dde9f46e2fbad231da0e17e7a

    all tests ran successfully on Debian/x86_64/clang++-18

  22. DrahtBot requested review from Bicaru20 on Aug 19, 2026
  23. jpk68 commented at 8:36 PM on August 24, 2026: contributor

    ACK 436921eb469a1c4dde9f46e2fbad231da0e17e7a

  24. achow101 commented at 9:32 PM on August 24, 2026: member

    ACK 436921eb469a1c4dde9f46e2fbad231da0e17e7a

  25. achow101 merged this on Aug 24, 2026
  26. achow101 closed this on Aug 24, 2026

  27. thomasbuilds commented at 8:16 AM on August 25, 2026: contributor

    Can we get @DrahtBot to unassign review requests once a PR gets merged?

  28. maflcko commented at 8:31 AM on August 25, 2026: member

    Can we get @DrahtBot to unassign review requests once a PR gets merged?

    Yes, the code is open source and pull request or feature requests can be submitted at https://github.com/maflcko/DrahtBot

    However, I don't see the point. What benefit is there to remove reviewers when a pull is merged? It just seems like extra churn for no reason. Also, it should be up to reviewers to decide for themselves whether they want to review a merged pull request, not to DrahtBot.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-09-06 07:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me