net: receive TCP urgent data inline #36688

pull darosior wants to merge 2 commits into bitcoin:master from darosior:2610_oobinline changing 2 files +45 −0
  1. darosior commented at 6:58 PM on October 28, 2026: member

    There is currently nothing preventing us to receive TCP urgent data, which would never be returned by recv(). Linux's poll() is indifferent to this, but the select() socket monitoring may keep reporting it as pending on some platforms.

    This would not prevent progress nor lead to a crash anymore after #36665, but it's still undesirable. Clean this up by setting SO_OOBINLINE on our sockets, so recv() returns any urgent data sent to us (and likely leads us to disconnect the peer).

    This is a slight behaviour change on Linux, but i think a desirable one. First of all because if a peer is sending us funky data we might as well hear about it and disconnect them, but also because i think one lesson to draw from the recent socket related security vulnerability is that we should reduce the amount of custom logic for less well-tested platforms.

    Failing to set SO_OOBINLINE was conservatively made a warning and not a failure, because its absence does not prevent node operations, it's just annoying.

  2. DrahtBot added the label P2P on Oct 28, 2026
  3. DrahtBot commented at 6:58 PM on October 28, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    External sites

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #36434 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36434.svg"></sub> (http: support unix sockets by pinheadmz)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  4. pinheadmz commented at 7:07 PM on October 28, 2026: member
  5. ViniciusCestarii commented at 8:12 PM on October 28, 2026: contributor

    ACK 4655c2305e80ef19353739a6f66e49442945ed44

    Ran the test and checked it fails with the SO_OOBINLINE change reverted

  6. DrahtBot requested review from pinheadmz on Oct 28, 2026
  7. optout21 commented at 10:08 PM on October 28, 2026: contributor

    ACK 4655c2305e80ef19353739a6f66e49442945ed44

    Reviewed the code, verified the test locally (on Linux). Checked that without setting the option the new test fails. Checked other socket creation places -- other instance found only in ipc/capnp. Checked other SetSockOpt usages. Looked into why TCP_NODELAY option is set again after accept, while SO_OOBINLINE is not. Apparently even setting TCP_NODELAY again is extra cautious, and not needed on any of the supported platforms. Inheritance of SO_OOBINLINE should be even more reliable, as it is a socket-level option, not TCP level. If the socket option is not inherited, the test would fail. Verified this by unsetting SO_OOBINLINE right after the Accept in the test (that causes the test to fail).

    I would've liked more adding the test first, then adding the fix and updating the test. It's fine this way too.

  8. xyzconstant commented at 2:10 AM on October 29, 2026: contributor

    ACK 4655c230

    Receiving urgent data inline makes total sense. Nice test and code change.

  9. in src/test/netbase_tests.cpp:643 in 4655c2305e
     638 | +    // Connect to server from client. Use `CreateSockOS` directly instead of the `ConnectDirectly`
     639 | +    // helper to make sure the peer doesn't become a mock (which could happen for instance if
     640 | +    // the test suite was updated to use SocketTestingSetup instead of BasicTestingSetup).
     641 | +    const auto peer{CreateSockOS(AF_INET, SOCK_STREAM, IPPROTO_TCP)};
     642 | +    BOOST_REQUIRE(peer);
     643 | +    (void)peer->Connect(reinterpret_cast<const sockaddr*>(&addr), sizeof(addr));
    


    vasild commented at 9:04 AM on October 29, 2026:

    nit: GetSockName() fills addr with some data and sets its size in the out parameter addr_len. I think logically, Connect() should take addr_len, not sizeof(addr). I guess this is mostly theoretical concern because my understanding is that both will be equal. Maybe add BOOST_REQUIRE_EQUAL(addr_len, sizeof(addr));, or s/sizeof(addr)/addr_len/ or just ignore.


    darosior commented at 1:43 PM on October 29, 2026:

    Yes, using addr_len is cleaner.


    darosior commented at 1:58 PM on October 29, 2026:

    Done.

  10. in src/netbase.cpp:574 in 4655c2305e
     568 | @@ -569,6 +569,12 @@ std::unique_ptr<Sock> CreateSockOS(int domain, int type, int protocol)
     569 |          if (sock->SetSockOpt(IPPROTO_TCP, TCP_NODELAY, &on, sizeof(on)) == SOCKET_ERROR) {
     570 |              LogDebug(BCLog::NET, "Unable to set TCP_NODELAY on a newly created socket, continuing anyway\n");
     571 |          }
     572 | +
     573 | +        // Receive TCP urgent data inline, like any other data. Out of band, it is not returned by
     574 | +        // recv() but some platforms keep reporting it as pending. Accepted sockets inherit this.
    


    vasild commented at 9:11 AM on October 29, 2026:

    I think it would be useful to mention which platforms report it as pending.


    darosior commented at 1:59 PM on October 29, 2026:

    Updated comment to apply to all platforms using select() to monitor.

  11. vasild approved
  12. vasild commented at 9:13 AM on October 29, 2026: contributor

    ACK 4655c2305e80ef19353739a6f66e49442945ed44

    Linux's poll() is indifferent to this, but the select() socket monitoring may keep reporting it as pending on some platforms.

    Do you plan to completely remove the select() branch and use poll() on all platforms?

    <details> <summary>Show Signature</summary>

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA256
    
    ACK 4655c2305e80ef19353739a6f66e49442945ed44
    
    > Linux's poll() is indifferent to this, but the select() socket monitoring may keep reporting it as pending on some platforms.
    
    Do you plan to completely remove the `select()` branch and use `poll()` on all platforms?
    -----BEGIN PGP SIGNATURE-----
    
    iQRPBAEBCAA5FiEE5k2NRWFNsHVF2czBVN8G9ktVy78FAmrjDhEbFIAAAAAABAAO
    bWFudTIsMi41KzEuMTIsMiwzAAoJEFTfBvZLVcu/EzMgAIP+VL+22/8xlzvwT4oj
    s96hKoKR5iukh2Lyd7M3VIhCG/ltDcQD93bO92TrvJABHxRTeAWOzYGogJlg02js
    LlIF9/6iliUXlB8JyhFpWaAAb8cygAVjmK0IsAFzWOmaH96b+7PDpLR0mgN7yce9
    Yrv5Ag/PNcsd0APnGJSuONGfonPWr98QFL2yVQ/0RoSt9PUh4pjF74Ejvj5h0pSr
    23C0zMGudapj3TgBcdsmi455YD11DInkJdltBex3QnCa/zQIcroYv1RAYsZx+lIu
    ZlZbcJlI7SiRLPfGCaseQv8RMa3WO5CD6HgO7F8gQlWuvTa+RXTZKIgTyMSkxIaU
    hw1roQ/Fv9LdiC6Z7Tn/vQzBhKSc6EeUSJKE1O8d8FdABHkOo1cxJJLzOVxiGh6u
    7RR7fZIfhSrb4oN7H6kfVyS5PZmuE4FlH2y5UakdJHqq3T99SkQnz/3syFSvay9J
    zGRcMC2a5+lv38guZHHNYJLGSSxD0z8DbT+jU8uP0ji6UslhHY8LRbk04UbFQKh5
    h27p+PpNoP3TmleXWLbJZga9eCNSQXul37JV7+lgqciYGGJcrKUYYV8r8wkdpR5T
    kxw1SDSdIM5KQBnSPIH29fU5FN2DXqrPtyYXPPGC8Gg01e76KBtGzNk27Wb/LSP7
    iu0Zyx1QruhsHdF4IvUkPVlFWlsRtDPxF6pZSE/OUD6uaJ2jwuuQdB01sqbTaDjf
    QlRnddEci9GMAWJ8cvyenNowgDruk6U9FbeTUvCematHrcrXd6IntAZHQbH5bRxy
    HQ5RPdpUAqMgMXr01hlnZEHmi3wHSdOBONVOUUi+JLa5aFmkD/18cBsjIevv22wq
    VA9ETbhKeNfQNWm4GrBrAfCKJkU48kHe11qHUqq8Ige7z3GDUlzk4ELSpjALGkL/
    UYdeoEk1QZpuc9jt+rmFmi7QxWWCdn26E95Lxd9kC3SZZI18rVriF/oEmxt1hvTx
    vhWKA+zfcU7d3u0K89DI3MsBykuW0417RJIFAdNCc5f5LekgB7oTWNLvKUqO3q92
    HOlDivcg07kL4BrkCd8gZ459yAb3J+rrOn+OzhxZox1a6TiT53H0MlvQajv3liMK
    ybGyu8VBl0g32ajm6CS6y0bjIneZ8++dUJh1p1RtGnkh1ux4Fun/MIeWUZnGFRW/
    vXdQ0oPT4AKfEuWMriOD48yECYJe7E0XPwVzQTtO2MforCnf5GwD+z23cUhh+D3x
    chKd0CpITKAVayBDz3dBBNv/ITL1b58vP7u0D6kZ4Pu6wcbQsaqiwTehihKC9QJk
    4Hy0yzspZ3g6wodtLGd4+CGTegZmVh8rQbndyaMijdfcCb9EqtVHSaIUOWAfymHE
    5WI=
    =aXRI
    -----END PGP SIGNATURE-----
    

    vasild's public key is on openpgp.org

    </details>

  13. fanquake added this to the milestone 33.0 on Oct 29, 2026
  14. darosior force-pushed on Oct 29, 2026
  15. net: receive TCP urgent data inline
    Set SO_OOBINLINE on TCP sockets, to receive urgent data like any other
    data. Otherwise, recv() does not return it, while select() keeps
    reporting it as pending on some platforms.
    23fe594b5b
  16. qa: test TCP urgent data is received inline de87cb8407
  17. pinheadmz commented at 2:01 PM on October 29, 2026: member

    While all the reviews are stale, quick question: should the test live in sock_tests.cpp? There is a struct TcpSocketPair that might be handy

  18. darosior commented at 2:06 PM on October 29, 2026: member

    Thought about this, but CreateSockOS is netbase's function, which is a caller of util/sock. I think sock_tests should be contained to testing util/sock: it would awkward to include netbase.h there when we have netbase_tests.

  19. vasild commented at 2:18 PM on October 29, 2026: contributor

    ACK de87cb84079105968d10c8208565e65d5e7670c5

    <details> <summary>Show Signature</summary>

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA256
    
    ACK de87cb84079105968d10c8208565e65d5e7670c5
    -----BEGIN PGP SIGNATURE-----
    
    iQRPBAEBCAA5FiEE5k2NRWFNsHVF2czBVN8G9ktVy78FAmrjVXUbFIAAAAAABAAO
    bWFudTIsMi41KzEuMTIsMiwzAAoJEFTfBvZLVcu/sVkf/2Zmy+bfcvipHHZ7qLpp
    QWKUY5CgrX85+C1t9lffaI3Jc4fJvE1srgHE8sukvJmcgR2GaRUt/di3k7pqNjh6
    i8uT1Dc4aE6PvKbXwCVZIBtaLjYo3Kl6UH7ZmHrpg/qHMMLuOaigJGp0nQv02TT7
    zaK8la7Zd6jnL8RJ95ZChSxfHWE0B0iVpaQ84js8r7ZOiApOBbt9eWP5GfluGxHc
    T/YNHE0VpR4tpzu9qoL4qBQf+iIFjvWnzLKr7yFEHWKpaRh82Yh3/Lb2v5zixr4O
    iP5I/xthMtsKBRhfHRLZLLXi10Doc+2JVRG3sArtseBeQMZNXnWz8vjvNhsh2ilu
    PoRDWKRguFS32/PwLinYpm1cRqYeH4wDd8DSuVzexLF+m9QxOj7fdtityfar8k1O
    /1GAgcCceMKyMW9SoA3BczNFoCNgRiHB7RCLfWEwfCHRV5LKl76aKZjsMMr4hs4E
    3C1YQ6mbCXF7hmTVzB0EMJUvZQ9CmUgqYsrvF6O9xRswPm6E8PfWMLXkQS83ffbm
    bL0D1znMGEb40S8R3oF4CiEApPSUK8OZjpDpYxhKQBEm17SUovhINDHB2aoewltw
    hLIORRnN2R7MwHVSUDLYpaRG30NqSggsOe7bZNwRGbddLjeRxNsRK6bUqLe056Zb
    WNhlHIknpq/2zrOqe2iDUAaJNVKm+fzSs0YuyGyOqfUfqx9v9ewoNClNQ/ZZTimm
    5fEvmmVCzRaf3FdCrDtgnLM+ntZS5oD/0ZG0mlwB3jsj4qXss274lIQP8bdN5JC/
    XheETOx1Ygd41Zky6Q517exC40RJ2GgW7GmF10n37bfFjx/P0/R/bVfSIfKUGJQo
    fHiBuwWmIW/sxIoBq3A60ShwsJwhuAquxGOAjvjh4MfHuQGNyEtia18qwKclSKJt
    9jQMl/Y1zdC9cEq7u7E/fD5NBvT88R79Y/loL8c++G/QaAEDaYiepnZ9oiT/KNQB
    Oqth3TW2IzXo9tpiAHCa901a3BhlS1kowoolvU21d9ZOnerUxoenyxQ/au5eNUax
    wwBYmV/KDlExKvqKHTZI0l5IFh8KgZ3meHdcBnuc4x4bulraruS3JLdDtItJdKLe
    fEnBqR2BdPIfLavYFcdNR9KjvQovLsFMPRTz6ODvhGv4lnF/8sZyioZ5GQt0SDV+
    K3178BgwBofYHITQGIXUGE+h2mOh/1lswAXz92pVS92CZsj+v5GjN7GbnLThwdtq
    Eg7GhhuSdpGURPe5+ztLn5WnqW/n0IJ6Ywtuoev3Hcdo+HAicF9DosrWt0ZvVGzz
    B+6oCqs/McCBPKziR6bCT14Dqh9BeS4tFJDPizKYgzWrG1Cv5iAzg/SMBO4p8DYd
    qhg=
    =qCHp
    -----END PGP SIGNATURE-----
    

    vasild's public key is on openpgp.org

    </details>

  20. DrahtBot requested review from ViniciusCestarii on Oct 29, 2026
  21. optout21 commented at 2:41 PM on October 29, 2026: contributor

    reACK de87cb84079105968d10c8208565e65d5e7670c5

  22. in src/netbase.cpp:575 in de87cb8407
     568 | @@ -569,6 +569,12 @@ std::unique_ptr<Sock> CreateSockOS(int domain, int type, int protocol)
     569 |          if (sock->SetSockOpt(IPPROTO_TCP, TCP_NODELAY, &on, sizeof(on)) == SOCKET_ERROR) {
     570 |              LogDebug(BCLog::NET, "Unable to set TCP_NODELAY on a newly created socket, continuing anyway\n");
     571 |          }
     572 | +
     573 | +        // Receive TCP urgent data inline, like any other data. Out of band, it is not returned by
     574 | +        // recv() but select() reports it as pending.
     575 | +        if (sock->SetSockOpt(SOL_SOCKET, SO_OOBINLINE, &on, sizeof(on)) == SOCKET_ERROR) {
    


    pinheadmz commented at 2:49 PM on October 29, 2026:

    nit if u touch, on and set above could be cleaned up like we did in httpserver.cpp

    static constexpr int SOCKET_OPTION_TRUE{1};

  23. pinheadmz approved
  24. pinheadmz commented at 2:52 PM on October 29, 2026: member

    ACK de87cb84079105968d10c8208565e65d5e7670c5

    built and tested on macos/arm64. Generated functional tests that send urgent bytes to the http server: https://gist.github.com/pinheadmz/f5bb77865f870865e44c54d5751b569a these pass on the branch and fail on master. I also wanted to check what happens if the "urgent" flag is set but the urgent byte itself has not been sent yet.

    <details><summary>Show Signature</summary>

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA256
    
    ACK de87cb84079105968d10c8208565e65d5e7670c5
    -----BEGIN PGP SIGNATURE-----
    
    iQJPBAEBCAA5FiEE5hdzzW4BBA4vG9eM5+KYS2KJyToFAmrjXRUbFIAAAAAABAAO
    bWFudTIsMi41KzEuMTIsMCwzAAoJEOfimEtiick6itYQAKeKz4XjNHEcyiGMmrYC
    iBu3JrcyY5qKZONatgo+vMIJurzwKxZTR7DPrqVr7ZjnBx1AhJpDUTWEed3fkLj0
    leoDWGFnt+PUnn8BX8RzRQPdjmddSuXVEVHyVTYVJriUVpSUTwQ1PQC2TaK+c4jF
    Biyry9lWry+onb6s4V3nKqmF3GcYwc08LhFaev+6L6s7SjlzEJge0oh9dNUKXH67
    XUWXb9w34TowVuC5z6bicjzgYKDIeVH1y/jXDFNjaSW7aF4D7P6bknvEAmjT1uRx
    zLZ63F754VZQd47s6A4gEsYdUZzwoxt1TRsMQ7Lyr+jNIwJogVL7kh+Z+ZI82LqZ
    8szhu7EjoIs8ORr10a9UpKSagkHvq4X8T4EyVhBmGpowLu0P+0wSxotQkRTaFVfx
    wZ8Wc046sTg85+R2LTyxbnzh90l2opPiXNyKrcGmlW9Y+bxgasuuo4TNocyUtwvm
    YchBjVpgyzn2e6PzM8AODLCbrtplncZdQbTqC5lzE8bbhG+xY5uTGOlYpy3FqKjf
    ccS7ohwgKSz+gKeV7coZ4jFUrwARfTGS4909TDpkz9ytFneDt0vkF2y+DjBDeRzj
    s13N8gVSJBfmqifuTOjnjQbPYAnFBXcPxn8fInzvCkQk7G3Y3wmLoSF4O5/k7r7a
    B3oQYUzIGxxBIvOK9GbEptfr
    =Z1pC
    -----END PGP SIGNATURE-----
    

    pinheadmz's public key is on openpgp.org

    </details>

  25. darosior commented at 3:03 PM on October 29, 2026: member

    Do you plan to completely remove the select() branch and use poll() on all platforms?

    Yes. I am currently working on pinning down all the OS-specific behaviour each implementation is relying on first, to provide more assurance a switch to poll() is safe (and help minimize the risk of more security vulns in our socket logic in the future).

  26. winterrdog commented at 3:12 PM on October 29, 2026: contributor

    ACK de87cb84079105968d10c8208565e65d5e7670c5

  27. xyzconstant commented at 3:40 PM on October 29, 2026: contributor

    re-ACK de87cb84079105968d10c8208565e65d5e7670c5

  28. ViniciusCestarii commented at 4:51 PM on October 29, 2026: contributor

    reACK de87cb84079105968d10c8208565e65d5e7670c5

  29. darosior commented at 8:38 PM on October 29, 2026: member

    Let's hold off on merging this, i think i found something wrong here.

  30. darosior commented at 10:04 PM on October 29, 2026: member

    The patch here is correct, but insufficient to address the issue entirely on macOS/BSD.

    This comment from @pinheadmz got me wondering:

    I also wanted to check what happens if the "urgent" flag is set but the urgent byte itself has not been sent yet.

    So i checked this as part of my socket tests hardening work, and it turns out that if you also set the urgent data pointer in the TCP segment header past the last byte you sent, select() on macOS / BSD keeps waking up while the socket is waiting to receive, until data past the mark has been read. Testing this is annoying (and can't reliably be done in our test suite) because the socket API won't let you set that pointer arbitrarily. So this requires writing IP packets directly in a raw socket, or some more involved hackery on Windows [^1].

    So a peer receiving such a packet on:

    • Linux is unaffected (poll() without POLLPRI)
    • Windows will not raise the exception set (with SO_OOBINLINE)
    • macOS will raise the exception set (it does so as soon as it sees the pointer, even if the byte itself hasn't arrived yet, regardless of SO_OOBINLINE)

    Thankfully the exception set is only necessary on Windows, to detect a failed connection attempt. So we could ifdef it away, which would fix the macOS/BSD issue. But this is a bigger change, and i think the right solution is to just move to using poll() for all platforms.

    Given the patch in this PR is simple, reviewed, and already fixes Windows, i think the complete fix for macOS should happen in a followup that either switches it to using poll() or gates the exception set to only be set for Windows.

    [^1]: My bot was able to test this by crafting Ethernet frame and sending them to QEMU over UDP for it to inject them into the emulated NIC of the Windows VM.. Whatever.

  31. darosior commented at 10:05 PM on October 29, 2026: member

    Looks like Ralph had caught the issue, too. cc @willcl-ark

  32. jpk68 commented at 10:55 PM on October 29, 2026: contributor

    ACK de87cb84079105968d10c8208565e65d5e7670c5

  33. Sjors commented at 7:29 AM on October 30, 2026: member

    Would it make sense to add this?

    diff --git a/src/net.cpp b/src/net.cpp
    index 9bc7887a3c..3e6c1a0ae3 100644
    --- a/src/net.cpp
    +++ b/src/net.cpp
    @@ -2200,4 +2200,15 @@ void CConnman::SocketHandlerConnected(const std::vector<CNode*>& nodes,
             }
     
    +#ifndef USE_POLL
    +        if (errorSet) {
    +            // select() can report an exceptional condition that recv() cannot clear,
    +            // such as urgent data if SO_OOBINLINE failed, or an urgent pointer to
    +            // data not yet received. Disconnect instead of repeatedly waking.
    +            LogDebug(BCLog::NET, "socket exceptional condition, %s", pnode->DisconnectMsg());
    +            pnode->CloseSocketDisconnect();
    +            continue;
    +        }
    +#endif
    +
             if (sendSet) {
                 // Send data
    

    In other words: hang up if we receive an urgent byte, on platforms with select().

    We could avoid SO_OOBINLINE altogether if we treat the presence of URG (after a connection is established) as misbehavior. We could do that on poll() implementations too, even though it's not a problem there currently.

    I'm not sure if the errorSet() can be triggered by issues other than a peer sending URG, besides what we already handle in ConnectToSocket. That might cause an "underserved" disconnect. But with SO_OOBINLINE such conditions would probably put random bytes in the buffer that would lead to a disconnect, per the original motivation of this PR.

    More subjectively, I find SO_OOBINLINE to be a confusing feature. So if we don't need it, that would be nice.

  34. vasild commented at 8:42 AM on October 30, 2026: contributor

    The patch is fine and I retain my ACK as it is, but this angle is worth considering:

    More subjectively, I find SO_OOBINLINE to be a confusing feature. So if we don't need it, that would be nice.

    So, we do not use out-of-bound data and we do not care about it, but we would set SO_OOBINLINE even though we do not expect to receive any OOB, just to fix a problem elsewhere, hmm?

    From this angle, @Sjors's idea to disconnect if we receive OOB seems more in line with our expectations to not receive OOB. Also, the #ifndef USE_POLL snippet would naturally go away if we use poll(2) on all platforms.

  35. darosior commented at 2:31 PM on October 30, 2026: member

    @Sjors @vasild i think that works but is fragile. The exception set is not specific to urgent data, it is a general set of descriptors being watched for "exceptional conditions". In practice, it is only used for urgent data for sockets on POSIX, but Windows also uses it to report failed outgoing connections for non-blocking sockets. So your patch works because it only disconnects on exceptions after the connection was established. I think this is brittle to rely on the exception set to always indicate presence of urgent data, and to implement a low-level socket check at the connection handler level.

    More subjectively, I find SO_OOBINLINE to be a confusing feature. So if we don't need it, that would be nice.

    I would say the opposite, actually. It's the default of pulling the urgent byte out of the stream which is confusing, and SO_OOBINLINE restores the socket API to behave as you would expect. As mentioned above, even RFC 6093 calls the default "broken" and states that SO_OOBINLINE restores the "semantics intended by the IETF specifications".

    I think a better approach is what i mentioned in my earlier message: at the socket level, only ever register the exception set for Windows.

    --- a/src/util/sock.cpp
    +++ b/src/util/sock.cpp
    @@ -216,7 +216,11 @@ bool Sock::WaitMany(std::chrono::milliseconds timeout, EventsPerSock& events_per
             const auto& s = sock->m_socket;
             if (events.requested & RecvEvent) {
                 FD_SET(s, &recv);
    +#ifdef WIN32
    +            // Windows reports a failed non-blocking connect() only in the exception set. Other
    +            // platforms report it as readable and writable.
                 FD_SET(s, &err);
    +#endif
             }
             if (events.requested & SendEvent) {
                 FD_SET(s, &send);
    

    That would solve the remaining issue in this PR. I would still prefer ditching select() for macOS/BSD, but i guess we could do both, so we can backport that smaller change.

  36. theuni commented at 3:41 PM on October 30, 2026: member

    So, we do not use out-of-bound data and we do not care about it, but we would set SO_OOBINLINE even though we do not expect to receive any OOB, just to fix a problem elsewhere, hmm?

    I agree with @vasild here. OOB data has no meaning in our protocol. We would never set it ourselves, and we would never take any action based on it.

    Either the data was never necessary and can be ignored, or the data was never necessary and should be treated like an error. But I don't see "the data was previously ignored but is now considered valuable", as this PR does, as a solution.

    Imo, we shouldn't be worrying ourselves by having to research what platforms might be setting OOB data and what it may mean. Because it has no value to us, I'd rather just always treat it like an error. And same goes for any other exception cases.

    Something like (simplified and untested):

    diff --git a/src/net.cpp b/src/net.cpp
    index 1201d3bcb47..f47a7277560 100644
    --- a/src/net.cpp
    +++ b/src/net.cpp
    @@ -2184,6 +2184,14 @@ void CConnman::SocketHandlerConnected(const std::vector<CNode*>& nodes,
                 }
             }
    
    +        if (errorSet) {
    +            if (!pnode->fDisconnect) {
    +                LogDebug(BCLog::NET, "socket recv exception, %s", pnode->DisconnectMsg());
    +            }
    +            pnode->CloseSocketDisconnect();
    +            continue;
    +        }
    +
             if (sendSet) {
                 // Send data
                 auto [bytes_sent, data_left] = WITH_LOCK(pnode->cs_vSend, return SocketSendData(*pnode));
    @@ -2201,7 +2209,7 @@ void CConnman::SocketHandlerConnected(const std::vector<CNode*>& nodes,
                 }
             }
    
    -        if (recvSet || errorSet)
    +        if (recvSet)
             {
                 // typical socket buffer is 8K-64K
                 uint8_t pchBuf[0x10000];
    diff --git a/src/util/sock.cpp b/src/util/sock.cpp
    index 4204ffe399c..99dc1488dc5 100644
    --- a/src/util/sock.cpp
    +++ b/src/util/sock.cpp
    @@ -196,7 +196,7 @@ bool Sock::WaitMany(std::chrono::milliseconds timeout, EventsPerSock& events_per
             if (pfds[i].revents & POLLOUT) {
                 events.occurred |= SendEvent;
             }
    -        if (pfds[i].revents & (POLLERR | POLLHUP)) {
    +        if (pfds[i].revents & (POLLERR | POLLHUP | POLLPRI)) {
                 events.occurred |= ErrorEvent;
             }
             ++i;
    
  37. theuni commented at 3:49 PM on October 30, 2026: member

    Heh, I missed @Sjors's comment. I agree with that. My suggestion is the same as his, just with poll's behavior unified.

  38. theuni commented at 4:08 PM on October 30, 2026: member

    I would say the opposite, actually. It's the default of pulling the urgent byte out of the stream which is confusing, and SO_OOBINLINE restores the socket API to behave as you would expect. As mentioned above, even RFC 6093 calls the default "broken" and states that SO_OOBINLINE restores the "semantics intended by the IETF specifications".

    You missed this part:

    Even though applications SHOULD NOT employ the urgent mechanism

    The brokenness of the semantics is only relevant if you're interested in the OOB data to begin with, which we're not. Following the spec would be to ignore the urgent mechanism or treat it as an error.

  39. darosior commented at 4:20 PM on October 30, 2026: member

    @theuni i did not. I am not suggesting to set SO_OOBINLINE because our application needs urgent data, on the contrary. SO_OOBINLINE is simply the standard way to opt out of the broken default urgent-data handling, as RFC 6093 puts it.

    I agree we could instead disconnect on an urgent data notification [^1], but because Windows also needs the exception set, what you suggest leads to a layer violation by which the connection handler needs to know that an exception raised by the socket handler is necessary related to TCP urgent data. Instead of watching for urgent data on more platforms, why not go the other way instead, and only register the exception set on Windows where it's actually necessary?

    I think we agree on the observation. Our application does not need urgent data, and we should discard it and preferably disconnect anyone trying to send it to us. I think your suggestion is acceptable, to be clear, but i think it actually bakes more assumption on this obscure TCP urgent data mechanism, and more platform-specific assumptions at a higher level of our stack, than is necessary to discard such data. (And now this platform-specific assumption needs to spread to other higher-level parts of our stack, such as the HTTP server, which your patch lacks.)

    [^1]: This PR does disconnect on an urgent byte read in non-adversarial cases too, because it would break parsing unless it is strategically positioned.

  40. theuni commented at 4:35 PM on October 30, 2026: member

    because Windows also needs the exception set, what you suggest leads to a layer violation by which the connection handler needs to know that an exception raised by the socket handler is necessary related to TCP urgent data

    I'm not following. Or at least, select() already violates this implicitly. Adding it as an error for poll makes it explicit that we consider this (previously unspecified) case should be treated as an exception, as select does. That makes it so that any exception can be treated as an error, which means that Windows will just do the right thing and disconnect.

    Am I missing something?

    This PR does disconnect on an urgent byte read in non-adversarial cases too, because it would break parsing unless it is strategically positioned

    Relying on our checksum to notice the extra data seems like a much more substantial layer violation :)

  41. theuni commented at 4:49 PM on October 30, 2026: member

    And now this platform-specific assumption needs to spread to other higher-level parts of our stack, such as the HTTP server, which your patch lacks.

    I don't see how it's platform-specific assumption. Imo lumping the read/exception cases together has always been wrong, and should indeed be fixed in both places.

  42. darosior commented at 5:44 PM on October 30, 2026: member

    I'll reply because i want to understand if i'm missing something, but if you all prefer the Cory-Sjors suggestion, i am fine with that too. Fixing and backporting that is more important than the details here, i think. And anyways, down the line, i think we should get rid of the select() exception set in favour of using poll() everywhere.

    Could you PR your suggestion?

    Adding it as an error for poll makes it explicit that we consider this (previously unspecified) case should be treated as an exception, as select does.

    Watching for exceptions is only necessary as a select()-specific workaround for Windows. Starting to watch for out-of-band data in poll() is in my view a step backwards in the direction of not needing it anywhere. Especially as the alternative is to register it only where needed, fixing the BSD/macOS issue created by the Windows-specific workaround.

    Relying on our checksum to notice the extra data seems like a much more substantial layer violation :)

    Heh, fair enough :), but the disconnection is merely an additional nice-to-have, not the core mechanism of the fix here.

    Imo lumping the read/exception cases together has always been wrong, and should indeed be fixed in both places.

    I agree, consumers shouldn't keep reading on ErrorEvent regardless of the decision here. I fixed one such case in #36676, but there are more (for instance I2P and SOCKS5 ignore what Wait() reports).

  43. darosior commented at 5:57 PM on October 30, 2026: member

    I think Cory is right here. Callers continuing to read on error/exception is the root cause here, and it's cleaner to fix that than to try and let them doing this by carefully patching what we watch for at the socket level.

    I still think we shouldn't be watching for urgent data anywhere, but i agree that for as long as we are forced to do it on one platform (Windows), we should be consistent across platforms and do it everywhere.

    Therefore i don't suggest pulling the change in this branch anymore at this time. Thanks @theuni.

  44. darosior closed this on Oct 30, 2026

  45. Sjors commented at 3:31 AM on October 31, 2026: member

    It makes sense for us to follow the RFC 6093 recommendation of setting SO_OOBINLINE, just to be consistent with other software, in order to make our code is easier to understand for experts in this area.

    Update: the recommendation is conditional on using the urgent mechanism in the first place. So we shouldn't set it, unless there's a good reason.

    Even though applications SHOULD NOT employ the urgent mechanism

    We have an additional reason to never deploy the urgent mechanism: having cleartext interrupts at the tcp level goes against the design of v2 encrypted traffic.

    It would help to have a code comment that references both the RFC and the v2 rationale.

    That should address my concern about intuitiveness (if we did support interrupts, rather than just disconnect on them, we would not want SO_OOBINLINE so v2 decryption can keep going).

    i think that works but is fragile. The exception set is not specific to urgent data, it is a general set of descriptors being watched for "exceptional conditions".

    I also agree with this. In general it's probably fine to disconnect when something exceptional happens, it does introduce a risk of spurious disconnects.

    The robust strategy is to ignore urgent data, except when we're specifically interested in it (like the Windows case), in a way that can't hog resources.

    I think a better approach is what I mentioned in my earlier message: at the socket level, only ever register the exception set for Windows.

    Agreed. Though I still need to study @theuni's suggestion.

    Ignoring OOB data, rather than disconnecting on it, has the additional benefit that fake data can be used for traffic shaping. We don't get that with SO_OOBINLINE, but that's not a strong argument against it.

  46. Sjors commented at 4:14 AM on October 31, 2026: member
  47. okonendurance20-sudo commented at 4:49 AM on November 5, 2026: none

    The MIT License (MIT)

    Copyright (c) 2009-2026 The Bitcoin Core developers Copyright (c) 2009-2026 Bitcoin Developers

    Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

    The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

  48. midnightmagic commented at 12:59 AM on November 6, 2026: contributor

    @theuni i did not. I am not suggesting to set SO_OOBINLINE because our application needs urgent data, on the contrary. SO_OOBINLINE is simply the standard way to opt out of the broken default urgent-data handling, as RFC 6093 puts it.

    fwiw, I described a mechanism by which it wouldn't actually do that, over here:

    Comment in PR 36716

    I agree we could instead disconnect on an urgent data notification 1, but because Windows also needs the exception set, what you suggest leads to a layer violation by which the connection handler needs to know that an exception raised by the socket handler is necessary related to TCP urgent data. Instead of watching for urgent data on more platforms, why not go the other way instead, and only register the exception set on Windows where it's actually necessary?

    I strongly recommend treating oob data as a banning disconnect for the above reason. The implications based on historical data handling are default unfriendly.

    Footnotes

    1. This PR does disconnect on an urgent byte read in non-adversarial cases too, because it would break parsing unless it is strategically positioned. ↩

    Yes please! :-)


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-11-20 15:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me