This should not replace inbound-outbound connections, and isn’t a replacement of natpmp either, but offers a potential best-effort extension for extra connectivity of home nodes. We discussed this at a recent in-person Bitcoin Core developer meeting and want to keep the conversation going here. We are far from having a proposal for standardization nor an implemenation. At the moment, there are likely more open questions than answers.
TCP hole punching
TCP hole punching lets two computers behind certain NATs connect directly, without relaying traffic through a server. My understanding of one way we could use this:
Alice & Bob: unreachable home nodes behind a NAT
Charlie: reachable node
Both nodes Alice and Bob connect via TCP to a reachable coordinator node Charlie. Charlie could advertise it’s coordinator/matchmaking capabilities in a service flag.
Charlie learns the public endpoints of Alice and Bob. When Alice’s packets pass through her NAT, the NAT rewrites them with a public IP and port. Charlie sees this “outside” address and does the same for Bob.
Charlie swaps endpoints. It tells Alice Bob’s public IP:port, and tells Bob Alice’s.
Both sides simultaneously initiate a TCP connection to each other (See “Figure 7: Simultaneous Connection Synchronization” in RFC 9293) using the same local port they used to talk to Charlie (this requires SO_REUSEADDR/SO_REUSEPORT on the socket). Each side sends a SYN to the other’s public endpoint.
The outbound SYNs punch holes. When Alice’s SYN leaves her NAT heading for Bob, her NAT creates a mapping: “traffic coming back from Bob’s address is allowed in.” Bob’s NAT does the same in reverse.
The SYNs cross in flight. Now each NAT has a hole that permits the other side’s SYN through. The two stacks see this as a “simultaneous open” and complete the handshake directly - no server in the middle.
The tricky part is that TCP is stateful and timing-sensitive, so it’s flakier than UDP hole punching.
Additionally, it only works for certain types of NAT: RFC 4787 defines three NAT address and port mapping behaviors that are relevant here:
Endpoint-Independent Mapping (EIM): picks one external (IP, port) for a given internal socket and reuses it regardless of where the packet is going. Once Alice has sent any outbound packet, her external endpoint is fixed and predictable. Hole punching works. Possibly common on consumer residential routers (cable, fibre, DSL).
Address-Dependent Mapping (ADM): the external port stays the same while Alice is talking to the same destination IP, but a new external port is assigned the first time she contacts a different destination IP. Hole punching is hard. One option is to try to predict ports, but this likely does not work reliably.
Address+Port-Dependent Mapping (APDM; or symmetric NAT): Every distinct destination (IP, port) gets a fresh external port. We cannot predict what external port Alice’s NAT will assign for any specific peer. Hole punching does not work. Typical of CGNAT deployments, mobile carriers, and restrictive enterprise networks.
Usage in the Bitcoin protocol
Two approaches were discussed initially, but I feel like the design space is not fully explored yet.
Rendevouz node: The home-nodes Alice and Bob both want to offer connection slots, but aren’t reachable from the internet. Both happen to pick node Charlie offering a rendezvous/matchmaking service flag and connect to it. Alice might connect first, and needs to wait until Bob connects too. Once both are connected, Charlie then tells them the respective other IP (step 3.). and that they should try to holepunch now. The connection to Charlie now stops. Alice and Bob try for a while to connect to each other.
Inbound handoff instead of eviction: An alternive approach is that both Alice and Bob connect to Charlie, but since Charlies inbound slots are full, it tells them it’s going to hand them off to each other and the connection stops. Currently, when inbound slots are full and peers are evicted, we close the connection. Here, we offer them a way to retain a connection, but with a different node.
In both of these approaches, Charlie learns that Alice and Bob might now be connected, but not for how long. Alice learns that Bob and Charlie were connected. Bob learns that Alice and Charlie were connected. Both don’t know how long they were connected for.
There also was brief discussion on how to make this more private by not requiring a coordnator Carol. Can Alice and Bob, once they’ve figured out their public IP:port behind a EIM NAT, use a e.g. Tor connection to communicate their IP:port to establish a clearnet connection. A protocol without a coordinator makes it a lot easier to reason about the connection either being inbound or outbound, and is likely a lot easier to implement.
Implementation in Bitcoin Core?
Bitcoin Core currently uses the notion of “inbound” and “outbound” connections. What would hole-punched connections be? Outbounds are generally more trusted since we choose them, relay transactions faster, etc. Additionaly, there’s the concept of “outbund-full-relay” and “outbound-block-only” connections.
The connections intially were two “outbound”'s to Charlie. But since Alice didn’t choose the connection to Bob from her addrman (Charlie did while matchmaking), this is not a “we-picked-this-address-from-our-addrman” outbound. So would they be inbound-inbound? We probably have to rethink (and refactor) how we think about connections a bit.
Additional, e.g. BIP-324 has the concept of a Initiator and Responder for encrypted transport. Charlie can assign this to one of the sides. There might be other places where this is useful.
Open Questions
How well does TCP hole punching work in practice? Are there any stats on this by people using it? Does it work with most home routers? How common are EIM NATs and A(P)DM NATs?
Is there theoretical/academic research or P2P projects using this in the wild that we can learn from? Can we come into contact with some them?
When using a coordinator, what happens to Alice if either Bob or Charlie are malicious? What attacks can be done? This likely opens opportunities for sybils or ecplise attacks. How dangerous are these?
Does having “holepunch connections” really address problems we have today? Does it increase the connection slots the network and allows for more peer diversity for unreachable nodes? Does it offer better node inter-connectivity and increases resistance against partition and eclipse attacks? Does it allow for shorter relay paths between unreachable nodes (and is this something we want to optimize for)?
How much new and changed code is needs to implement this? It might change a lot of our current reasoning and understanding of P2P connections. How do we refactor the existing code for this?
For me, at home and also using a phone hotspot indicates APDM for IPv4 and “no NAT” for IPv6:
$ python3 nat-check.py http://b10c.me:7770 http://b10c.me:7771 http://bnoc.xyz:7770 http://bnoc.xyz:7771
nat-check TCP NAT mapping classifier
── IPv4 ──
local source port: 53321
destination external addr
────────────────────────────────────────────────────────────────
b10c.me:7770 (x.x.x.1) x.x.x.2:61314
b10c.me:7771 (x.x.x.1) x.x.x.2:59431
bnoc.xyz:7770 (x.x.x.3) x.x.x.2:63172
bnoc.xyz:7771 (x.x.x.3) x.x.x.2:64569
classification: ADDRESS+PORT-DEPENDENT MAPPING (APDM, 'symmetric')
External port varies per destination (IP, port). TCP hole punching is very
unlikely to work for you: the coordinator cannot predict the external port
your NAT will assign for any given peer. This pattern is typical of CGNAT,
mobile carriers, and restrictive enterprise networks.
── IPv6 ──
local source port: 53322
destination external addr
────────────────────────────────────────────────────────────────
b10c.me:7770 (x::x:4) x::x:5:53322
b10c.me:7771 (x::x:4) x::x:5:53322
bnoc.xyz:7770 (x::x:6) x::x:5:53322
bnoc.xyz:7771 (x::x:6) x::x:5:53322
classification: NO NAT
External address (x::x:5:53322) equals the local address. There is no NAT;
hole punching is unnecessary. However, your home router may still have a
stateful firewall that blocks unsolicited inbound connections. You may need to
open the port on your router to be reachable.
python3 nat-check.py http://b10c.me:7770 http://b10c.me:7771 http://bnoc.xyz:7770 http://bnoc.xyz:7771
── IPv4 ──
local source port: 53486
destination external addr
────────────────────────────────────────────────────────────────
b10c.me:7770 (x.x.x.1) x.x.x.2:53486
b10c.me:7771 (x.x.x.1) x.x.x.2:53486
bnoc.xyz:7770 (x.x.x.3) x.x.x.2:53486
bnoc.xyz:7771 (x.x.x.3) x.x.x.2:53486
classification: ENDPOINT-INDEPENDENT MAPPING (EIM)
All four destinations saw the same external port. Your NAT maps (internal IP,
internal port) to a single external port regardless of destination. TCP hole
punching has a strong chance of working: a coordinator can reliably tell a
peer which external port to send to.
── IPv6 ──
local source port: 53487
destination external addr
────────────────────────────────────────────────────────────────
b10c.me:7770 (x::x:4) x::x:5:53487
b10c.me:7771 (x::x:4) x::x:5:53487
bnoc.xyz:7770 (x::x:6) x::x:5:53487
bnoc.xyz:7771 (x::x:6) x::x:5:53487
classification: ENDPOINT-INDEPENDENT MAPPING (EIM)
All four destinations saw the same external port. Your NAT maps (internal IP,
internal port) to a single external port regardless of destination. TCP hole
punching has a strong chance of working: a coordinator can reliably tell a
peer which external port to send to.
So IPv4 TCP hole punching would not work at home nor via phone hotspot due to being APDM and IPv6 likely requiring opening the firewall, but would work when using Obscura VPN on both IPv4 and IPv6 as it’s EIM.
I would be interested in seeing results from others.
I also vibe-coded a “fun” holepunch program in python to test out the process a little bit, in case it’s of interest to anyone else:
It has an optional stun command which will hit a stuntman server I am running to get your external IP address and port as viewed by a remote entity. (In bitcoin core we can get this info from our peers, if we don’t already know it).
Swapping this info with a friend (or second machine) and using the peer command at ~ the same time on both instances will attempt a simultaneous connection.
stdin is sent to the other side, so you can send characters, files etc.
We had pretty decent results in testing, although we occasionally had to wait a few seconds for router firewalls to forget about their mappings (when we changed ip addresses but not ports), and were thwarted by one hotel NAT configuration.
As noted in the readme we also tried from within various types of containers, over starlink, through a VPN, and all of those chained together, and still saw success.
Thanks for this. It’s a topic I’ve also been interested in regarding node reachability although from having messed around with other hole-punching tech like Iroh.
I’ll take some more time to read up, but just wanted to share the results running your script:
On my home internet connection:
IPv4: EIM;
IPv6: no NAT
I’ve been thinking about this a bit and think something like this might work:
A Bitcoin node run at home might not be reachable via clearnet due to NAT, however, many home nodes now support Tor and/or I2P, which allows inbound connections via Tor. It seems possible to coordinate TCP hole punching through Tor or I2P.
A node might want to offer inbound slots via clearnet, but is not reachable. One way to work around this could be the following protocol with node A and node B both being nodes behind a EIM NAT, while also being connected to the Tor and/or I2P network.
Node A thinks it’s unreachable (TBD how to figure this out reliably) but wants to offer clearnet inbound connections.
It first needs to figure out if it’s behind a EIM NAT. It can do so by opening and establishing two (possibly feeler?) connections to other nodes using SO_REUSEADDR/SO_REUSEPORT on the socket and checking if the peers return the same port in the version message. This needs to be done only once per network, assuming the NAT configuration does not change.
Node A starts to listen on a dedicated hole-punch coordination Tor or I2P endpoint only for coordinating hole-punching. Address, transaction, or block relay is not supported on this endpoint. We don’t link any other Tor or I2P endpoints that this node might have to this dedicated endpoint.
This coordination Tor or I2P endpoint is advertised via addrv2 message on clearnet only (to not link clearnet and Tor/I2P address) with a (new) service flag: e.g. NODE_HOLEPUNCH(?). These addresses are relayed as a best effort, but not stored in addrman. This has the goal of them not being relayed after around 10 minutes and not using up space in addrman. The frequency we relay these is TBD. We don’t need to self-announce them, if we don’t want any more holepunch-inbound connections. This message could also be a new BIP-155 address type for hole punching which could include the Tor or I2P coordination endpoint and the clearnet address (coordinate on abcdef.onion to connect with 203.0.113.24) the nodes wants inbound connections on. This allows other nodes to filter by e.g. netgroup / AS without needing to make a connection to the coordination endpoint.
Node B receives such an announcement via an addrv2-message. It decides it wants to try to open an outbound connection to Node A. It has previously figured out that it’s behind a EIM NAT.
Node B connects to node A’s hole-punch coordination endpoint, and does a version handshake. As part of the connection to this hole-punch coordination endpoint, a request for a hole-punch connection is implicit.
Node A now opens a new outbound connection (e.g. a feeler?) to a known good address with SO_REUSEADDR/SO_REUSEPORT on the socket. The goal is to do a version handshake and learn the NAT IP:port of the socket. This likely requires some rate-limiting for some external party to cause Node A to make too many outbound connections.
Node B does the same.
Node A & B now have a EIM NAT mapped socket they can use to do the simultaneous TCP open and punch through their NATs.
A few notes on how to make this easier, but not without introducing tradeoffs:
With EIM NATs, we might be able to predict our NAT port (it’s often the same we bind on locally, at least from my limited observations). This might allow us to skip the outbound connections Node A and B need to make while already coordinating, making this a lot easier, faster, and causing less churn on the Bitcoin network. This will fail, when our NAT has already a mapping for the same port. The tradeoff here is possibly higher failure rates.
An alternative might be to have well-known, static, (and centralized!) Bitcoin-protocol-speaking-servers-but-not-nodes run on the P2P network that just do a version handshake returning the IP:port they see your NAT IP from and then close the connection. These might be run by community members, similar to the DNS seeds. They’d learn about who is trying to open a hole-punch connection, and a passive observer (ISP) would see you making connections to them.
Yet another alternative is that all nodes on the network would allow a special IPv4/IPv6 NAT check connection to them where it’s implicit that the connection will be shut down right after the version handshake. No need for them to be centralized, but would basically require implementing a STUN service for the P2P network in node software (e.g. Bitcoin Core).
Compared to the approach with a coordinator, this has the benefit of having clear inbound-outbound mechanics. Node B chooses Node A as an outbound. Node B is an inbound to A. The downside is that we need connectivity over Tor/I2P/(CJDNS). However, many node-in-a-box home nodes seem to ship with Tor / I2P on by default.
A description of a reasonable design (that others came up with) for the Bitcoin P2P part of the rendezvous protocol that is over clearnet and requires a third coordinating node, but that might get a real outbound rather than something mutual-inbound:
Alice wants to receive inbound connections to her Bitcoin node but is behind NAT.
Alice learns through her ADDRMAN that Charlie runs a node that offers RENDEZVOUS services and opens a RENDEZVOUS-only connection to Charlie.
Alice uses a new ADDRv2 network type to advertise that she can be reached through Charlie.
Bob is looking for nodes to make an outbound connection to and decides to connect to Alice via RENDEZVOUS with Charlie.
All three use the procedure described above to attempt a TCP-hole-punched connection from Bob to Alice.
If Alice still has inbound slots available she reconnects to Charlie and awaits new connections.
I am having a hard time reasoning about whether or not there is any trust assumption in Charlie, or if Charlie has any power he wouldn’t have had otherwise. Since Charlie does get to decide whether or not Bob’s connection to some chosen Alice succeeds iff Charlie is the rendezvous, Charlie seems to have some sway over Bob’s outbounds, but how is this any different than if Charlie had advertised addresses that don’t work?
If there is some trust in Charlie, this can be avoided by treating all addresses Charlie or routable via Charlie as one address, so Bob will make a random choice of e.g. Charlie to connect to then a second choice randomly from the pool of Charlie+Routable_via_charlie. In this case Bob can treat the connection as a real outbound, this achieves the effect of increasing the number of inbound slots on the network, but I think that the number of independent parties n on the network that you have a 1-of-n assumption for when bootstrapping onto the network remains the same.
And I think that the Tor/I2P approach described above is an idealized case of this where each node can serve as it’s own RENDEZVOUS, but this would still have the problem that you have to be picking/trusting the tor/i2p rendezvous address, not the destination address, since an attacker could be spamming the network with fake/bad relays for real addresses.
Reading up on the discussion here - hole punching seems like an interesting idea to increase network connectivity and robustness to network adversaries.
A campus network that I used for testing had APDM and varying external IP addresses. IPv6 was disabled.
This recent paper claims a ~70% success rate with both TCP and QUIC (UDP-based transport) using libp2p’s DCUtR.
Thinking ahead if UDP hole punching proves to be much more effective, has there been any discussion in implementing QUIC or a custom UDP-based transport in core?
I wonder if this “hole punching” also works for VPN connections, making the node semi clear-net somewhat private. (Proton has some special NAT options)
I like this approach without a central coordinator via TOR / I2C. Don’t think it’s the easiest option to get working.
A custom UDP-based transport sounds like a very ambitious task. The P2P protocol is inherently stateful in many little ways; there is no real reason why it would need to be, but right now it is, and changing the application layer to drop statefulness would be a huge undertaking. So, directly running the existing application protocol over UDP instead of TCP wouldn’t work; duplicates, dropped messages, and out of order delivery would break things.
That means the options here are effectively either using QUIC, or a custom wrapper that reinvents it. I’m not familiar with how complicated the implementation of this is, but I suspect it wouldn’t really be worth the trade-off.
I’m somewhat hopeful that just TCP hole punching can give us a significant part of the benefits already though, knowing that it’s more fragile and less supported than UDP hole punching.
On @0xB10C‘s approach of pairing clearnet addresses with an externally reachable Tor service endpoint by the same node for establishing clearnet connections: it seems like it would be fine for nodes that do not also run a P2P Tor service. It makes me wonder if it could increase chances of an adversary in correlating addresses when a node which also runs a P2P Tor service shuts down though. Maybe it’s already bad. Having nodes create new Tor services at the same interval as the new ephemeral addr messages would maybe help.
Regarding building a TCP-like protocol over UDP as touched upon by @cedarctic & @sipa I agree it is a major undertaking, but probably a smaller lift than for example multiprocess support being worked on in Core. It still needs champions with a lot of grit though. Maybe the adversarial aspects is what makes people pessimistic? I’ve only seen such protocols used in less adversarial environments, in closed source.
Having the ability to bypass strict ordering / stream semantics on a per packet basis would be at least somewhat beneficial for things like compact block propagation.
Agree on attempting TCP hole punching first though.
Not sure how relevant, but one thing I’ll add is that NATs with EIM may still have “Address and Port-Dependent Filtering” (APF). Meaning that EIM:APF ↔ APM:APF won’t work.
Docker desktop on Mac using bridged networking on office internet
APDM
Failed to connect
Linux Laptop on office internet
EIM
Failed to connect
Docker using bridged networking on Linux laptop on office internet
EIM
Failed to connect
T-Mobile US hotspot
EIM
EIM
Home router (OPNSense)
APDM
No IPV6 is configured
*Update, there is no IPV6 at the office or on T mobile hotspot so the V6 results are wrong. I think Mac is being helpful and connecting over IPV4 as a fallback.
Iroh has a 90%+ holepunching rate, and 99%+ connection rate, but uses federated relays similar to a webRTC STUN & TURN servers to achieve these numbers, which I’m assuming is not entirely appropriate for bitcoin. With that said, we’ve spent a lot of time on the topic, Including integrating with TOR, encrypting the exchange of candidate addresses, and employing novel techniques for holepunching itself.
ill keep an eye on this thread & can try to pull in experience & figures where useful!
@0xB10C Perhaps it’s overkill, but when applying the hole-punching technique to “No NAT” situations (incl. IPv6 where it seems somewhat common), connections look like they succeed as well. That’s expected of course, but means hole punching could be used for those even when a firewall or other configuration issue prevents publicly opening a port. So I think you could show those as bold in the results table as well.
The sidecar script connects to the matchmaking server and will be matched with a peer by the server. one side is assigned initiator and one reciever. They then holepunch each other and initiator runs a bitcoin-cli addnode command (automatically or manually) to initiate the connection. The connection is proxied via the sidecar on both sides, so both nodes see a local (127.0.0.1) address.
This seems to work pretty nicely, but is reliant on a 3rd party server for “matchmaking”.
The default server is running with a signet client connected, in case anyone wants to test it out (using something like ./contrib/holepunch/sidecar.py --cli-command "bitcoin-cli -signet" --bitcoind-host "127.0.0.1:38333" --network signet).
We’ve picked up work on this again a bit to come up with an LLM spec/BIP and an implementation in Bitcoin Core to experiment with it a bit more. Will share more on this later. First, I wanted to recap the (four?) approaches that were discussed here and share some thoughts:
All four approaches share can share the same rendezvous (you can find me here) and punch (start the hole punching now) messages.
Third-node coordinator: A announces “find me through C” to the network. B picks that announcement, asks C for a rendezvous with A, and C sends punch to both. B chose A, and A opted in to being found through C. C sees a pairing happened but nothing else. The network learns that A and C are/were connected, which might leak network topology (which is not great).
The advertiser’s own Tor service: A runs a second, unadvertised onion service whose only job is this, and announces “find me through this onion”. B reaches it over Tor, asks, and A answers with punch itself. No third party C. Tor only carries the request, the punched connection is clearnet. Needs Tor on both ends, and the onion identity has to be replaced whenever A’s address changes so it can’t be used to follow A across ISP-assigned addresses. Does not leak network topology.
Matchmaking pool: Nodes tell a coordinator “put me in your pool”, and the coordinator pairs up two members on its own. Here, nodes don’t choose who they connect to, and both sides of the connections are essentially “inbound”, which is weird and not ideal.
Handoff instead of eviction: When a full node has to drop an inbound peer, it first introduces that peer to another of its inbound peers so the two connect directly, then drops it.
In 1 and 2 every punch is the answer to something the receiving node asked for, so a node only ever dials a peer it chose (B picked A) or agreed to be found by (A opted in through C). In 3 and 4 the node doesn’t choose. The coordinator decides who it connects to, and the punch arrives unasked.
A question I’ve been thinking about more generally: While this feature sounds and looks very cool, what problem does it actually solve? And do the different approaches solve different problems? Which one of them is actually worth the work that would need to go into this?
In general, is accepting (random) inbound connections actually desirable for most nodes? Being able to connect to a victim via multiple inbound connections can be a useful tool for attackers. Maybe for people who don’t go to the trouble of having a publicly routable IP address, not being reachable is actually mostly a net benefit, even though it makes the p2p graph less uniform?
Being able to connect directly to a known node (either your own or a friend/associate’s) despite NAT or changing IP addresses seems useful, though. But that probably wants authentication, and maybe a VPN or similar is more appropriate at that point?
On the other hand, perhaps having more short-lived connections with random peers (syncing headers, blocks, addresses, top of mempool) would be valuable without adding much danger (since an attacker no longer has much control over the connection’s timing) – in that case, connecting to a random public node and saying “hey introduce me to someone” (recursively?) might make more sense.
One framing of the problem: “Honest nodes are not as reachable as attacker-controlled nodes. This gives attackers an extra advantage when trying to occupy outbound connection slots.”
With that framing, I think having some sort of holepunching would be very valuable. Option 2 seems like a good start, given that the % of nodes advertising a Tor address (already have Tor working) is quite high.
I came up with option 6: coordinator-less hole punching
Every option so far needs a coordinator, because two nodes behind NATs have to (a) learn each other’s IP:port and (b) agree on when to fire. In option 6 addresses come from normal addr-relay, and holepunch timing is derived from the next blocks (e.g. header timestamp) that arrive.
A node that wants to hole punch announces a new addrv2 type with its hole-punchable IP:port and an ephemeral EC public key P. Nodes keep their ephemeral private key p.
Nodes receiving it store the address and calculate a ECDH shared secret with P_other and p_own.
On every new block, each node computes SHA256(tag || shared_secret || block_hash) for every entry it had received before the block. It sorts the resulting hashes and keeps the few lowest. Those are the peers it tries this round.
For each candidate, it calls connect() to that IP:port from its punch port and leaves the socket pending.
Both sides derive the same shared secret, which also causes them to derive the same hash. Two peers find themself through being their sets of the lowest hash candidates. The timing is derived from the block header (exact way is tbd) and the same for both nodes. We don’t need the timing to be exact. During testing, a few seconds plus or minus didn’t matter. So clocks can be slightly out of sync.
The (next) block hash is used to stop an attacker from grinding. Without it, someone with my P could try private keys until their hash is tiny and be my top pick forever. When using the block hash as salt, there’s nothing to grind, since the block doesn’t exist yet when they publish their key. Nodes just have to ignore blocks they heard about before receiving the address. Privacy-wise, two public keys don’t reveal the shared secret, so an outside observer can’t tell who connects to whom or when. Participation is public though, since both sides publish an IP:port.
Both sides have to announce and have each other’s address, so success depends on how widely these spread. Additionally, it needs careful design of an holepunch address manager and rate-limiting to mitigate an attacker flooding the node with fake addresses or public keys that will never result in a holepunch connection.
Wouldn’t it also be effective for an attacker to flood with real addresses/public keys, to attempt to monopolize all connections made via this mechanism?
To maintain 8 connections to 100k peers, you’d need ~1M active ports; but for the actual address sharing step, I think you’d likely only need to advertise ~10k ports (800k connections cycling once a week is about 800 per block, so with 10k attacker addresses advertised at any one time, you should be an early match against each victim – about 85% of honest nodes will have the top two matches be attackers, in that scenario; and even when that happens, the odds of the other peer not having an attacker’s node at a higher ranked seems very low?
Yes, that’s the main downside we discussed in-person too. I’m lagging behind a bit with writing up the current ideas here (still traveling).
For sharing addresses between two home nodes, idea is to use a “message board” instead of “normal” address relay. Listening nodes can announce a service bit that they are acting as message boards for holepunching peers (next to being a normal node). Home nodes connect to them (probably via short-lived connections; but maybe long-lived is fine too) from the same port as they would do the hole puch and tell the message board about their IP. They don’t do any other relay of these connections. The board only adds the home node IP:port if it matches what it sees (protects against someone spamming the boards with fake IPs). The board also tells the home node about the IPs of the other home nodes that announced to it. If we use short-lived connections, a home-node probably has to refresh it’s own IP periodically to avoid the entry becoming stale and to learn about other home-nodes that registered on the board after it did. Board entries aren’t relayed (other than to the home-nodes who’ve chosen a node as message board). A home node chooses multiple boards from it’s addrman. The hope is that at minimum one choosen board is honest (same as home-nodes currently hope that one choose node is honest).
Then, the method described above for selecting deriving who to connect to and when happens. My instinct is that this should be more reliable than address relay. We don’t need to rely on addr-relay working well for good home node endpoints to spread across the network symmetrically (we need the peers endpoint and the peer needs ours) and it’s significantly more expensive for someone to produce failures with fake peer-endpoints: You either need a lot of IPs or you need to control many boards (in distinct netgroups / AS with asmap).
IPv6 needs a special rule as it’s cheap to get many IPs.
There’s more details in this LLM-written spec, which I don’t think is ready for human consumption yet.
In that case, honest message board nodes aren’t helping each other out, so an attacker will probably advertise the same ips/ports on every message board, while honest nodes will only appear on a few, which seems like it gives an additional multiplicative advantage to the attacker? (Attacker controlled message boards will presumably just give connections to the attacker either way ofc)
I think there’s probably two high-priority goals we should consider for p2p connectivity:
high probability of having a connection to an honest node (to avoid censorship)
low probability of having a connection to a particular attacker (to avoid spying, denial of service, or targeted exploits)
Obviously if our code and p2p protocol is perfect, and there are no vulnerabilities or privacy leaks possible from a peer connecting to our node, then (2) isn’t worth worrying about; but I don’t think that’s how things are today.
I think hole punching has a high risk of making the second goal worse – making it easier for an attacker to get a direct connection to more honest nodes. I don’t think the message board does much to improve the first goal – if you can find an honest message board node, generally you could just stay connected to it for blocks and txs; and if you can’t find an honest message board node, you’re screwed anyway in this model?
To be clear, that feature defaulting to on also makes goal 2 worse, as far as I can see. I think we haven’t seen much increased reachability as a result of that change, so I guess I’m thinking about that goal as something of a revealed preference from Bitcoin users, and seeing if it seems rational after the fact.
(writing down WIP thoughts here. Still thinking about how the interaction with a board could look like).
Both short-lived and long-lived connections have trade-offs. Periodical short-lived connections would refresh a home-node IP in a board periodically by connecting+handshake, sending it’s address, and receiving addresses from the board. A long lived connection to the board would stay open for as long as the home-node wants to receive or make a connection. These connections are very light-weight and only consists of ping/pong as well as holepunch-addr-relay (board telling the home node about candidates to punch with).
On long-lived connections, the boards can push newly registered home-nodes to connected nodes shortly after they registered. With a short-lived connection, a home node has to register with the board and fetch from the board, wait until other nodes fetched its entry from the board, and then re-fetch entries from the board. This makes short-lived connections generally slower in finding endpoints, they require more connections to succeed (which might be a problem if inbound slots on boards are full), require slightly more bandwidth (one handshake+ping/pong vs multiple handshakes), and short-lived entries might be stale more often as long-lived connections can be removed from a board shortly after the connection drops. Short-lived ones need to remain on the board for some X minutes until the next reconnect is scheduled (+ maybe some grace).
With a home-node making short-lived connections, you also need to keep the NAT and firewall mapping alive (routers might expire this after a few minutes of inactivity). This can be done by making one (or all) outbound connections from the hole punch port.
Long-lived connections are more costly to board nodes, so boards might limit how many to maintain. Short-lived ones are cheaper and a board can store many more candidates (up to a limit), which might be better for finding a more diverse set of peers, but worse in case an attacker attacks a board?
I have a slight preference for long-lived connections at the moment, as they make hole-punching more reliable and raises the cost for an attacker a bit (one connection every N minutes is slightly cheaper than maintaining a connection for longer).
My current thoughts are: (Assuming only IPv4 for now and long-lived connections), an attacker can control as many entries on all boards as he has access to IPv4 addresses he can sustain long-lived connections from. An home-node would want to sample candidates from a few boards, and deduplicate by IPv4. Boards and home-nodes would want to also limit by netgroup/AS, which makes it slightly more expensive for an attacker (as his IPs need to come from different netgroups/ASNs) with the trade-off of causing some netgroups/AS with many home-nodes to fail more often (or we could also use the netgroup/AS in the shared secret).
Today, the way for an attacker to get more direct connections seems to be to run more nodes accepting inbound connections in a more diverse set of netgroups / AS (as some entities like e.g. this, this, or this seem to be doing already). Accepting and maintaining long-lived inbounds on many IPs is more expensive for an attacker than making long-lived outbound connections (to boards and home nodes) or accepting hole-puches. So I tend to agree. Would treating them as “inbounds” (in terms of fast we relay) on both sides help? I guess only for the spy-angle. They’d still be able to DoS and otherwise exploit.
One thing that came up in an in-person discussion was: Today, the network partly relies on nodes in Hetzner, AWS, Google Cloud and Digital Ocean to offer inbound connections. If these don’t want or can’t allow you to run nodes on their platform anymore, then there’s a need for something that connects home-nodes together. They create direct edges between two home-nodes that currently are only connected via other’s. Hole punching would change topology from a hub topology (where the hubs are the reachable nodes in data centers) to a more “mesh” topology.
With fewer honest nodes being able to accept inbounds than today (e.g. ban on Bitcoin nodes in data centers), it might actually be preferable to have hole-punching to address the anti-censorship goal. You only need one board that has a few honest home-nodes for the hole punching to work at some point. The board node might not have enough resources (bandwidth or CPU) to handle the new need for connection slots to serve blocks and txs to, but can share addresses of home-nodes in a low-bandwidth fashion, which then have CPU and bandwidth resources for inbound slots.
I agree that on the other hand we want to avoid a situation where e.g. “Sorry honey, we can’t watch Netflix right now, someone is DoSing our internet by downloading blocks from my hole-punch home node.”…
My idea was to have a board node mode in Bitcoin Core, so each normal listening node eventually (when and if on-by-default) can act as a board. However, I think there’s nothing stopping people from running other software that acts a board. Maybe even with higher limits and performance, as you don’t need to handle transaction and block relay, and care a bit less about resource usage. Along the way, we should probably make sure to limit what other non-Bitcoin infrastructure can use these boards to store information.. (even if they hard to maintain an open connection for storing an address and 32 bytes of data).
It probably makes sense to rerun these numbers to see if things changed now. However, I’m not sure it’s fair to come to the conclusion that Bitcoin users prefer their home nodes to be unreachable. Checking if more reachable nodes in ISPs vs cloud show up measures router and ISP compatibility and not user consent or preference.
I guess I’m more worried about home ISPs preventing you from running a bitcoin node than all data centres doing so – it’s much harder to get a new ISP (or move house) than it is to fire up a different VPS; and it’s likewise potentially more damaging if a p2p exploit compromises your home network rather than a random VPS. The VPS provider is likely getting explicitly paid by the node operator as well, unlike the ISP who might figure “they’ll keep paying for netflix and facebook”.
I think of the topology today as two layer – a mesh of listening nodes each connected to each other, with end-points having long-term connections to a handful of random points within that mesh. Listening nodes get better performance vs higher risk (and to a small extent higher cost), while end-points get the opposite. Attackers and spy-nodes can occur at either layer, of course.
I think having more short-lived connections (perhaps lasting a few hours or a day) relaying blocks and transactions might be an interesting trade-off in this space – it reduces the risk of an attack somewhat (the attacker has to get lucky repeatedly), makes the network topology more dynamic (so attacks relying on knowing the topology are harder), and perhaps equalises the behaviour of high-uptime nodes vs new nodes which might be beneficial. Not really sure how to analyse/think about that in a concrete way though.
Possibly we should think of “outbound” and “inbound” with “chosen-partner” (I don’t think this guy is an attacker, so can prioritise them higher) vs “just-a-stranger”; so pairings might be something like:
For option 6 without a board but relying on addr-relay of addrv2(a, P) self-annoucements as mentioned in #22:
When looking into address-relay a bit more in a simulator (unrelated to hole-punching), I noticed that home-nodes (or unreachable nodes in general) receive self-annoucement far less reliable than listening nodes. Listening nodes have a lot more connections that can tell it about a self-announcement. Home-nodes only have 8 connections that could relay an address to it. We only forward an address to up to 2 peers (both also ask for getaddr on their outbound connections).
Since option 6 relies on addr self-announcements arriving at both home-nodes, I think it’s probably going to be very unreliable.
The addr relay algos we use currently wouldn’t be suitable anyway – they’re designed to prioritise long-lived addresses (announced infrequently, gets remembered/reused for a long time), whereas hole-punching data is extremely short-lived and becomes invalid after the first successful connection.
Ah, that’s cleverer behaviour from SO_REUSEADDR/PORT than I was expecting.
I was thinking the scenario for honest peers was “I’d like an additional connection, I’ll use new port number N for it, and try the holepunch-y scheme. [five minutes later] Great, it worked! No more holepunching for me for now!” which only leaves a small window for an attacker to make the connection. Doesn’t help if the attacker is constantly trying to make connections, while honest nodes are only trying rarely, but that is hard in general without adding a cost to making connections (money or PoW) or adding identity.
If you do have ports open long-term for multiple successful holepunch connection, I suppose you have to split messages into listening and connecting pairs – otherwise mapping long-term listening ports to each other, you’d just have them fill all their connection slots up with other long-term peers, and new peers would have no one to connect to.
For an attacker trying to connect to an honest peer behind a NAT, I guess the process would be:
find their announcement
publish 100 or 1000 holepunch requests in the same place
figure out which ones are at the top of the honest peer’s priority list and attempt connections via those IPs/ports
keep trying every block until a connection goes through
which still seems like it makes it a bit too easy for spys/attackers to have a direct connection to your node.
The thing that seems most compelling to me is being able to hole-punch to your own node (eg to get trusted data, to avoid random nodes spying on your requests, or to query indexes that aren’t widely available), so adding two-way auth to p2p, and adding a way to interactively trigger/coordinate hole-punching would be better. I’m not really sure an intermediary is desirable there though, so maybe that’s really just “connect via tor/i2p; authenticate; switch the connection to direct tcp via holepunch to improve performance”.
Have not fully thought this through, but wanted to note it here:
A few ways of raising the cost against an attacker that wants to fill up slots at a board came to mind:
generally, you limit number of slots on a board
when full, the board evicts connections based on a score. The score consists of e.g. hash(board IP | node IP | date). A “more zero-bits” score is preferred over a “less zero-bits” one.
honest nodes can calculate the boards they should try to connect to per day to not to get evicted
An attacker can try to spam and connect with his IPs to multiple boards, but will get evicted by honest nodes over time. To get more slots at more boards, an attacker needs to buy more IPs. As an observer, you can tell that an attacker-board is likely full of attacker-controlled IPs, as the IPs the board offers are all (compared to an honest board) “less zero-bits”.
One down side: that this allows (if you know which home-node IPs are currently looking for a connection) to guess who might be connected to which board. This also gives you a set of home-nodes potentially opening a connection. But everyone at a board can already learn about who else is at a board.
Another one: As home-nodes fill up their slots, less of them are going to be present on boards. More room for attackers.
Concretely: in the context of bugs like CVE-2026-117350 (tcp stack bug in bitcoin), CVS-2026-117349 (protocol bug in bitcoin) and CVE-2026-98432 (tcp bug in linux), isn’t it more sensible for home users not to be listening for incoming connections, as a precautionary measure?
0xB10C · 
wifi (Viasat)
will · 
Pieter Wuille · 
dunxen · 
David Gumberg · 
cedarctic · 
janb84 · 
Hodlinator · 
Carl Dong · 
Max Edwards · 
Eugene · 
Brendan · 
all, first time poster, I work on iroh, a UDP-based holepunching library built on QUIC: 
Anthony Towns · 
Jonathan Harvey-Buschel · 
chosen-partner
just-a-stranger