BIP 155: addrv2 BIP proposal

laanwj commented at 9:33 am on February 27, 2019: member

Proposal for wider v2 address messages.

Last time this went around (as a github gist) there were some considerations still to be discussed, for lack of a working mailing list I’ll post them here:

Considerations

‘‘Client MAY store and gossip address formats that they do not know about’’: does it ever make sense to gossip addresses outside a certain overlay network? Say, I2P addresses to Tor? I’m not sure. Especially for networks that have no exit nodes as there is no overlap with the globally routed internet at all.
Lower precision of time field? seconds precision seems overkill, and can even be harmful, there have been attacks that exploited high precision timestamps for mapping the current network topology.
- (gmaxwell) If you care about space time field could be reduced to 16 bits easily. Turn it into a “time ago seen” quantized to 1 hour precision. (IIRC we quantize times to 2hrs regardless).
Rolling port into addr, or making the port optional, would make it possible to shave off two bytes for address types that don’t have ports (however, all of the currently listed formats have a concept of port.). It could also be an optional data item (see below).
(gmaxwell) Optional (per-service) data could be useful for various things:
- Node-flavors for striping (signalling which slice of the blocks the node has in selective pruning)
- Payload for is alternative ports for other transports (e.g. UDP ports)
- If we want optional flags. I guess the best thing would just be a byte to include the count of them, then a byte “type” for each one where the type also encodes if the payload is 0/8/16/32 bits. (using the two MSB of the type to encode the length). And then bound the count of them so that the total is still reasonably sized.

addrv2 BIP proposal ec8b400259

luke-jr added the label New BIP on Mar 29, 2019

luke-jr renamed this:
~~addrv2 BIP proposal~~
BIP 155: addrv2 BIP proposal
on Mar 29, 2019

luke-jr commented at 6:05 am on March 29, 2019: member

Assigned BIP 155

laanwj cross-referenced this on Jun 15, 2019 from issue [Feature] Support i2p by ghbjklhv

bip155: fill in BIP number 34534b5ee1

bip155: Fix comments URL e993478060

bip155: Add table entry f5174192e2

laanwj commented at 10:39 pm on July 17, 2019: member

Thanks! Filled in the BIP number and fixed the travis issues, this should be ready for merge I think?

in bip-0155.mediawiki:51 in f5174192e2

46+It is serialized in the standard encoding for P2P messages.
47+Its format is similar to the current <code>addr</code> message format
48+<ref>[https://bitcoin.org/en/developer-reference#addr Bitcoin Developer Reference: addr message]</ref>, with the difference that the 
49+fixed 16-byte IP address is replaced by a network ID and a variable-length address, and the time and services format has been changed to VARINT.
50+
51+This means that the message contains a serialized <code>std::vector</code> of the following structure:

MarcoFalke commented at 12:30 pm on July 18, 2019:

I know this is how bips were written in the past, but I’d slightly prefer not to leak cpp details or Bitcoin Core implementation details such as std::vector (can be replaced by “list” or “vector”), pchCommand (can be replaced by “message type”), and static const int GOSSIP_ADDRV2_VERSION (can be omitted without loss of information).

If someone is eager to look at the cpp implementation, there is already a section that links to it.

laanwj commented at 1:47 pm on July 18, 2019:

fine with me

laanwj commented at 1:55 pm on July 18, 2019:

I think I went with explicit C++ types here for clarity as to how things would be serialized, after all there is not really a spec of the serialization format (and how the certain structures are named, like “how to serialize a list”, wouldn’t “a list” be vague and ambigious?) besides the implementation

MarcoFalke commented at 2:08 pm on July 18, 2019:

Our python implementation calls it “string” https://github.com/bitcoin/bitcoin/blob/e5abb59a9a66723dd1d9a01f65e467636eb24f2d/test/functional/test_framework/messages.py#L89

Anyway, no strong opinion :sweat_smile:

laanwj commented at 7:59 pm on July 31, 2019:

string seems to be worse as it implies a human-readable text

braydonf cross-referenced this on Jul 18, 2019 from issue Failed to use TOR-proxy by kFTY

in bip-0155.mediawiki:135 in f5174192e2

130+==Compatibility==
131+
132+Send <code>addrv2</code> messages only, and exclusively, when the peer has a certain protocol version (or higher):
133+<source lang="c++">
134+//! gossiping using `addrv2` messages starts with this version
135+static const int GOSSIP_ADDRV2_VERSION = 70016;

MarcoFalke commented at 6:22 pm on July 18, 2019:

I haven’t put much thought into this, but could you explain the tradeoffs here a bit? I think there are two ways to announce support for a new p2p feature:

Bump the protocol version
Send a p2p message to announce your support for the feature

Bumping the minimum protocol version seems to imply that nodes would have to support all protocol features that were introduced prior (like compactblocks, feefilter, …)

Whereas with a new protocol message, even mobile wallets could make use of the new feature?

laanwj commented at 8:00 pm on July 31, 2019:

~~it does add a new protocol message (that’s the addrv2), but the version tells it that the other side understands them~~

would you prefer to add another protocol message to signal support, which is always sent after version?

MarcoFalke commented at 10:03 pm on July 31, 2019:

No, you can keep this as is. mobile wallets tend to get their addresses from dns seeds, so this shouldn’t matter.

MarcoFalke commented at 8:09 pm on October 18, 2019:

In light of yesterday’s IRC discussion and on further thought, I think support for addrv2 should be announced by a service bit. This has the following advantages:

leech nodes (nodes that only want to consume addrv2 messages, e.g. “SPV clients”) can find addrv2 nodes by the service bit and they can expect them to respond to sendaddrv2 messages
address relay between address relay nodes (e.g. full nodes/Bitcoin Core nodes) can be made explicit, instead of relying on heuristics such as NODE_NETWORK. See https://github.com/bitcoin/bitcoin/pull/17163#issuecomment-543072194

dongcarl commented at 8:16 pm on October 18, 2019:

* address relay between address relay nodes (e.g. full nodes/Bitcoin Core nodes) can be made explicit, instead of relying on heuristics such as `NODE_NETWORK`. See [bitcoin/bitcoin#17163 (comment)](https://github.com/bitcoin/bitcoin/pull/17163#issuecomment-543072194)

To be clear, you mean address relay can be made explicit by an additional service bit, e.g. NODE_ADDR_GOSSIP or something like that?

MarcoFalke commented at 8:45 pm on October 18, 2019:

To be clear, you mean address relay can be made explicit by an additional service bit, e.g. NODE_ADDR_GOSSIP or something like that?

Yes. Currently we use heuristics such as NODE_NETWORK for that, which is fragile and not future proof. For example, pruned nodes may not have any NODE_NETWORK* bit set. See also https://github.com/bitcoin/bitcoin/pull/17194

in bip-0155.mediawiki:60 in f5174192e2

55+!Name
56+!Description
57+|-
58+| <code>VARINT</code> (unsigned)
59+| <code>time</code>
60+| Time that this node was last seen as connected to the network. A time in Unix epoch time format, up to 64 bits wide.

MarcoFalke commented at 9:38 pm on July 18, 2019:

Would help to explain what “last seen” means, since it appears easy to get wrong. E.g. https://github.com/bitcoin/bitcoin/pull/5860#issuecomment-78397140

laanwj commented at 8:02 pm on July 31, 2019:

The meaning of this field doesn’t change from the current implementation. I’m fine with adding an explanation though, if you supply it, but I’m not sure what to add here.

luke-jr merged this on Jul 23, 2019

luke-jr closed this on Jul 23, 2019

laanwj commented at 3:36 pm on July 31, 2019: member

Came up during discussion on IRC, need to be explicit about representing IPv4 as network ID IPV4, that the old wrapped representation is no longer allowed:

==Appendix X: IPv4 address encoding== Clients MUST send IPv4 addresses with this network ID, with the 32-bit address in the address field.

Clients SHOULD ignore IPv4 Mapped Address (::FFFF:0:0/96) addresses on receive if they come with the IPV6 network ID.

(thanks @dongcarl)

laanwj commented at 8:16 pm on July 31, 2019: member

As for changing the signaling from protocol version-based to sending a message. I think this message, when sent from one peer to another, would signal support for the first peer to receive addrv2 messages from the other one. So something like sendaddrv2? (like bip130)

I wish we had some kind of standard message for this.

I think for simplicy of implementation we’d want to mandate that this should be sent after version but before any other (non-BIP-signalling) messages, so a connection won’t switch to v2 suddenly halfway other things.

Edit2: This sendaddrv2 message could also contain wider addresses that doesn’t fit in the version message. e.g. addrMe, the TorV3 address that we used to connect to the peer. The version message would contain a dummy address (localhost) then.

laanwj commented at 8:04 pm on August 7, 2019: member

There was some discussion on IRC that there should be a max cap for network addresses, but it should not be 256 bits, but more something like 512 bytes, to accommodate future extensions but have some kind of limit for DoS. Currently it says:

Field addr has a variable length, with a maximum of 32 bytes (256 bits). Clients SHOULD reject longer addresses.

Change this to: a maximum of 512 bytes, and the whole message (not just the single address) SHOULD be rejected if it contains any larger item.

dongcarl commented at 3:19 pm on August 22, 2019: none

@laanwj Just to be clear, addrv2 entries that claims they are IPv6 but come with either the IPv4-prefix, OnionCat-prefix, or CJDNS-prefix should be ignored?

For Bitcoin Core, I’m assuming our weird g_internal_prefix would also be added to that ignore list?

dongcarl commented at 8:03 pm on August 23, 2019: none

As for changing the signaling from protocol version-based to sending a message. I think this message, when sent from one peer to another, would signal support for the first peer to receive addrv2 messages from the other one. So something like sendaddrv2? (like bip130 )

I wish we had some kind of standard message for this.

I think for simplicy of implementation we’d want to mandate that this should be sent after version but before any other (non-BIP-signalling) messages, so a connection won’t switch to v2 suddenly halfway other things.

Edit2: This sendaddrv2 message could also contain wider addresses that doesn’t fit in the version message. e.g. addrMe, the TorV3 address that we used to connect to the peer. The version message would contain a dummy address (localhost) then.

It would seems that that addrMe has been deprecated: https://github.com/bitcoin/bitcoin/pull/8740

dongcarl cross-referenced this on Aug 28, 2019 from issue [WIP] Add support for addrv2 (BIP155) by dongcarl

naumenkogs commented at 5:24 pm on October 22, 2019: member

‘‘Client MAY store and gossip address formats that they do not know about’’: does it ever make sense to gossip addresses outside a certain overlay network? Say, I2P addresses to Tor? I’m not sure. Especially for networks that have no exit nodes as there is no overlap with the globally routed internet at all.

I think relaying unknown networks is very important, because the nodes are often multi-homed. So this might be actually useful for somebody who your node thinks is simply ipv4. Also, I believe it would be easier for an attacker to attack, if we don’t relay these at all. Lastly, depending on the implementation, this specific behavior might leak that you do support Tor, while you look ipv4 to an attacker.

There is no reason to not. If an attacker wants to spend our bandwidth, might as well flood with ipv4, we don’t check if the node is alive before forwarding them.

Storing is another thing. With the current rate-limiting, the bandwidth we’re spending here is pretty low. But addrman size is bounded, so I wouldn’t store everything. Maybe, storing a couple exotic things might make sense if you will be switching in future.

vasild referenced this in commit 4e7714b8a8 on Apr 10, 2020

vasild cross-referenced this on Apr 10, 2020 from issue BIP155: include changes from followup discussions by vasild

vasild referenced this in commit 181a224b47 on Apr 10, 2020

dongcarl commented at 9:39 pm on April 10, 2020: none

Following up on a discussion started in #907 (comment)

Was it decided how to signal support for the new addrv2 type of messages? 3 options have been discussed:

Bump the protocol version (to 70016). pros: simple to implement and the info will be available right after the handshake, without changing the handshake itself. cons: the protocol version is not a bitmap, so if an implementation wants to support addrv2 then it also has to support all features prior to 70016, see also #766 (comment)

Use a new network service bit (NODE_ADDRV2 = (1 << 4)). pros: same as 1., see also #766 (comment) cons: hmm?

Use a new message sendaddrv2 during the handshake. pros: can send wider addresses during handshake that would not otherwise fit in addrMe, but addrMe is deprecated. See #766 (comment) cons: complicates and requires changes to the handshake and requires handling if the new message sendaddrv2 arrives after the handshake.

To overgeneralize, I think that all else being the same I prefer the “new message” mechanism over the “service bit” mechanism over the “protocol version” mechanism. However, the “service bit” mechanism is well-suited for making sure that certain specific node configurations can discover useful peers.

The support signaling can be broken down into 2 parts:

Something for my node to signal to peers that it wants `addrv2` messages.

This should be indicated with a sendaddrv2 message, perhaps something like I’ve implemented here: https://github.com/bitcoin/bitcoin/pull/16748/commits/c0414a6a17ea2cf5812b45dbddf318059cde0b7e#diff-eff7adeaec73a769788bb78858815c91R2099-R2102

Something for my node to signal to peers that it serves `addrv2` messages.

This should be indicated with a NODE_ADDRV2 service bit so that both:

Nodes which specify an -onlynet with post-addrv2 networks, and
Nodes (probably light/custom nodes) which only support addrv2

Can discover useful peers during DNS bootstrapping

Why not just use the `NODE_ADDRV2` service bit for both signals?

As Marco implies here, if we use the same service bit for both signals, leech nodes (which only really want addrv2 messages) will be seen as signaling that they serve addrv2 messages, which they won’t.

I’d love some feedback on whether the above seem reasonable, and perhaps some feedback on wether or not -onlynet makes sense for all the NetworkIDs specified here.

vasild commented at 12:53 pm on April 13, 2020: contributor

I’d love some feedback on whether the above seem reasonable,

What about using a service bit (different one) also for the first capability?

The second capability is actually addr/addrv2 agnostic, isn’t it? It means “I will forward addresses via gossip to my peers (as addr or addrv2 messages)”. This will fit perfectly into #17194. Maybe use a name like NODE_GOSSIP_ADDRESSES to avoid confusion that this is addr or addrv2 specific?

and perhaps some feedback on whether or not -onlynet makes sense for all the NetworkIDs specified here.

ipv4, ipv6, torv2, torv3, i2p, cjdns - I think yes, although it would be strange to use just -onlynet=torv2. Maybe most often people will use -onlynet=torv2 -onlynet=torv3. Currently the options are ipv4, ipv6 and onion, the latter could be extended to do “torv2 or torv3”.

MarcoFalke commented at 1:53 pm on April 13, 2020: member

What about using a service bit (different one) also for the first capability?

I don’t remember what this post to the mailing list said: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-October/017428.html, so it might prove my point invalid. But I think it said something that both those properties should be negotiated on a per-link (per-connection) basis. In which case both should be done via a new message type.

If you think of service bits as a “static global” across the whole Bitcoin network, then it makes sense to only use them for things that don’t change often and don’t change on a per-connection basis. E.g. NODE_WITNESS that indicates you can serialize witness data comes to my mind. Non-consensus related stuff like transaction relay or addr relay can be negotiated on a per-connection basis.

dongcarl commented at 4:26 pm on April 15, 2020: none

@vasild

The second capability is actually addr/addrv2 agnostic, isn’t it? It means “I will forward addresses via gossip to my peers (as addr or addrv2 messages)”. This will fit perfectly into bitcoin/bips/pull/17194. Maybe use a name like NODE_GOSSIP_ADDRESSES to avoid confusion that this is addr or addrv2 specific?

Unfortunately, it is not addr/addrv2 agnostic. Think of this scenario: I am a node that is -onlynet=torv3, when doing DNS bootstrapping, if we deploy NODE_GOSSIP_ADDRESSES and I use the service bits to filter for those, it is likely that I always connect to old nodes who have no idea about addrv2, however, if we deploy NODE_ADDRV2 and filter for that, the DNS seeds will be able to help connect me to nodes who at least claim to serve addrv2 messages. @MarcoFalke

But I think it said something that both those properties should be negotiated on a per-link (per-connection) basis. In which case both should be done via a new message type.

I’m think that the “my node wants addrv2 messages” signaling should be done via a new message type, but not the “my node serves addrv2 messages” signaling. Mostly so that we don’t break DNS bootstrapping (as detailed above) for both:

Nodes which specify an -onlynet with post-addrv2 networks, and
Nodes (probably light/custom nodes) which only support addrv2

Perhaps @naumenkogs has more thoughts.

vasild commented at 7:55 pm on April 15, 2020: contributor

The second capability is actually addr/addrv2 agnostic, isn’t it? It means “I will forward addresses via gossip to my peers (as addr or addrv2 messages)”. This will fit perfectly into bitcoin/bips/pull/17194. Maybe use a name like NODE_GOSSIP_ADDRESSES to avoid confusion that this is addr or addrv2 specific?

Unfortunately, it is not addr/addrv2 agnostic. Think of this scenario: I am a node that is -onlynet=torv3, when doing DNS bootstrapping, if we deploy NODE_GOSSIP_ADDRESSES and I use the service bits to filter for those, it is likely that I always connect to old nodes who have no idea about addrv2, however, if we deploy NODE_ADDRV2 and filter for that, the DNS seeds will be able to help connect me to nodes who at least claim to serve addrv2 messages.

I meant two service bits:

NODE_ADDRV2 - support for addrv2.
- The node can parse addrv2 messages, if it receives them.
- The node can generate addrv2 messages - if the node relays addresses to his peers and if the given peer has NODE_ADDRV2 then the address will be relayed as addrv2 message. If the peer does not have NODE_ADDRV2 then the address will be relayed encoded in the old addr format.
NODE_GOSSIP_ADDRESSES - the node would relay some of the addresses to some of its peers, as it works now. This is addr vs addrv2 agnostic and would fit perfectly to https://github.com/bitcoin/bitcoin/pull/17194 - we will not relay addresses to nodes that did not signal NODE_GOSSIP_ADDRESSES.

So, for example, a multi-homed ipv4 and i2p node may seek to connect to ipv4 nodes that have both capabilities in the hopes to learn more i2p addresses from them.

vasild referenced this in commit 42ee3f5c15 on Jul 16, 2020

Sjors commented at 1:06 pm on September 16, 2020: member

//! gossiping using addrv2 messages starts with this version static const int GOSSIP_ADDRV2_VERSION = 70016;

That particular version is now used for WTXID_RELAY_VERSION and https://github.com/bitcoin/bitcoin/pull/19845 proposes to OR a version flag instead:

ADDRV2_FORMAT = 0x20000000;

vasild commented at 1:18 pm on September 16, 2020: contributor

@Sjors, #907 updates the signalling mechanism. Here is a render preview of it: https://github.com/bitcoin/bips/blob/350189a/bip-0155.mediawiki#signaling-support-and-compatibility

BIP 155: addrv2 BIP proposal #766

Considerations

Something for my node to signal to peers that it wants addrv2 messages.

Something for my node to signal to peers that it serves addrv2 messages.

Why not just use the NODE_ADDRV2 service bit for both signals?

Something for my node to signal to peers that it wants `addrv2` messages.

Something for my node to signal to peers that it serves `addrv2` messages.

Why not just use the `NODE_ADDRV2` service bit for both signals?