[net] support for xored DNS seed output #6021

jonasschnelli commented at 12:43 PM on April 16, 2015: contributor

Sorry. I forget to correctly pointing out the benefits of this PR: In the past, ISP had been blocking (removing from DNS relaying) IPs coming from Bitcoin DNS seeders.

ISP could also block whole IP and port ranges. They won't becaus this would affect the business models of ISP.

Blocking DNS relaying and saying, the blocked IP serves malware, could be a low-noise incentive to reduce traffic.

Therefore I think the network overall quality can be improved if there is at least one DNS seed providing obfuscated (in this PR xored) IPs.

Especially SPV clients could improve their network independence by having a fallback (the xored DNS seed) when a ISP decides to try to block bitcoin through DNS manipulating.

client part of https://github.com/sipa/bitcoin-seeder/pull/24

(edit: added more information about this PR)

in src/chainparams.cpp:None in d582226632 outdated

 155 | @@ -156,6 +156,7 @@ class CMainParams : public CChainParams {
 156 |          vSeeds.push_back(CDNSSeedData("dashjr.org", "dnsseed.bitcoin.dashjr.org"));
 157 |          vSeeds.push_back(CDNSSeedData("bitcoinstats.com", "seed.bitcoinstats.com"));
 158 |          vSeeds.push_back(CDNSSeedData("xf2.org", "bitseed.xf2.org"));
 159 | +        vSeeds.push_back(CDNSSeedData("bitcoin.jonasschnelli.ch", "seed.bitcoin.jonasschnelli.ch", true));

jonasschnelli commented at 12:49 PM on April 16, 2015:

This seed is running in a gnu screen session over a shell until do script and will startup in case of a server reboot (over rc.local). It's a non-virtual dedicated linux server with recent debian version (Debian 7.8) using a Intel(R) Xeon(R) CPU E31245 @ 3.30GHz with 16GB RAM. Next to the dns-seed the server is also used for automatic gitian builds as well as for a bitcoind mainnet full-node.

jonasschnelli commented at 12:54 PM on April 16, 2015:

Server is in "Rechenzentrumspark Falkenstein" (Germany)

cdecker commented at 10:06 AM on April 19, 2015:

I added your seed to my DNS seed monitoring (http://bitcoinstats.com/network/dns-servers/) looks nice and stable, I assume you take the XORed peer IPs out of the same pool you use to serve the non-XORed, right?

jonasschnelli commented at 11:53 AM on April 19, 2015:

@cdecker: yes. Its the same seeder software and the same data. If you query the seeder DNS with a "x." in front of the query domain it will xor the output. https://github.com/sipa/bitcoin-seeder/pull/24

jonasschnelli commented at 2:09 PM on April 16, 2015: contributor

dnsseed at seed.bitcoin.jonasschnelli.ch supports xored output. By querying x.seed.bitcoin.jonasschnelli.ch ("x." at subdomain indicates client request xored output) it will xor the IPs with a static key of 0xFA9CFB199AC5ABFF (key t.b.d.)

laanwj commented at 3:05 PM on April 16, 2015: member

So for completeness: the motivation for this is to avoid the DNS seeds getting blocked at ISPs, because they are seen to deal out IPs that may host malware as well?

laanwj added the label P2P on Apr 16, 2015

ajweiss commented at 3:09 PM on April 16, 2015: contributor

Just a random thought, but maybe it makes sense to only xor the last 16 bits? Otherwise it may be possible for the xor to produce reserved addresses (broadcast, multicast, internal, etc) that could trigger blocks by DNS firewalls/etc...

jonasschnelli commented at 4:23 PM on April 16, 2015: contributor

@laanwj: Right. This would obfuscate DNS IP seeding data to avoid blocking through ISPs and other net authorities. IMHO this would make clients more robust (especially SPV) and could avoid centralized IP seeding services like https://github.com/mikehearn/httpseed.

jonasschnelli renamed this:
~~[net] support for xored dns seed output~~
[net] support for xored DNS seed output
on Apr 16, 2015

jonasschnelli commented at 6:08 PM on April 16, 2015: contributor

Here are some relevant documents: http://sourceforge.net/p/bitcoin/mailman/message/33187042/ http://sourceforge.net/p/bitcoin/mailman/message/33169876/

jonasschnelli commented at 7:03 AM on April 17, 2015: contributor

Sorry. I forget to correctly pointing out the benefits of this PR: In the past, ISP had been blocking (removing from DNS relaying) IPs coming from Bitcoin DNS seeders.

ISP could also block whole IP and port ranges. They won't becaus this would affect the business models of ISP.

Blocking DNS relaying and saying, the blocked IP serves malware, could be a low-noise incentive to reduce traffic.

Therefore I think the network overall quality can be improved if there is at least one DNS seed providing obfuscated (in this PR xored) IPs.

Especially SPV clients could improve their network independence by having a fallback (the xored DNS seed) when a ISP decides to try to block bitcoin through DNS manipulating.

jonasschnelli commented at 11:11 AM on April 20, 2015: contributor

@ajweiss: Right. Xorin IP addresses might swap addresses to be within a private range like 10.x.x.x. But there i still think they will be transferred through DNS (IMO there is no DNS policy that says you can can't rely IPs in private ranges). But even when a IP in a private range gets removed from the response, there will be enough IPs to allow a bitcoin-client to get a reasonable connection to the P2P net. On the other hand it would be possible (changing the seeders behavior) to remove xored IP in private ranges and fill up the response with xored ips which are not in private ranges.

sipa commented at 11:05 AM on April 21, 2015: member

@ajweiss Do you have any evidence of ISPs/DNS services that filter based on IP ranges? (other than blacklisting for using specific IPs)?

ajweiss commented at 8:19 PM on April 21, 2015: contributor

I just took a quick look at bind and djbdns and they don't have anything built in right now. But, OpenDNS talks about blocking bogon range responses here:

https://labs.opendns.com/2013/09/03/what-are-the-suspicious-responses-and-why-you-should-block-them/

(Fuller range discussion here: http://en.wikipedia.org/wiki/Martian_packet).

I've tested this and it appears to only take place when using their enhanced pay/secure DNS service (the public stuff is not currently affected). However, with all the DGA tricks/abuse that is going on these days, it's not too crazy to expect more DNS inspection in the near future for public services like Google DNS, OpenDNS and ISPs.

So maybe it makes sense to modify the seeder to drop XORed addresses that fall into bogon ranges as a precaution? Although I suppose it also becomes possible that perfectly reasonable addresses get XORed into blacklisted addresses anyway...

jonasschnelli commented at 9:25 AM on April 23, 2015: contributor

Agreed with @ajweiss. After xoring a IP there should be a check if the xored output results in a IP within a reserved and non public IP Addresses according to rfc6890 (ipv4) as well as rfc5156 (ipv6). Such IPs need to be removed (and the response can be filled up with other xored IPs).

spinza commented at 3:44 PM on April 23, 2015: none

AFAIK private ips as results of dns lookups are not generally blocked. Bind also does not limit the use of private ips or caching of such ips. I can confirm that google dns (8.8.8.8 / 8.8.4.4) does not block private ips being returned via lookups and seems to cache them as well.

in src/netbase.cpp:None in d582226632 outdated

1028 | @@ -1029,6 +1029,18 @@ int CNetAddr::GetReachabilityFrom(const CNetAddr *paddrPartner) const
1029 |      }
1030 |  }
1031 |  
1032 | +void CNetAddr::XORAddress()
1033 | +{
1034 | +    unsigned char xorKey[16] = {0xF, 0xA, 0x9, 0xC, 0xF, 0xB, 0x1, 0x9, 0x9, 0xA, 0xC, 0x5, 0xA, 0xB, 0xF, 0xF};
1035 | +    int offset = 0;
1036 | +    if (!IsIPv6())

laanwj commented at 8:29 AM on April 28, 2015:

This should be IsIPv4(), not !IsIPv6() (which would include IsTor())

jonasschnelli commented at 12:15 PM on April 28, 2015:

Thanks! Will fix this.

in src/net.cpp:None in d582226632 outdated

1101 | +            if (LookupHost(((seed.fXoredIPs ? "x." : "") + seed.host).c_str(), vIPs))
1102 |              {
1103 |                  BOOST_FOREACH(CNetAddr& ip, vIPs)
1104 |                  {
1105 | +                    if (seed.fXoredIPs)
1106 | +                        ip.XORAddress();

laanwj commented at 8:34 AM on April 28, 2015:

Be really careful with references (CNetAddr& ip) and this function mutating its state in-place, as it could garble existing data structures. In this case it doesn't give problems as the vector being iterated over is being thrown away after use. But from a peace of mind point of view I'd prefer a function ip.XORedAddress() const that returns a new address instead of overwrites itself.

jonasschnelli commented at 12:15 PM on April 28, 2015:

Hmm... i thought about this when i was writing to code. But because the XORed ip is kind of a cyphertext, i was thinking a in-state manipulating of the instance variable ip is appropriate. But i also dislike state changing functions for such cases.

sipa commented at 7:48 PM on April 28, 2015:

You'll need to pass the result of your GetRand(25) to XORAddress somehow.

jonasschnelli force-pushed on Apr 28, 2015

[net] support for xored dns seed output

client part of https://github.com/sipa/bitcoin-seeder/pull/24

ebcfd5e06d

add jonasschnellis dns seeder 4bc0a47acc

[squashme] Fix DNS Xor IPv6/Tor interfearing fcf0a2cb28

[squashme] XOR remove whitespaces e62d69c313

jonasschnelli force-pushed on Apr 28, 2015

jonasschnelli commented at 7:35 PM on April 28, 2015: contributor

Updated. Not it will queries randomly one of 26 subdomains x[a-z].<seed.hostname.etc> to lower the risk if a ns domain got blocked. The xor key stays the same between different subdomains.

My try to use the first ip in the DNS response a xor key failed. It looks like some DNS servers/implementations will rotate the IP sorting by random.

xor support for multiple xor key according to subdomain a-z 3531cc6f78

in src/net.cpp:None in e62d69c313 outdated

1099 | @@ -1100,10 +1100,16 @@ void ThreadDNSAddressSeed()
1100 |          } else {
1101 |              vector<CNetAddr> vIPs;
1102 |              vector<CAddress> vAdd;
1103 | -            if (LookupHost(seed.host.c_str(), vIPs))
1104 | +
1105 | +            //if the seed supports xored output, use a random subdomain x+(a-z)
1106 | +            //x[a-z].<NS> indicates that the dns seed server should response with XORed ips
1107 | +            if (LookupHost(((seed.fXoredIPs ? "x"+std::string(1, static_cast<char>(97+(int)GetRand(25)))+"." : "") + seed.host).c_str(), vIPs))

sipa commented at 7:48 PM on April 28, 2015:

GetRand(26)

sipa commented at 7:49 PM on April 28, 2015:

Why x + [a-z] rather than just [a-z] ?

jonasschnelli commented at 7:56 PM on April 28, 2015:

To avoid collision. But probably no one will run a seed with [a-z].<domain.etc>. IMO the x can be removed. On the other hand it is a more clear indicator of when a response should be xored. So unsure.

jonasschnelli commented at 8:31 PM on April 28, 2015: contributor

Added xor key selecting based on subdomain x[a-z]

(matches with https://github.com/jonasschnelli/bitcoin-seeder/commit/1e3e9746fab344db02f7ac944ab364bdc79d2cc8).

laanwj commented at 7:16 AM on April 29, 2015: member

It appears DNS isn't a good seeding mechanism if obfuscation is desirable.

This may well bring the IPs into forbidden ranges. 'encoding to non-blacklisted data of the same size' seems an impossible challenge, given that the blacklists are an unknown in the first place.

Well maybe DNS is good for this - but maybe we can use another record type? so it's not interpreted at all by intermediate servers?

in src/netbase.cpp:None in 3531cc6f78

  48 | @@ -49,6 +49,35 @@ static const unsigned char pchIPv4[12] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0xff, 0
  49 |  // Need ample time for negotiation for very slow proxies such as Tor (milliseconds)
  50 |  static const int SOCKS5_RECV_TIMEOUT = 20 * 1000;
  51 |  
  52 | +static const unsigned char xorKey[26][16] = {

laanwj commented at 7:18 AM on April 29, 2015:

Do it really need a huge key like this? Wasn't the simple obfuscation enough? I mean it's just a token obfuscation, if your goal is to do encryption we have better algorithms available.

jonasschnelli commented at 7:26 AM on April 29, 2015:

After a discussion on IRC sipa was pointing out, that if a ISP blocks a dns query domain like (xa.seed.bitcoin.sipa.be) because a xored ip was in a blacklist, we then still would have 25 other domains [a-z] to allow dns queries. The xor key must be different within the 26 [a-z] subdomains to ensure we not xor again to a blacklisted ip.

jonasschnelli commented at 7:23 AM on April 29, 2015: contributor

@laanwj What if we check if the xored ip lays within a reserved range like 127.0.0.0/24, etc.? Still there is a chance that we xor a non-blacklisted ip to a blacklisted ip. Using a DNS TXT could be possible. But there i don't know how ISPs/DNS relays handle TXT only records. And it would also require to go down "another layer" and send raw dns responses.

I think by xoring and checking if the xored ip is within a reserved/private range as fallback if all other seeds are blocked is a feasible way of improving the network stability.

I think there is no need to completely reinvent the ip seeding for now.

spinza commented at 7:54 AM on April 29, 2015: none

TXT responses as mentioned above seems a good idea. I also thought of having some DNS seeds available on non-standard ports?

jonasschnelli commented at 11:22 AM on April 29, 2015: contributor

Closing. This is controversial.

After implementing, thinking and discussion about this, XORin DNS seed output goes into the wrong direction and might bring in new risks while not improving the overall quality of a quick connect to some reliable nodes.

SPV clients should use multiple dns seeds and when all connection to all seeds fail, they should fallback to a recent static list of node-ips.

jonasschnelli closed this on Apr 29, 2015

MarcoFalke locked this on Sep 8, 2021