Add new DNS seed from bitnodes.io. #3875

I have been working on http://getaddr.bitnodes.io to get a real-time estimate of reachable nodes in the network. The crawler (https://github.com/ayeowch/bitnodes) takes a full snapshot of reachable nodes in the network every 5 minutes.

IPv4 and IPv6 nodes with uptime greater or equal to 24 hours are sampled and a subset of 30 IPv4 nodes and 30 IPv6 nodes are selected randomly with unique ASN and published to seed.bitnodes.io (DNS seeder powered by NSD). The A and AAAA records are given 60 seconds TTL. The sampling is repeated as soon as a new snapshot is taken. This happens approximately every 5 minutes.

The DNS seeder is maintained and operated by myself as part of the Bitnodes project. The server is sponsored by the Bitcoin Foundation and is currently hosted on a dedicated machine based in Nuremberg, Germany.

Serving the same 30 seeds to the whole network for a full minute seems a lot. That could lead to many many connections to those 30 peers...

@sipa What is the recommended number of nodes should we return? I believe all the current seeders are using 60 seconds TTL as well except for bitseed.xf2.org.

Mine also uses TTL=60s, but returns different results for every request.

I guess the extra variation in software running it is a benefit, though.

@sipa I have reconfigured the seeder to flush the zone file every 5 seconds instead of 60. Record's TTL stays at 60s. So new request after 5 seconds gets another set of nodes from the sample of stable nodes. Also, I have increased the number of published nodes per request from 30 to 60 per record type.

Beyond filtering on protocol versions what other health/sanity filtering does this do? E.g. service flags, claimed height? I looked briefly at the repository and didn't see any— am I missing it?

@gmaxwell Only nodes with protocol version 70001 (i.e. >= /Satoshi:0.7.x/) are taken into account. Nodes are considered stable if they have been up (connected to the crawler) for more than 24 hours. The claimed height of published nodes must be greater or equal to that of the instance of Bitcoin-Qt that runs locally on the server when the snapshot was taken.

The code that generates the zone file every 5 seconds is publicly available now from this commit https://github.com/ayeowch/bitnodes/commit/3daec9e78f6b65ad1e1a1f7fdd868454e1638edc

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/89d72f3d9b6e7ef051ad1439f266b809f348229b for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

I had to cut down the number of published A/AAAA records from 60 to 25 to avoid the ";; Truncated, retrying in TCP mode." error when you do e.g. dig A seed.bitnodes.io.

cdecker has been kind enough to help include seed.bitnodes.io in his DNS seeders status page at http://bitcoinstats.com/network/dns-servers/. Nodes reachability from seed.bitnodes.io is now above 90% on average.

Thanks Addy. That new page from Chris is sweet too - looks like Matt's node might have stopped crawling on Sunday. I sent him a mail about it.

Looks good to me: I'll add this to the bitcoinj side too.

Does anyone have any other concerns? If not can we get this merged and ticked off the list?

ACK

ACK.