p2p: update hardcoded mainnet seeds for 25.x #27488

1. 
   jonatack commented at 5:14 pm on April 18, 2023: contributor

   Update the hardcoded P2P network seeds for 25.x after updating the manual seeds and the generation script as necessary. Previous update was #25911.

   The manual seeds are selected for reachability, uptime and service bit 1 and/or curated trusted peers. We need more Tor and CJDNS seeds and some of the current Tor and I2P seeds are no longer reachable.

   Can be tested by following the steps in contrib/seeds/README.md and verifying the manual seeds by checking their presence and services in getnodeaddresses and/or connecting to them and checking their services with getpeerinfo and behavior with -netinfo.

   Tool output:

    0$ python3 makeseeds.py -a asmap-filled.dat -s seeds_main.txt > nodes_main.txt
    1
    2Loading asmap database "asmap-filled.dat"…Done.
    3Loading and parsing DNS seeds…Done.
    4  IPv4   IPv6  Onion Pass                                               
    5  3972   1118      0 Initial
    6  3972   1118      0 Skip entries with invalid address
    7  3972   1118      0 After removing duplicates
    8  3946   1112      0 Enforce minimal number of blocks
    9  3946   1112      0 Require service bit 1
   10  2791    798      0 Require minimum uptime
   11  2757    788      0 Require a known and recent user agent
   12  2757    788      0 Filter out hosts with multiple bitcoin ports
   13   512    289      0 Look up ASNs and limit results per ASN and per net```
   

2. 
   DrahtBot commented at 5:14 pm on April 18, 2023: contributor

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Reviews

   See the guideline for information on the review process.

   Type	Reviewers
   ACK	achow101, mzumsande
   Stale ACK	Emzy

   If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

3. DrahtBot added the label P2P on Apr 18, 2023
4. in contrib/seeds/makeseeds.py:2 in 58279d002f outdated

   0@@ -1,5 +1,5 @@
   1 #!/usr/bin/env python3
   2-# Copyright (c) 2013-2022 The Bitcoin Core developers
   3+# Copyright (c) 2013-2023 The Bitcoin Core developers
   

   ---

   ---

   

   fanquake commented at 5:32 pm on April 18, 2023:

   Manually bumping dates in copyright headers is never required

   ---

   

   jonatack commented at 5:36 pm on April 18, 2023:

   Will drop in next push, thanks.
5. in contrib/seeds/makeseeds.py:201 in 58279d002f outdated

   197@@ -197,7 +198,7 @@ def main():
   198     print(f'{ip_stats(ips):s} Skip entries with invalid address', file=sys.stderr)
   199     # Skip duplicates (in case multiple seeds files were concatenated)
   200     ips = dedup(ips)
   201-    print(f'{ip_stats(ips):s} After removing duplicates', file=sys.stderr)
   202+    print(f'{ip_stats(ips):s} Remove duplicates', file=sys.stderr)
   

   ---

   ---

   

   fanquake commented at 5:42 pm on April 18, 2023:

   I think this can also just be left as-is. “After removing duplicates” certainly fits the tone/amongst the other sentences better. Not really sure why it needs changing.

   ---

   

   jonatack commented at 5:53 pm on April 18, 2023:

   Every other sentence takes the form of a non-gerund verb followed by a noun. I don’t mind dropping that, but it seems more in line with the other printed lines.

   ---

   

   jonatack commented at 5:55 pm on April 18, 2023:

   Skip/remove/enforce/require etc.

   ---

   

   fanquake commented at 5:57 pm on April 18, 2023:

   I don’t really mind, I just don’t see why there’s multiple unrelated changes in this PR (manual copyright header bumping, rewriting log output etc), when it could just be a straight-forward seed update.
6. jonatack force-pushed on Apr 18, 2023
7. 
   fanquake commented at 7:51 pm on April 18, 2023: member

   cc @Emzy
8. 
   jonatack commented at 7:51 pm on April 18, 2023: contributor

   Bringing out of draft after finishing the manual seeds updates and taking review feedback.
9. jonatack marked this as ready for review on Apr 18, 2023
10. jonatack renamed this:
    p2p: update manual tor/i2p/cjdns mainnet seeds for 25.x
    p2p: update hardcoded mainnet seeds for 25.x
    on Apr 18, 2023
11. 
    Emzy commented at 8:11 pm on April 18, 2023: contributor

    ACK feed5290f9794747444d85c1eff0dae1798b3c13 Verified that my nodes are added. They are long running. And I plan to run them for the foreseeable future.
12. 
    achow101 commented at 8:48 pm on April 18, 2023: member

    What criteria is used to choose the manually selected nodes?
13. 
    jonatack commented at 9:02 pm on April 18, 2023: contributor

    What criteria is used to choose the manually selected nodes?

    Reachability, uptime, returned by getnodeaddresses (!IsTerrible), and service bit 1. Some are run by colleagues or known persons in the space, in some cases since a long time.

14. 
    jonatack commented at 9:03 pm on April 18, 2023: contributor

    https://github.com/sipa/bitcoin-seeder/issues/92 and https://github.com/sipa/bitcoin-seeder/pull/101 could simplify the process when done.
15. 
    brunoerg commented at 0:04 am on April 19, 2023: contributor

    Perhaps it’s time to encourage people to build their own asmap database instead of just dowloading it?

    0curl https://bitcoin.sipa.be/asmap-filled.dat > asmap-filled.dat
    

16. 
    jonatack commented at 0:46 am on April 19, 2023: contributor

    Perhaps it’s time to encourage people to build their own asmap database instead of just dowloading it?

    That might be tangential to this pull but feel free to propose a concrete diff or a follow-up.

17. 
    fanquake commented at 10:52 am on April 19, 2023: member

    cc @wiz
18. jonatack force-pushed on Apr 19, 2023
19. 
    jonatack commented at 12:53 pm on April 19, 2023: contributor

    Pushed an update that adds seeds from @Wiz and colleagues. I was having issues connecting to some of them yesterday and got in touch. Infra on their end is being updated, and I was able to connect to all of them now.
20. 
    mzumsande commented at 10:32 pm on April 19, 2023: contributor

    ACK b330a6621d8801ec4c408dc0ca69ec2d02b88564

    Not 100% sure how to best test this, but I did the following:

    I followed the steps described in contrib/seeds/README.md, the overall result is similar. There are some differences (~60 in total) - I guess that’s expected due to changed input data and asmap data from sipa’s seeder.

     0$ python3 makeseeds.py -a asmap-filled.dat -s seeds_main.txt > nodes_main.txt
     1Loading asmap database "asmap-filled.dat"…Done.
     2Loading and parsing DNS seeds…Done.
     3  IPv4   IPv6  Onion Pass                                              
     4476234  75286      0 Initial
     5476234  75286      0 Skip entries with invalid address
     6476234  75286      0 After removing duplicates
     7 11932   3808      0 Enforce minimal number of blocks
     8 10003   2833      0 Require service bit 1
     9  2922    833      0 Require minimum uptime
    10  2873    815      0 Require a known and recent user agent
    11  2847    809      0 Filter out hosts with multiple bitcoin ports
    12   512    294      0 Look up ASNs and limit results per ASN and per net
    

    I also made manual connections to some of the manually added tor, i2p and cjdns seeds, which worked well. I think it’s great that we now have a few more tor seeds than 11, which seemed rather low to me considering the large number of bitcoin nodes reachable via tor.

21. jonatack force-pushed on Apr 19, 2023
22. 
    jonatack commented at 11:39 pm on April 19, 2023: contributor

    I think it’s great that we now have a few more tor seeds than 11, which seemed rather low to me considering the large number of bitcoin nodes reachable via tor.

    Credit to you, @mzumsande, for suggesting to me yesterday that we add more.

    I’ve repushed after pulling in 3cc989da5c750e740705131bed05bbf93bfdf169 from #26681 and re-running the steps in contrib/seeds/README.md.

23. DrahtBot added the label CI failed on Apr 20, 2023
24. DrahtBot removed the label CI failed on Apr 20, 2023
25. 
    fanquake commented at 9:05 am on April 20, 2023: member

    Needs rebase
26. fanquake added this to the milestone 25.0 on Apr 20, 2023
27. p2p: update manual tor/i2p/cjdns mainnet seeds for 25.x

    selected for reachability, uptime, and service bit 1

    f5c8788628
28. contrib: make-seeds updates for 25.x

    and make the steps in /contrib/seeds/README.md easier to copy-paste

    04dd1d3926
29. p2p: update hardcoded mainnet seeds for 25.x 31b1798d2c
30. 
    jonatack commented at 1:10 pm on April 20, 2023: contributor

    Needs rebase

    Done.

31. jonatack force-pushed on Apr 20, 2023
32. 
    achow101 commented at 3:44 pm on April 20, 2023: member

    ACK 31b1798d2c3fa3c479eb2d1896240e0b7fad600b

    I think we should reconsider how we do the tor, i2p, and cjdns seeds for the next release.

33. 
    mzumsande commented at 3:46 pm on April 20, 2023: contributor

    reACK 31b1798d2c3fa3c479eb2d1896240e0b7fad600b

    After running the (now bugfixed) script based on sipa’s seeder data locally, I have fewer differences compared to my ACK from yesterday for IPv4/6.

34. achow101 merged this on Apr 20, 2023
35. achow101 closed this on Apr 20, 2023

    ---

36. jonatack deleted the branch on Apr 20, 2023
37. bitcoin locked this on Apr 19, 2024

Contributors
jonatack DrahtBot fanquake Emzy achow101 brunoerg mzumsande

Labels
P2P

Milestone
25.0