CAddrMan: stochastic address manager #787

1. 
   sipa commented at 4:16 am on January 27, 2012: member

   Design goals:

   • Only keep a limited number of addresses around, so that addr.dat does not grow without bound.
   • Keep the address tables in-memory, and occasionally write the table to addr.dat.
   • Make sure no (localized) attacker can fill the entire table with his nodes/addresses.

   See comments in addrman.h for more detailed information.

2. 
   gavinandresen commented at 2:23 pm on January 27, 2012: contributor

   What needs testing with these changes?

   I assume:

   • Deals with existing addr.dat
   • Is faster
   • Works with -noirc -nodnsseed and a removed addr.dat (bootstrap from seed nodes)

   Do you have a tool for comparing two addrman databases? Might be nice to verify that two copies of bitcoin started at about the same time end up with mostly different peer-sets.

3. 
   sipa commented at 2:21 am on January 28, 2012: member

   Some improvements and bugfixes done. @gavinandresen:

   • It will convert an old addr.dat to the new format. During the first start-up, all addr.dat entries are imported. During a second start-up, all old addr entries are removed (using CDB::Rewrite).
   • It is faster in the sense that all locking issues involving CAddrDB and cs_mapAddresses are gone, and address-operations are fast. Reaching 8 connections may be a bit slower, as the new code much less strongly favors trying recently seen addresses - that is the price to pay for some protection against sybil attacks. When connection count is low, it will however favor self-tried addresses.
     • Yes, booting from seed nodes is no problem.

   The main thing to be tested is whether the bucket count (256 and 64) and bucket size (64) are large enough, knowing that they do not get quickly filled completely. For example, a new-style addr.dat of a week old, will it easily find connections when ran using -noirc -nodnsseed -nolisten?

4. 
   gavinandresen commented at 7:28 pm on January 31, 2012: contributor

   Testing on my Mac: I get a core dump on exit, in:

   CDB::Write<std::string, CAddrMan> (this=0xb0594f20, key=@0xb0594ed0, value=@0x372560, fOverwrite=true) at db.h:106 106 int ret = pdb->put(GetTxn(), &datKey, &datValue, (fOverwrite ? 0 : DB_NOOVERWRITE));

   Looks like a use-after-free problem:

   (gdb) p _this $1 = { pdb = 0x5005da0, strFile = { _M_dataplus = { std::allocator = { <__gnu_cxx::new_allocator> = {}, }, members of std::basic_string<char,std::char_traits,std::allocator >::_Alloc_hider: _M_p = 0x500168c “addr.dat” } }, vTxn = { <std::Vector_base<DbTxn,std::allocator<DbTxn*> » = { _M_impl = { std::allocator<DbTxn*> = { <__gnu_cxx::new_allocator<DbTxn*» = {}, }, members of std::_Vector_base<DbTxn*,std::allocator<DbTxn*> >::_Vector_impl: _M_start = 0x0, _M_finish = 0x0, _M_end_of_storage = 0x0 } }, }, fReadOnly = false }

5. 
   gavinandresen commented at 7:35 pm on January 31, 2012: contributor

   Also: I tested running with your patch and then running an old bitcoind; works nicely (old bitcoind thinks the addr.dat is empty, which is OK; run the new bitcoind and it thinks it needs to rewrite addr.dat again, which is also OK).
6. 
   sipa commented at 12:37 pm on February 24, 2012: member

   @luke-jr recently had a crash that seemed related to addrman; I am unable to reproduce the problem though. Stack traces or valgrind information from anyone who can reproduce it, is very welcome.
7. CAddrMan: stochastic address manager

   Design goals:
    * Only keep a limited number of addresses around, so that addr.dat does not grow without bound.
    * Keep the address tables in-memory, and occasionally write the table to addr.dat.
    * Make sure no (localized) attacker can fill the entire table with his nodes/addresses.
   
   See comments in addrman.h for more detailed information.

   5fee401fe1
8. gavinandresen merged this on Mar 22, 2012
9. gavinandresen closed this on Mar 22, 2012

   ---

10. destenson referenced this in commit 06b84a4e4a on Jun 26, 2016
11. in src/addrman.cpp: in 5fee401fe1

    37+bool CAddrInfo::IsTerrible(int64 nNow) const
    38+{
    39+    if (nLastTry && nLastTry >= nNow-60) // never remove things tried the last minute
    40+        return false;
    41+
    42+    if (nTime > nNow + 10*60) // came in a flying DeLorean
    

    ---

    ---

    

    rebroad commented at 10:04 am on October 29, 2016:

    always makes me laugh this line does :) thanks :)
12. in src/addrman.h: in 5fee401fe1

    153+
    154+// ... in at least this many days
    155+#define ADDRMAN_MIN_FAIL_DAYS 7
    156+
    157+// the maximum percentage of nodes to return in a getaddr call
    158+#define ADDRMAN_GETADDR_MAX_PCT 23
    

    ---

    ---

    

    rebroad commented at 10:05 am on October 29, 2016:

    Why 23%?
13. sipa referenced this in commit a951950cf7 on Aug 16, 2020
14. sipa referenced this in commit 0a0db15fc7 on Aug 16, 2020
15. sipa referenced this in commit 5d5bef0862 on Aug 28, 2020
16. sipa referenced this in commit 62da1445ac on Sep 4, 2020
17. sipa referenced this in commit f0c6824af2 on Sep 5, 2020
18. sipa referenced this in commit b9c1a76481 on Sep 11, 2020
19. DrahtBot locked this on Sep 8, 2021

Contributors
sipa gavinandresen rebroad