Deprecate banlist.dat #19748

1. 
   jnewbery commented at 2:21 pm on August 17, 2020: member

   banlist.dat was introduced in #6310, storing nodes that were automatically banned (due to exceeding the misbehavior threshold) and manually banned (through the setban RPC).

   Automatic bans were removed in #19219 and replaced with a discouragement filter. That filter is not saved to disk/persisted over shutdown/startup. The banlist.dat therefore now only contains addresses that have been banned through the setban rpc.

   Since #15935, we have a specific file for configuration that is updated through the RPC and persisted over shutdown/startup, namely settings.json. We should therefore move the manual ban configuration from banlist.dat to settings.json. Doing so has a couple of benefits:

   • settings.json is human readable and easily analyzable by any json parser.
   • we wouldn’t need to maintain custom [de]serialization code for the banlist.dat file.

   banlist.dat is expected to be fairly small and infrequently updated, so disk space/performance are not huge concerns.

2. jnewbery added the label Feature on Aug 17, 2020
3. 
   jnewbery commented at 2:23 pm on August 17, 2020: member

   @ryanofsky do you agree that this is a good use case for settings.json? The only unusual thing about the ban list is that items have expiry times, after which they should be removed from the config. Would that involve frequently reserializing the entire settings.json file?
4. 
   ryanofsky commented at 3:40 pm on August 17, 2020: member

   I think a json file would be a nice way to store this information (easy to read, write, and inspect), but that it would be better to use a different json file than settings.json if entries auto-expire and would be removed without user intervention. Also if there is a use-case for wanting to allow manual bans and run bitcoin with -nosettings or with the settings file in a read-only location, it could be another reason to use a different file.

   But thinking of settings.json as storage more under user control is just my own inclination. Best to chose an approach based on concrete advantages and disadvantages.

5. 
   jnewbery commented at 4:13 pm on August 17, 2020: member

   Each ban entry has an expiry time set by the user (with a default of 24 hours in the future). The software doesn’t make any autonomous decisions about expiring the ban. So one way to do this that is in line with the settings.json usage would be to only lazily remove the entries from the json either when the user bans/unbans an address or during shutdown.
6. 
   tryphe commented at 6:20 am on September 16, 2020: contributor

   Concept ACK, could go either way on a separate/monolithic file, but favor a separate file. Also favor a super-lazy file writer.
7. 
   jonatack commented at 6:53 am on September 16, 2020: member

   Concept ACK. Tend to agree with @ryanofsky on preferring keeping a separate banlist.json file. Keeping banned peers in their own file as a separate concern seems clearer and more convenient for users, the only observable change for them being the file format (a more convenient one).
8. MarcoFalke added the label Brainstorming on Oct 2, 2020
9. 
   vasild commented at 4:03 pm on January 13, 2021: member

   Concept ACK.

   This would make it possible to store ban entries of Torv3 addresses and would supersede #20904.

   Given that we dump the banlist periodically, it may be easier to implement if data is saved to its own file, e.g. banlist.json, rather than in settings.json.

10. vasild referenced this in commit 790727eeaf on Jan 19, 2021
11. 
    vasild commented at 7:07 pm on January 19, 2021: member

    A tentative implementation is at #20966.
12. vasild referenced this in commit e1db22bd54 on Jan 28, 2021
13. vasild referenced this in commit 899a8c3307 on Feb 15, 2021
14. vasild referenced this in commit 304aa85825 on Mar 10, 2021
15. vasild referenced this in commit 8372aad9cc on Mar 15, 2021
16. vasild referenced this in commit e2a5584e77 on Mar 24, 2021
17. vasild referenced this in commit ec1c4f4ad3 on Apr 6, 2021
18. vasild referenced this in commit cdeb38a2ea on Apr 6, 2021
19. vasild referenced this in commit 51909ded1e on Apr 19, 2021
20. vasild referenced this in commit c7cb9fc74d on Apr 20, 2021
21. vasild referenced this in commit 31001540e2 on Apr 20, 2021
22. vasild referenced this in commit 391b4d5e84 on Apr 26, 2021
23. vasild referenced this in commit d586283296 on May 18, 2021
24. vasild referenced this in commit cd9da1b2a2 on May 24, 2021
25. vasild referenced this in commit e5774fac4d on Jun 14, 2021
26. vasild referenced this in commit d197977ae2 on Jun 21, 2021
27. MarcoFalke closed this on Jun 23, 2021

    ---

28. MarcoFalke referenced this in commit d6e0d78c31 on Jun 23, 2021
29. josibake referenced this in commit 152d72ab89 on Jul 21, 2021
30. janus referenced this in commit 03063373d5 on Nov 5, 2021
31. DrahtBot locked this on Aug 18, 2022

Contributors
jnewbery ryanofsky tryphe jonatack vasild

Labels
Feature Brainstorming