test: fix intermittent failures with test=addrman #29639

pull mzumsande wants to merge 1 commits into bitcoin:master from mzumsande:202403_fix_feature_asmap changing 2 files +5 −2
  1. mzumsande commented at 6:06 pm on March 12, 2024: contributor

    The nKey of the addrman is generated the first time the node is started with an empty peers.dat. Therefore, restarting a node or turning it off and on again won’t make a previously non-deterministic addrman deterministic. This could lead to intermittent failures in feature_asmap.py and rpc_net.py

    Fixes #29634

  2. DrahtBot commented at 6:06 pm on March 12, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage

    For detailed information about the code coverage, see the test coverage report.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK stratospher, brunoerg, kevkevinpal, 0xB10C
    Concept ACK BrandonOdiwuor

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #28998 (rpc: “addpeeraddress tried” return error on failure by 0xB10C)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. DrahtBot added the label Tests on Mar 12, 2024
  4. 0xB10C commented at 6:11 pm on March 12, 2024: contributor

    Concept ACK

    Hope you saw #29007 (comment)

  5. mzumsande force-pushed on Mar 12, 2024
  6. mzumsande force-pushed on Mar 12, 2024
  7. mzumsande commented at 6:29 pm on March 12, 2024: contributor

    Concept ACK

    Hope you saw #29007 (comment)

    Thanks, yes I saw that. It’s the same underlying issue - in feature_asmap.py we didn’t use restart_node but stop_node and then start_node, so the current fix is not to use clear_addrman=True, but make addrman deterministic from the start.

  8. test: fix intermittent failures with test=addrman
    The nKey of the addrman is generated the first time the node is
    started. Therefore, restarting a node or turning it off and on
    again won't make a previously non-deterministic addrman
    deterministic.
    
    Co-authored-by: 0xb10c <b10c@b10c.me>
    432a542e27
  9. in test/functional/rpc_net.py:324 in bdf80d1096 outdated
    318@@ -319,7 +319,7 @@ def test_getnodeaddresses(self):
    319 
    320     def test_addpeeraddress(self):
    321         self.log.info("Test addpeeraddress")
    322-        self.restart_node(1, ["-checkaddrman=1", "-test=addrman"])
    323+        self.restart_node(1, ["-checkaddrman=1", "-test=addrman"], clear_addrman=True)
    


    0xB10C commented at 6:30 pm on March 12, 2024:
    I’m also doing this in https://github.com/bitcoin/bitcoin/pull/28998/commits/76536d834565f1cb14ae70e5bc243041950048ae#diff-355d4500a3163e554fc3c3051b21b6bc9a97bcb35b146bff17d5fc1b4e4fd650R322-R324 (happy to drop it if it’s merged here first). I think a comment similar to mine could help someone down the road to understand why -test=addrman is not enough (and possibly required for other sub-tests needing a deterministic addrman too).

    mzumsande commented at 6:42 pm on March 12, 2024:
    Sounds good, I’ll add you comment. I haven’t seen a CI failure in rpc_net.py yet, but just from looking at the code it seemed that this could fail too.
  10. mzumsande force-pushed on Mar 12, 2024
  11. brunoerg commented at 9:27 am on March 13, 2024: contributor
    Concept ACK
  12. stratospher commented at 5:49 am on March 14, 2024: contributor

    Tested ACK 432a542e271f5b6ecb1c6ea4fa9108ad4b3a5a43.

    oh nice catch @mzumsande, @0xB10C! so nKey from Addrman’s constructor is always overwritten by previously stored nKey obtained from Unserialize().

    I’ve checked the test logs in feature_asmap.py and rpc_net.py to verify that deterministic nKey is being used in the tests now.

  13. DrahtBot requested review from brunoerg on Mar 14, 2024
  14. DrahtBot requested review from 0xB10C on Mar 14, 2024
  15. brunoerg approved
  16. brunoerg commented at 1:17 pm on March 14, 2024: contributor
    crACK 432a542e271f5b6ecb1c6ea4fa9108ad4b3a5a43
  17. BrandonOdiwuor commented at 2:46 pm on March 14, 2024: contributor
    Concept ACK
  18. kevkevinpal commented at 1:25 am on March 19, 2024: contributor

    ACK 432a542

    ran this command grep -nri "restart_node" ./test/functional to check for where -test=addrman' is used and saw that we were just missing it in test/functional/rpc_net.py`

    was not able to test this locally on my machine as I am on mac

  19. DrahtBot requested review from BrandonOdiwuor on Mar 19, 2024
  20. maflcko approved
  21. in test/functional/feature_asmap.py:43 in 432a542e27
    38@@ -39,7 +39,8 @@ def expected_messages(filename):
    39 class AsmapTest(BitcoinTestFramework):
    40     def set_test_params(self):
    41         self.num_nodes = 1
    42-        self.extra_args = [["-checkaddrman=1"]]  # Do addrman checks on all operations.
    43+        # Do addrman checks on all operations and use deterministic addrman
    44+        self.extra_args = [["-checkaddrman=1", "-test=addrman"]]
    


    rkrux commented at 10:05 am on March 19, 2024:
    Isn’t clear_addrman = true required in the restart_node call inside test_asmap_interaction_with_addrman_containing_entries function below?

    0xB10C commented at 12:44 pm on March 19, 2024:
    In this case it’s not needed. set_test_params() is called before the node is first started. clear_addrman = True would only be required if the node was running and a non-deterministic addrman had been created before.

    rkrux commented at 1:06 pm on March 19, 2024:
    Thanks for the explanation. 🙌
  22. 0xB10C approved
  23. 0xB10C commented at 12:45 pm on March 19, 2024: contributor
    ACK 432a542e271f5b6ecb1c6ea4fa9108ad4b3a5a43
  24. in test/functional/rpc_net.py:322 in 432a542e27
    318@@ -319,7 +319,9 @@ def test_getnodeaddresses(self):
    319 
    320     def test_addpeeraddress(self):
    321         self.log.info("Test addpeeraddress")
    322-        self.restart_node(1, ["-checkaddrman=1", "-test=addrman"])
    323+        # The node has an existing, non-deterministic addrman from a previous test.
    


    rkrux commented at 1:02 pm on March 19, 2024:
    Which is that previous test? Should we add the name in the comment?

    0xB10C commented at 1:23 pm on March 19, 2024:
    Any previous test that started the node without a deterministic addrman. I don’t think it’s needed in the comment.
  25. fanquake merged this on Mar 19, 2024
  26. fanquake closed this on Mar 19, 2024

  27. mzumsande deleted the branch on Mar 20, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-01 10:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me