scripted-diff: Regenerate key_io data deterministically #21279

1. 
   MarcoFalke commented at 2:56 pm on February 23, 2021: member
2. MarcoFalke force-pushed on Feb 23, 2021
3. 
   MarcoFalke commented at 2:57 pm on February 23, 2021: member

   Requested by @Sjors here: #20861#pullrequestreview-596345213
4. MarcoFalke force-pushed on Feb 23, 2021
5. DrahtBot added the label Scripts and tools on Feb 23, 2021
6. 
   Sjors commented at 3:53 pm on February 23, 2021: member

   tACK fab2a4a, modulo the numbers in the README should be updated…

   Also, I find it less tedious to run this from the root of the project:

   0PYTHONPATH=test/functional/test_framework contrib/testgen/gen_key_io_test_vectors.py valid 61 > src/test/data/key_io_valid.json
   1PYTHONPATH=test/functional/test_framework contrib/testgen/gen_key_io_test_vectors.py invalid 60 > src/test/data/key_io_invalid.json
   2git diff
   

7. MarcoFalke force-pushed on Feb 23, 2021
8. in contrib/testgen/README.md:7 in fa2fb314ea outdated

    1@@ -2,7 +2,9 @@
    2 
    3 Utilities to generate test vectors for the data-driven Bitcoin tests.
    4 
    5-Usage:
    6+To use inside a scripted-diff (or just execute directly):
    7 
    8-    PYTHONPATH=../../test/functional/test_framework ./gen_key_io_test_vectors.py valid 50 > ../../src/test/data/key_io_valid.json
    9-    PYTHONPATH=../../test/functional/test_framework ./gen_key_io_test_vectors.py invalid 50 > ../../src/test/data/key_io_invalid.json
   10+    -BEGIN VERIFY SCRIPT-
   

   ---

   ---

   

   Sjors commented at 4:35 pm on February 23, 2021:

   The -BEGIN VERIFY SCRIPT- bit is confusing. Besides, this has to run, and the result committed, whenever you change something in gen_key_io_test_vectors.py.

   ---

   

   MarcoFalke commented at 4:48 pm on February 23, 2021:

   fixed
9. 
   sipa commented at 4:38 pm on February 23, 2021: member

   This is generally a good idea, and we should do it. But this does still mean that changes to what is generated based on the RNG output will result in an entirely different output. E.g. adding an option to a random.choice() may cause more entropy to be extracted from the RNG, and thus cause all future data from it to be shifted. In theory this could be addresses using consistent hashing, but that’s probably overkill.

   Concept ACK

10. MarcoFalke force-pushed on Feb 23, 2021
11. 
    DrahtBot commented at 0:42 am on February 24, 2021: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    No conflicts as of last run.

12. 
    Sjors commented at 10:16 am on March 2, 2021: member

    re-utACK fa21feb0b9c93fbc205d102510d3bb7a474431c5
13. in contrib/testgen/gen_key_io_test_vectors.py:8 in fa21feb0b9 outdated

     3@@ -4,14 +4,9 @@
     4 # file COPYING or http://www.opensource.org/licenses/mit-license.php.
     5 '''
     6 Generate valid and invalid base58 address and private key test vectors.
     7-
     8-Usage:
     9-    PYTHONPATH=../../test/functional/test_framework ./gen_key_io_test_vectors.py valid 50 > ../../src/test/data/key_io_valid.json
    10-    PYTHONPATH=../../test/functional/test_framework ./gen_key_io_test_vectors.py invalid 50 > ../../src/test/data/key_io_invalid.json
    11 '''
    12 # 2012 Wladimir J. van der Laan
    

    ---

    ---

    

    MarcoFalke commented at 11:52 am on March 7, 2021:

    @laanwj This says your name, which makes you qualified to review this? :grimacing:

    ---

    

    laanwj commented at 5:23 pm on March 29, 2021:

    I definitely don’t remember writing this :sweat:

    ---

    

    MarcoFalke commented at 6:42 am on April 7, 2022:

    Your name is removed in current master, but you are still welcome to review
14. DrahtBot added the label Needs rebase on Mar 18, 2021
15. laanwj renamed this:
    scripted-diff: Regenerate key_io data deterministically
    scripted-diff: Regenerate key_io data deterministically
    on Mar 22, 2021
16. MarcoFalke force-pushed on Mar 24, 2021
17. DrahtBot removed the label Needs rebase on Mar 24, 2021
18. DrahtBot added the label Needs rebase on Apr 6, 2022
19. contrib: make gen_key_io_test_vectors deterministic

    Also, remove instructions which are redundant with the README

    fafb4796d3
20. MarcoFalke force-pushed on Apr 6, 2022
21. MarcoFalke force-pushed on Apr 6, 2022
22. scripted-diff: Regenerate key_io data deterministically

    -BEGIN VERIFY SCRIPT-
     ./contrib/testgen/gen_key_io_test_vectors.py valid 70 > ./src/test/data/key_io_valid.json
     ./contrib/testgen/gen_key_io_test_vectors.py invalid 70 > ./src/test/data/key_io_invalid.json
    -END VERIFY SCRIPT-

    fa506add25
23. MarcoFalke force-pushed on Apr 6, 2022
24. fanquake requested review from laanwj on Apr 6, 2022
25. fanquake requested review from Sjors on Apr 6, 2022
26. DrahtBot removed the label Needs rebase on Apr 6, 2022
27. 
    Sjors commented at 10:01 am on April 19, 2022: member

    ACK fa506add25cbe5efbbabca647f5378c4128cf945

    I get the same JSON files when running the script. There are no duplicate addresses and they all change if you modify the 42 seed value (as expected).

28. 
    laanwj commented at 11:33 am on April 19, 2022: member

    Tested ACK fa506add25cbe5efbbabca647f5378c4128cf945 Good idea to put it in the scripted-diff. I also verified locally that the output is the same.
29. laanwj merged this on Apr 19, 2022
30. laanwj closed this on Apr 19, 2022

    ---

31. MarcoFalke deleted the branch on Apr 19, 2022
32. sidhujag referenced this in commit 9a2f0bc257 on Apr 19, 2022
33. DrahtBot locked this on Apr 19, 2023

Contributors
MarcoFalke Sjors sipa DrahtBot laanwj

Review Requested
Sjors laanwj

Labels
Scripts and tools