Continuing port of java comparison tool #8141

1. 
   mrbandrews commented at 6:55 pm on June 2, 2016: contributor

   This continues the port of the java comparison tool to python.

   Changes to the blockstore and a small change in main.cpp are in separate commits.

   This includes and runs both the “inexpensive” and “barely expensive” tests. The “barely expensive” test (which consists of a few re-orgs) take a minute or two on my machine.

   Java test is here: https://github.com/TheBlueMatt/test-scripts/blob/master/FullBlockTestGenerator.java

2. Tests: Rework blockstore to avoid re-serialization. 8c9e681ff8
3. Continuing port of java comptool 291f8aa5da
4. Catch exceptions from non-canonical encoding and print only to log 12c5a16c4e
5. 
   jonasschnelli commented at 6:58 pm on June 2, 2016: contributor

   Nice! Concept ACK.
6. jonasschnelli added the label Tests on Jun 2, 2016
7. 
   sipa commented at 11:28 pm on June 2, 2016: member

   Why is it necessary to accept/ignore overly long encoded varints?
8. 
   sdaftuar commented at 0:11 am on June 3, 2016: member

   @sipa I think the right test would be to make sure that receiving a block with an overly long encoded varint doesn’t cause the block to be marked permanently bad. The test included here is overly prescriptive by requiring that the overly long encoding be rejected; we should just loosen that and still check that the normally encoded block be accepted.
9. 
   laanwj commented at 7:42 am on June 7, 2016: member

   Awesome! Concept ACK.
10. Tests: Edit bloated varint test and add option for 'barely expensive' tests ff2dcf274d
11. 
    laanwj commented at 1:30 pm on June 10, 2016: member

    Anything holding back merging this?
12. 
    theuni commented at 7:15 pm on June 10, 2016: member

    It’d be helpful to know what of the Java test remains to be ported.
13. 
    mrbandrews commented at 7:51 pm on June 10, 2016: contributor

    Just the “expensive test,” which consists of: “Test massive reorgs (in terms of tx count)”
14. 
    sdaftuar commented at 7:53 pm on June 10, 2016: member

    ACK

    I’ve looked through this test and the java test, and I believe this faithfully implements all the consensus tests from the java comparison tool that we run in travis.

    I think before we turn off running the java tool in travis, we should first do more careful experiments to make sure that the java tool wasn’t catching things that this python test would miss, say by introducing intentional bugs to bitcoind and verifying that this test fails. I haven’t done this at all yet.

15. 
    theuni commented at 8:03 pm on June 10, 2016: member

    @sdaftuar agreed. @mrbandrews To my knowledge, the “expensive” test has been broken for ages and has not been used in any automated testing.
16. 
    sipa commented at 1:22 pm on June 11, 2016: member

    Concept ACK
17. 
    sipa commented at 1:23 pm on June 11, 2016: member

    Agree with @sdaftuar. Some tests with intentional bugs (including historical actual bugs) would be nice.
18. 
    laanwj commented at 9:34 am on June 13, 2016: member

    It’d be helpful to know what of the Java test remains to be ported.

    But not a requirement for merging this, as this doesn’t disable the java tester. Improving the python tests is always good, independent from anything else.

    Although the deadline for that is approaching too.

19. 
    laanwj commented at 9:37 am on June 13, 2016: member

    utACK ff2dcf2
20. laanwj merged this on Jun 13, 2016
21. laanwj closed this on Jun 13, 2016

    ---

22. laanwj referenced this in commit 44c1b1c9bb on Jun 13, 2016
23. mrbandrews deleted the branch on Jun 14, 2016
24. codablock referenced this in commit 340d6c9e31 on Sep 16, 2017
25. codablock referenced this in commit e87da41ad3 on Sep 19, 2017
26. codablock referenced this in commit e7e3a80712 on Dec 22, 2017
27. andvgal referenced this in commit af9f85764f on Jan 6, 2019
28. DrahtBot locked this on Sep 8, 2021

Contributors
mrbandrews jonasschnelli sipa sdaftuar laanwj theuni

Labels
Tests