[tests] [utils] test bitcoin-cli #10798

1. 
   jnewbery commented at 10:36 pm on July 11, 2017: member

   We don’t test bitcoin-cli at all. That means that we can miss inconsistencies between the bitcoin-cli client and the RPC interface, such as #10698 and #10747. It also means that the various bitcoin-cli options and features are untested and regressions could be silently introduced.

   Let’s fix that.

   This PR adds bitcoin-cli testing in the python functional test_framework:

   1. Add a bitcoin_cli.py test script that tests bitcoin-cli. At the moment it only tests that the result of getinfo is the same if you run it as an RPC or through bitcoin-cli, but can easily be extended to test additional bitcoin-cli features

   EDIT: --usecli option is moved to a separate PR. This PR now only covers the bitcoin_cli.py test.

   2. ~Add a --usecli option to the test framework. This changes the test to use bitcoin-cli for all RPC calls instead of using direct HTTP requests. This is somewhat experimental. It works for most tests, but there are some cases where it can’t work transparently because:~

   • ~the testcase is asserting on a specific error code, and bitcoin-cli returns a different error code from the direct RPC~
   • ~we’re sending a very large RPC request (eg submitblock) and it can’t be serialized into a shell bitcoin-cli call.~

   ~I think that even though --usecli doesn’t work on all tests, it’s still a useful experimental feature. Future potential enhancements:~

   • ~enhance the framework to automatically skip tests that are known to fail with bitcoin-cli if the --usecli option is used.~
   • ~run a subset of tests in Travis with -usecli~

   This builds on and requires the TestNode PR #10711 . As an aside, this is a good demonstration of how tidy it is to add additional features/interfaces now that test node logic/state is encapsulated in a TestNode class.

   Addresses #10791

2. jnewbery renamed this:
   test bitcoin-cli
   [tests] [utils] test bitcoin-cli
   on Jul 11, 2017
3. fanquake added the label Tests on Jul 11, 2017
4. jnewbery force-pushed on Jul 12, 2017
5. jnewbery force-pushed on Jul 13, 2017
6. 
   jnewbery commented at 11:29 am on July 13, 2017: member

   Changed subprocess.run() to subprocess.check_output() to be compatible with pre3.5 python
7. [tests] add TestNodeCLI class for calling bitcoin-cli for a node b23549f6e6
8. [tests] Add bitcoin_cli.py test script c6ec4358a7
9. jnewbery force-pushed on Aug 15, 2017
10. 
    jnewbery commented at 10:01 pm on August 15, 2017: member

    Rebased now that #10711 is merged
11. 
    MarcoFalke commented at 7:59 pm on August 24, 2017: member

    Looks really straightforward.

    utACK c6ec4358a797b7a11283238a0cf0b4531def9e92

12. MarcoFalke merged this on Aug 24, 2017
13. MarcoFalke closed this on Aug 24, 2017

    ---

14. MarcoFalke referenced this in commit 4ae6d0fbef on Aug 24, 2017
15. jnewbery deleted the branch on Aug 24, 2017
16. laanwj added the label Needs backport on Sep 6, 2017
17. laanwj added this to the milestone 0.15.1 on Sep 6, 2017
18. MarcoFalke referenced this in commit 016b9ada21 on Oct 3, 2017
19. MarcoFalke referenced this in commit 716066dbea on Oct 3, 2017
20. MarcoFalke removed the label Needs backport on Oct 4, 2017
21. PastaPastaPasta referenced this in commit f524b78783 on Sep 19, 2019
22. PastaPastaPasta referenced this in commit 78c04e8126 on Sep 23, 2019
23. PastaPastaPasta referenced this in commit 095eb5d1cf on Sep 24, 2019
24. codablock referenced this in commit 1adc2001a8 on Sep 24, 2019
25. barrystyle referenced this in commit e8bcd62ce1 on Jan 22, 2020
26. DrahtBot locked this on Sep 8, 2021

Contributors
jnewbery MarcoFalke

Labels
Tests

Milestone
0.15.2