Regression Tests: Migrated rpc-tests.sh to all Python rpc-tests.py #6616

pull ptschip wants to merge 1 commits into bitcoin:master from ptschip:all_python changing 7 files +139 −98
  1. ptschip commented at 9:16 pm on September 1, 2015: contributor
    1. created rpc-tests.py
    2. deleted rpc-tests.sh
    3. deleted tests-config.sh.in
    4. travis.yml points to rpc-tests.py
  3. ptschip force-pushed on Sep 1, 2015
  4. ptschip commented at 10:11 pm on September 1, 2015: contributor

    I’ll do that

    I was wondering if we should move that README.md up to the /qa folder …I think it would be more visible there. I didn’t even know it existed until you just mentioned it.

    On 01/09/2015 2:47 PM, MarcoFalke wrote:

    Don’t forget to adjust https://github.com/ptschip/bitcoin/blob/all_python/qa/rpc-tests/README.md#notes and https://github.com/ptschip/bitcoin/blob/all_python/README.md as well

    — Reply to this email directly or view it on GitHub #6616 (comment).

  5. ptschip force-pushed on Sep 2, 2015
  6. theuni commented at 2:46 am on September 2, 2015: member

    The hard-coded values are a major step backwards. File existence is a very bad metric considering that many of us often cross-compile. Also, anything that requires a certain PWD is inflexible.

    Please re-add a pre-processed config file and import it from rpc-tests.py.

  7. ptschip force-pushed on Sep 2, 2015
  8. ptschip force-pushed on Sep 2, 2015
  9. ptschip force-pushed on Sep 2, 2015
  10. theuni commented at 5:08 pm on September 2, 2015: member

    @ptschip like this:

     0diff --git a/configure.ac b/configure.ac
 1index a524bde..c0a4e54 100644
 2--- a/configure.ac
 3+++ b/configure.ac
 4@@ -878,7 +878,7 @@ AC_SUBST(MINIUPNPC_CPPFLAGS)
 5 AC_SUBST(MINIUPNPC_LIBS)
 6 AC_CONFIG_FILES([Makefile src/Makefile share/setup.nsi share/qt/Info.plist src/test/buildenv.py])
 7 AC_CONFIG_FILES([qa/pull-tester/run-bitcoind-for-test.sh],[chmod +x qa/pull-tester/run-bitcoind-for-test.sh])
 8-AC_CONFIG_FILES([qa/pull-tester/tests-config.sh],[chmod +x qa/pull-tester/tests-config.sh])
 9+AC_CONFIG_FILES([qa/rpc-tests/tests_config.py],[chmod +x qa/rpc-tests/tests_config.py])
10
11 dnl boost's m4 checks do something really nasty: they export these vars. As a
12 dnl result, they leak into secp256k1's configure and crazy things happen.
13diff --git a/qa/rpc-tests/tests_config.py.in b/qa/rpc-tests/tests_config.py.in
14new file mode 100644
15index 0000000..adc7860
16--- /dev/null
17+++ b/qa/rpc-tests/tests_config.py.in
18@@ -0,0 +1,11 @@
19+#!/usr/bin/env python2
20+BUILDDIR="@abs_top_builddir@"
21+EXEEXT="@EXEEXT@"
22+
23+# These will turn into comments if they were disabled when configuring.
24+@ENABLE_WALLET_TRUE@ENABLE_WALLET=1
25+@BUILD_BITCOIN_UTILS_TRUE@ENABLE_UTILS=1
26+@BUILD_BITCOIND_TRUE@ENABLE_BITCOIND=1
27+
28+REAL_BITCOIND=BUILDDIR + '/src/bitcoind' + EXEEXT
29+REAL_BITCOINCLI=BUILDDIR + '/src/bitcoin-cli' + EXEEXT

    Then in the migrated script you can use it like:

    0#!/usr/bin/env python2
1import tests_config
2
3print tests_config.REAL_BITCOINCLI
  11. ptschip commented at 5:19 pm on September 2, 2015: contributor

    duh, oh yeah, that’s good, i was wondering how to get rid of tests.config.sh..

    On 02/09/2015 10:09 AM, Cory Fields wrote:

    @ptschip https://github.com/ptschip like this:

    diff –git a/configure.ac b/configure.ac index a524bde..c0a4e54 100644 — a/configure.ac +++ b/configure.ac @@ -878,7 +878,7 @@ AC_SUBST(MINIUPNPC_CPPFLAGS) AC_SUBST(MINIUPNPC_LIBS) AC_CONFIG_FILES([Makefile src/Makefile share/setup.nsi share/qt/Info.plist src/test/buildenv.py]) AC_CONFIG_FILES([qa/pull-tester/run-bitcoind-for-test.sh],[chmod +x qa/pull-tester/run-bitcoind-for-test.sh]) -AC_CONFIG_FILES([qa/pull-tester/tests-config.sh],[chmod +x qa/pull-tester/tests-config.sh]) +AC_CONFIG_FILES([qa/rpc-tests/tests_config.py],[chmod +x qa/rpc-tests/tests_config.py])

    dnl boost’s m4 checks do something really nasty: they export these vars. As a dnl result, they leak into secp256k1’s configure and crazy things happen. diff –git a/qa/rpc-tests/tests_config.py.in b/qa/rpc-tests/tests_config.py.in new file mode 100644 index 0000000..adc7860 — /dev/null +++ b/qa/rpc-tests/tests_config.py.in @@ -0,0 +1,11 @@ +#!/usr/bin/env python2 +BUILDDIR="@abs_top_builddir@" +EXEEXT="@EXEEXT@" + +# These will turn into comments if they were disabled when configuring. +@ENABLE_WALLET_TRUE@ENABLE_WALLET=1 +@BUILD_BITCOIN_UTILS_TRUE@ENABLE_UTILS=1 +@BUILD_BITCOIND_TRUE@ENABLE_BITCOIND=1 + +REAL_BITCOIND=BUILDDIR + ‘/src/bitcoind’ + EXEEXT +REAL_BITCOINCLI=BUILDDIR + ‘/src/bitcoin-cli’ + EXEEXT

    Then in the migrated script you can use it like:

    #!/usr/bin/env python2 import tests_config

    print tests_config.REAL_BITCOINCLI

    — Reply to this email directly or view it on GitHub #6616 (comment).

  12. ptschip force-pushed on Sep 2, 2015
  13. ptschip force-pushed on Sep 3, 2015
  14. ptschip commented at 2:00 am on September 3, 2015: contributor
    @theuni switched over to tests_config.py and imported the vars directly and the tests run fine on Travis. However the BUILDDIR is not compatible with native windows (ie /c/bitcoin) and is not handled well by python so I had to keep the scripting that finds the builddir. Therefore there are a few options moving forward: 1) Keep the scripting that finds the builddir…I made changes so that it uses the system path rather than the os.path so we don’t have to be in the pull-tester folder to run the scripts. (2) I can do an os specific hack to normalize the builddir for native windows (3) if we could somehow find a way to put the native os path into the tests_config.py directly but I don’t know how…I’ll research this a little more tomorrow, any suggestions are welcome…
  15. ptschip commented at 5:41 pm on September 3, 2015: contributor

    @theuni The path for the builddir is now correct for each system. I put a few lines in rpc-tests.py to create the correct path if running in a mingw shell.

    I looked into putting it further upstream, in configure.ac when tests_config.py.in gets called, but I don’t see we can override the @abs_top_builddir@ with our own function and I’m not sure what we would gain.

  16. in qa/rpc-tests/README.md: in da7187bc00 outdated 
    36@@ -37,9 +37,13 @@ Helper functions for creating blocks and transactions.
37 Notes
38 =====
39 
40-You can run a single test by calling `qa/pull-tester/rpc-tests.sh <testname>`.
41+You can run any single test by calling qa/pull-tester/rpc-tests.py <testname>
    MarcoFalke commented at 6:07 pm on September 3, 2015:

    Make sure to get the backticks ` right. (L44 as well)

    Also, https://github.com/ptschip/bitcoin/blob/all_python/README.md#automated-testing needs update?

    ptschip commented at 6:20 pm on September 3, 2015:

    thanks, done both

    On 03/09/2015 11:08 AM, MarcoFalke wrote:

    In qa/rpc-tests/README.md #6616 (review):

    @@ -37,9 +37,13 @@ Helper functions for creating blocks and transactions. Notes

    -You can run a single test by calling qa/pull-tester/rpc-tests.sh <testname>. +You can run any single test by calling qa/pull-tester/rpc-tests.py

    Make sure to get the backticks ` right. (L44 as well)

    Also, https://github.com/ptschip/bitcoin/blob/all_python/README.md#automated-testing needs update?

    — Reply to this email directly or view it on GitHub https://github.com/bitcoin/bitcoin/pull/6616/files#r38677104.

  17. theuni commented at 7:49 am on September 4, 2015: member
    @ptschip I’ll be away for a few days, will review when I’m back. There are a few PRs going on parallel to this though (libevent httpd, zmq, etc) that may make sense before merging this anyway, since those things will conflict.
  18. ptschip force-pushed on Sep 4, 2015
  19. ptschip commented at 9:36 pm on September 4, 2015: contributor
    @theuni Have a few good days off. I did manage to fix the issue with the path further upstream by adding a script at the end of configure.ac in order to make the correct path edits when running in mingw .
  20. ptschip force-pushed on Sep 6, 2015
  21. ptschip force-pushed on Sep 7, 2015
  22. ptschip force-pushed on Sep 7, 2015
  23. MarcoFalke commented at 10:30 am on September 22, 2015: member
    Looks like the last conflict is #6686. I’d say it’s safe to wait until #6686 is merged before you rebase.
  24. jgarzik commented at 11:27 am on September 22, 2015: contributor
    concept ACK
  25. laanwj added the label Tests on Sep 27, 2015
  26. laanwj commented at 6:32 pm on September 27, 2015: member
    tested ACK (but needs rebase)
  27. ptschip force-pushed on Oct 1, 2015
  28. ptschip commented at 5:46 pm on October 1, 2015: contributor
    @laanwj @MarcoFalke I added the zmq tests, rebased and Travis is happy.
  29. ptschip commented at 5:57 pm on October 1, 2015: contributor
    @laanwj @MarcoFalke just got rid of a little whitespace…should be good after this build finishes.
  30. ptschip force-pushed on Oct 1, 2015
  31. in qa/pull-tester/rpc-tests.py: in e8402da8cc outdated 
     99+    rpcTestDir = buildDir + '/qa/rpc-tests/'
100+    #Run Tests
101+    for i in range(len(testScripts)):
102+       if (len(opts) == 0 or (len(opts) == 1 and "-win" in opts ) or '-extended' in opts 
103+           or testScripts[i] in opts or  re.sub(".py$", "", testScripts[i]) in opts ):
104+            print  "Running testscript " + testScripts[i] + "..."
    laanwj commented at 6:09 pm on October 1, 2015:

    Nit: I think having the testscript name in bold was nice:

    0print  "Running testscript \033[1m" + testScripts[i] + "\033[0m..."
    ptschip commented at 6:27 pm on October 1, 2015:

    I’ve tried, tried double and triple and quadruple escaping that python usually needs, and also tried the termcolor module. It doesn’t work from python and in a “mingw” shell. It will probably work on a native unix shell but I can’t test that out as I’m on Windows. Any suggestions welcome…

    On 01/10/2015 11:09 AM, Wladimir J. van der Laan wrote:

    In qa/pull-tester/rpc-tests.py #6616 (review):

    +# ‘forknotify.py’,

    • ‘p2p-acceptblock.py’,
    • ‘mempool_packages.py’, +]

    +#Enable ZMQ tests +if ENABLE_ZMQ == 1:

    • testScripts.append(‘zmq_test.py’)

    +if(ENABLE_WALLET == 1 and ENABLE_UTILS == 1 and ENABLE_BITCOIND == 1):

    • rpcTestDir = buildDir + ‘/qa/rpc-tests/’
    • #Run Tests
    • for i in range(len(testScripts)):
    •   if (len(opts) == 0 or (len(opts) == 1 and "-win" in opts ) or '-extended' in opts 

    •       or testScripts[i] in opts or  re.sub(".py$", "", testScripts[i]) in opts ):

    •        print  "Running testscript " + testScripts[i] + "..."

    Nit: I think having the testscript name in bold was nice:

    |print “Running testscript \033[1m” + testScripts[i] + “\033[0m…” |

    — Reply to this email directly or view it on GitHub https://github.com/bitcoin/bitcoin/pull/6616/files#r40946395.

    laanwj commented at 6:39 pm on October 1, 2015:

    I’ve tried, tried double and triple and quadruple escaping that python usually needs, and also tried the termcolor module. It doesn’t work from python and in a “mingw” shell. It will probably work on a native unix shell but I can’t test that out as I’m on Windows. Any suggestions welcome…

    Ok, no problem, then leave it like this. Escape-code based formatting probably doesn’t work on windows at all. If we want this it needs to be conditional on the terminal type, which is too much bother…

  32. in qa/pull-tester/rpc-tests.py: in e8402da8cc outdated 
    110+
111+    #Run Extended Tests
112+    for i in range(len(testScriptsExt)):
113+        if ('-extended' in opts or testScriptsExt[i] in opts 
114+           or re.sub(".py$", "", testScriptsExt[i]) in opts):
115+            print  "Running testscript " + testScriptsExt[i] + "..."
    laanwj commented at 6:10 pm on October 1, 2015:

    Nit: do we still want to call this “2nd level testscript”?

    0echo -e "Running \033[1m2nd level\033[0m testscript \033[1m${testScriptsExt[$i]}...\033[0m"
    ptschip commented at 6:27 pm on October 1, 2015:

    i’ll fix that and push.

    On 01/10/2015 11:10 AM, Wladimir J. van der Laan wrote:

    In qa/pull-tester/rpc-tests.py #6616 (review):

    • #Run Tests
    • for i in range(len(testScripts)):
    •   if (len(opts) == 0 or (len(opts) == 1 and "-win" in opts ) or '-extended' in opts 

    •       or testScripts[i] in opts or  re.sub(".py$", "", testScripts[i]) in opts ):

    •        print  "Running testscript " + testScripts[i] + "..." 

    •        subprocess.call(rpcTestDir + testScripts[i] + " --srcdir " + buildDir + '/src ' + passOn,shell=True) 

    •   #exit if help is called so we print just one set of instructions

    •        p = re.compile(" -h| --help")

    •        if p.match(passOn):

    •            sys.exit(0)
    • #Run Extended Tests
    • for i in range(len(testScriptsExt)):
    •    if ('-extended' in opts or testScriptsExt[i] in opts 

    •       or re.sub(".py$", "", testScriptsExt[i]) in opts):

    •        print  "Running testscript " + testScriptsExt[i] + "..."

    Nit: do we still want to call this “2nd level testscript”?

    |echo -e “Running \033[1m2nd level\033[0m testscript \033[1m${testScriptsExt[$i]}…\033[0m” |

    — Reply to this email directly or view it on GitHub https://github.com/bitcoin/bitcoin/pull/6616/files#r40946519.

  33. laanwj commented at 6:12 pm on October 1, 2015: member
    No need to fix all the nits in this pull, we can do those later, if you don’t get around to them - probably more important to get this merged before any other changes to the test runner.
  34. Migrated rpc-tests.sh to all python rpc-tests.py 
    1) created rpc-tests.py
2) deleted rpc-tests.sh
3) travis.yml points to rpc-tests.py
4) Modified Makefile.am
5) Updated README.md
6) Added tests_config.py and deleted tests-config.sh
7) Modified configure.ac with script to set correct path in tests_config.py
    5467820be5
  35. ptschip force-pushed on Oct 1, 2015
  36. jonasschnelli commented at 7:00 pm on October 1, 2015: contributor

    Concept ACK. Keeping the bold output would be nice:

    This works for me (and should also work on bash mingw?):

     0old mode 100644
 1new mode 100755
 2index f55c5b6..8c7a6b6
 3--- a/qa/pull-tester/rpc-tests.py
 4+++ b/qa/pull-tester/rpc-tests.py
 5@@ -101,7 +101,7 @@ if(ENABLE_WALLET == 1 and ENABLE_UTILS == 1 and ENABLE_BITCOIND == 1):
 6     for i in range(len(testScripts)):
 7        if (len(opts) == 0 or (len(opts) == 1 and "-win" in opts ) or '-extended' in opts 
 8            or testScripts[i] in opts or  re.sub(".py$", "", testScripts[i]) in opts ):
 9-            print  "Running testscript " + testScripts[i] + "..." 
10+            print  "Running testscript \033[1m" + testScripts[i] + "\033[0m..." 
11             subprocess.call(rpcTestDir + testScripts[i] + " --srcdir " + buildDir + '/src ' + passOn,shell=True) 
12            #exit if help is called so we print just one set of instructions
13             p = re.compile(" -h| --help")
  37. laanwj commented at 7:18 pm on October 1, 2015: member
    @jonasschnelli That’s what I proposed too, it works on Linux and OpenBSD but it doesn’t on Win apparently. (outputing junk) So it would have to be conditional.
  38. jonasschnelli commented at 7:31 pm on October 1, 2015: contributor 
    0+#Set colors
1+cOn = cOff = ""
2+if (not "-win" in opts):
3+    cOn = "\033[1m"
4+    cOff = "\033[0m"

    Then use something like: print "Running testscript " + cOn + testScripts[i] + cOff + "..."

  39. ptschip commented at 8:45 pm on October 1, 2015: contributor
    @laanwj I fixed the nit regarding the “2nd level” and left the other for another time. Pushed and all is well…
  40. laanwj merged this on Oct 1, 2015
  41. laanwj closed this on Oct 1, 2015
  42. laanwj referenced this in commit 5ab5dca6f1 on Oct 1, 2015
  43. ptschip commented at 9:08 pm on October 1, 2015: contributor

    i like the idea, but -win tests will not always be disabled (i hope)…

    so it would have to be something operating specific like

    +#Set colors +cOn = cOff = "" +if (os.name == ‘posix’):

    • cOn = ‘\033[1m’
    • cOff = ‘\033[0m’

    (oddly python needs the double \ for a )

    If this is ok, i’ll add it and push.

    On 01/10/2015 12:32 PM, Jonas Schnelli wrote:

    +#Set colors +cOn = cOff = "" +if (not “-win” in opts):

    • cOn = “\033[1m”
    • cOff = “\033[0m”

    Then use something like: |print “Running testscript " + cOn + testScripts[i] + cOff + “…"|

    —

  44. btcdrak commented at 9:12 pm on October 1, 2015: contributor
    @ptschip This PR is now merged, any new changes would have to be in a new branch + PR
  45. mruddy commented at 11:51 am on October 3, 2015: contributor
    might want to update https://github.com/bitcoin/bitcoin/blob/master/README.md#automated-testing
  46. ptschip commented at 3:07 pm on October 3, 2015: contributor

    thanks, that one got missed….updated and new PR created.

    On 03/10/2015 4:51 AM, mruddy wrote:

    might want to update https://github.com/bitcoin/bitcoin/blob/master/README.md#automated-testing

    — Reply to this email directly or view it on GitHub #6616 (comment).

  47. laanwj referenced this in commit 6c2fe54fb8 on Oct 5, 2015
  48. laanwj referenced this in commit ea709970e0 on Oct 5, 2015
  49. laanwj referenced this in commit 2844b9e90e on Oct 5, 2015
  50. ptschip deleted the branch on Oct 17, 2015
  51. str4d referenced this in commit 47855b599d on Mar 25, 2017
  52. zkbot referenced this in commit 025bd44543 on Nov 21, 2020
  53. zkbot referenced this in commit 7a0a268054 on Dec 2, 2020
  54. zkbot referenced this in commit c8896f9907 on Dec 2, 2020
  55. DrahtBot locked this on Sep 8, 2021


ptschip MarcoFalke theuni jgarzik laanwj jonasschnelli btcdrak mruddy

Labels
Tests

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-10-05 01:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me