CCheckQueue Unit Tests #9497

pull JeremyRubin wants to merge 2 commits into bitcoin:master from JeremyRubin:checkqueue-tests changing 3 files +455 −10
  1. JeremyRubin commented at 7:04 pm on January 9, 2017: contributor

    This PR builds on #9495 to unit test the CCheckQueue for correctness.

    The cases covered in these tests are:

    1. Standard usage
    2. Failing checks are caught
    3. Prior blocks failing don’t interfere with future blocks
    4. No Memory leakage (all check destructors are called before new blocks allowed, memory is freed).
    5. Thread Safety
  2. in src/test/checkqueue_tests.cpp: in 7fd784a634 outdated
    0@@ -0,0 +1,395 @@
    1+// Copyright (c) 2012-2015 The Bitcoin Core developers
    


    instagibbs commented at 7:09 pm on January 9, 2017:
    couple of years out of date :)

    JeremyRubin commented at 7:22 pm on January 9, 2017:
    will fix!
  3. fanquake added the label Tests on Jan 9, 2017
  4. JeremyRubin force-pushed on Jan 9, 2017
  5. JeremyRubin force-pushed on Jan 9, 2017
  6. JeremyRubin commented at 11:55 pm on January 9, 2017: contributor
    Sorry for the line noise; the earlier build error should be addressed now.
  7. in src/test/checkqueue_tests.cpp: in f4f14263ad outdated
    16+#include <mutex>
    17+
    18+#include <unordered_set>
    19+#include <memory>
    20+#include "random.h"
    21+BOOST_FIXTURE_TEST_SUITE(checkqueue_tests, TestingSetup)
    


    ryanofsky commented at 9:51 pm on January 11, 2017:
    Maybe add a comment noting BasicTestingSetup can’t be used because it doesn’t set nScriptCheckThreads.
  8. in src/test/checkqueue_tests.cpp: in f4f14263ad outdated
    117+/** This test case checks that the CCheckQueue works properly
    118+ * with each specified size_t Checks pushed.
    119+ */
    120+void Correct_Queue_range(std::vector<size_t> range)
    121+{
    122+    auto small_queue = std::shared_ptr<Correct_Queue>(new Correct_Queue {128});
    


    ryanofsky commented at 9:51 pm on January 11, 2017:
    Maybe declare 128 and any other common parameters as constants above.

    ryanofsky commented at 10:01 pm on January 11, 2017:
    Why is small_queue a shared pointer, not a unique pointer or just plain stack variable? Maybe add a comment explaining. Also, you could probably use make_shared if it does need to be a shared pointer.
  9. in src/test/checkqueue_tests.cpp: in f4f14263ad outdated
    130+        {
    131+            CCheckQueueControl<FakeCheckCheckCompletion> control(small_queue.get());
    132+            while (total) {
    133+                size_t r = GetRand(10);
    134+                std::vector<FakeCheckCheckCompletion> vChecks;
    135+                for (size_t k = 0; k < r && total; k++) {
    


    ryanofsky commented at 9:56 pm on January 11, 2017:
    Maybe replace the loop with vCheck.resize(min(total, r)).
  10. in src/test/checkqueue_tests.cpp: in f4f14263ad outdated
    236+                    vChecks.emplace_back(k == 99 && end_fails);
    237+                control.Add(vChecks);
    238+            }
    239+            result[end_fails ? 0 : 1] = control.Wait();
    240+        }
    241+        BOOST_CHECK(!result[0]);
    


    ryanofsky commented at 10:08 pm on January 11, 2017:
    Would seem more direct to just BOOST_CHECK the control.Wait() call instead of putting the results in an intermediate array. This way is ok too, though.
  11. in src/test/checkqueue_tests.cpp: in f4f14263ad outdated
    120+void Correct_Queue_range(std::vector<size_t> range)
    121+{
    122+    auto small_queue = std::shared_ptr<Correct_Queue>(new Correct_Queue {128});
    123+    boost::thread_group tg;
    124+    for (auto x = 0; x < nScriptCheckThreads; ++x) {
    125+       tg.create_thread([=]{small_queue->Thread();});
    


    ryanofsky commented at 10:11 pm on January 11, 2017:
    Replacing [=] with [&] might allow small_queue not to be a shared_ptr.
  12. in src/test/checkqueue_tests.cpp: in f4f14263ad outdated
    270+        }
    271+    }
    272+    bool r = true;
    273+    for (size_t i = 0; i < COUNT; ++i)
    274+        r = r && UniqueCheck::results.count(i) == 1;
    275+    BOOST_REQUIRE(r);
    


    ryanofsky commented at 10:17 pm on January 11, 2017:
    Maybe also check that UniqueCheck::results.size == COUNT.
  13. in src/test/checkqueue_tests.cpp: in f4f14263ad outdated
    278+}
    279+
    280+
    281+// Test that blocks which might allocate lots of memory free their memory agressively.
    282+//
    283+// This test attempts to catch a pathalogical case where by lazily freeing
    


    ryanofsky commented at 10:18 pm on January 11, 2017:
    pathological (spelling)
  14. in src/test/checkqueue_tests.cpp: in f4f14263ad outdated
    282+//
    283+// This test attempts to catch a pathalogical case where by lazily freeing
    284+// checks might mean leaving a check un-swapped out, and decreasing by 1 each
    285+// time could leave the data hanging across a sequence of blocks.
    286+//
    287+// This test (failing) is dependent on not being able to handle
    


    ryanofsky commented at 10:28 pm on January 11, 2017:
    Instead of having the test be nondeterministic in this way, would anything be lost if you had the MemoryCheck constructor increment a static counter when passed a true arg, and the MemoryCheck destructor decrement the counter if the object was constructed with a true arg. Then you could detect the error case explicitly by checking the counter, and not have to allocate big chunks of memory.
  15. in src/test/checkqueue_tests.cpp: in f4f14263ad outdated
    338+    // Try waiting a second (which should be plenty of time to reach the hang
    339+    // point in t0)
    340+    //
    341+    // Note that this cannot cause a spurious failure, only could mean that
    342+    // the test doesn't actually end up checking that control waited.
    343+    MilliSleep(1000);
    


    ryanofsky commented at 10:39 pm on January 11, 2017:
    I think you could easily make this test deterministic as well, eliminating the long sleep and the while (frozen) busy loops. Would just need to have ~FrozenCleanupCheck increment a counter and signal a conditional variable so you could wait here for enough jobs to be frozen, and then do the boost check. Then this could signal another condition variable to unfreeze the jobs. I think it would be worth changing this to make the test more efficient and reliable.
  16. ryanofsky commented at 10:57 pm on January 11, 2017: member
    Overall great and very clever tests (thread safety one was fun to figure out). Added a bunch of minor comments. The only two comments I would really urge you to consider are the ones on the Memory and FrozenCleanup tests. I think it would be good to check same conditions without allocating large chunks of memory or doing 1-second busy loops so these tests can be more efficient and more reliable.
  17. JeremyRubin force-pushed on Jan 12, 2017
  18. JeremyRubin force-pushed on Jan 12, 2017
  19. in src/test/checkqueue_tests.cpp: in 27b8cbaea8 outdated
    399-        // Agressively check that we can never aquire the lock, once
    400-        // the thread has stopped check that we never got it.
    401-        while (!done && !fails2) {
    402-            fails2 = queue->ControlMutex.try_lock();
    403+        std::mutex m;
    404+        bool has_lock {false};
    


    ryanofsky commented at 10:58 pm on January 13, 2017:
    It might be clearer to replace all these bools with an enum like { STARTED, TRY_LOCK, TRY_LOCK_DONE, DONE, DONE_ACK }.

    JeremyRubin commented at 9:03 pm on January 16, 2017:
    Going to ignore this, I don’t think it makes it more clear (to me, it’s easier to debug several variables). If someone disagrees strongly, will change.
  20. ryanofsky commented at 10:59 pm on January 13, 2017: member
    utACK d44af13ddc3615686a3e76cf9a3412999db0d692. Left one minor comment, feel free to ignore.
  21. laanwj commented at 3:32 pm on January 19, 2017: member
    ACK, needs squashing
  22. JeremyRubin force-pushed on Jan 19, 2017
  23. JeremyRubin commented at 5:07 pm on January 19, 2017: contributor
    Squashed!
  24. JeremyRubin force-pushed on Jan 20, 2017
  25. JeremyRubin commented at 10:19 pm on January 20, 2017: contributor
    Rebased to be on top of #9495.
  26. Fix CCheckQueue IsIdle (potential) race condition and remove dangerous constructors. e2073424fd
  27. JeremyRubin force-pushed on Jan 20, 2017
  28. in src/test/checkqueue_tests.cpp: in 0949835100 outdated
    326+                    vChecks.emplace_back(total == 0 || total == i || total == i/2);
    327+                }
    328+                control.Add(vChecks);
    329+            }
    330+        }
    331+        BOOST_REQUIRE(MemoryCheck::fake_allocated_memory == 0);
    


    kallewoof commented at 8:05 am on January 30, 2017:
    The MemoryCheck struct destructor does not –, so this should not be == 0 unless no MemoryCheck constructors are ever called.

    JeremyRubin commented at 6:26 am on February 16, 2017:
    Yes, it was the latter. The for loop never made anything (i = 9999; i<9999). Will fix :)
  29. in src/test/checkqueue_tests.cpp: in 0949835100 outdated
    88+            fake_allocated_memory += 1;
    89+        }
    90+    };
    91+    ~MemoryCheck(){
    92+        if (b) {
    93+            fake_allocated_memory += 1;
    


    kallewoof commented at 8:38 am on January 30, 2017:
    fake_allocated_memory -= 1
  30. in src/test/checkqueue_tests.cpp: in 0949835100 outdated
    241+// Test that a block validation which fails does not interfere with
    242+// future blocks, ie, the bad state is cleared.
    243+BOOST_AUTO_TEST_CASE(test_CheckQueue_Recovers_From_Failure)
    244+{
    245+    auto fail_queue = std::unique_ptr<Failing_Queue>(new Failing_Queue {QUEUE_BATCH_SIZE});
    246+    std::array<FailingCheck, 100> checks;
    


    kallewoof commented at 8:43 am on January 30, 2017:
    Unused variable.
  31. in src/test/checkqueue_tests.cpp: in 0949835100 outdated
    357+    {
    358+        std::unique_lock<std::mutex> l(FrozenCleanupCheck::m);
    359+        // Wait until the queue has finished all jobs and frozen
    360+        FrozenCleanupCheck::cv.wait(l, [](){return FrozenCleanupCheck::nFrozen == 1;});
    361+        // Try to get control of the queue a bunch of times
    362+        for (auto x = 0; x < 100; ++x) {
    


    kallewoof commented at 8:46 am on January 30, 2017:
    Doing !fails && x < 100 here and simply fails = queue->ControlMutex.try_lock(); would break iteration on first fail rather than iterate over all 100 (e.g. if first try_lock() fails).

    JeremyRubin commented at 6:27 am on February 16, 2017:
    sure..
  32. Add CheckQueue Tests 96c7f2c345
  33. in src/test/checkqueue_tests.cpp: in 0949835100 outdated
    419+                    cv.wait(l, [&]{return done_ack;});
    420+                    });
    421+            // Wait for thread to get the lock
    422+            cv.wait(l, [&](){return has_lock;});
    423+            bool fails = false;
    424+            for (auto x = 0; x < 100; ++x) {
    


    kallewoof commented at 8:47 am on January 30, 2017:
    Same here with !fails && x < 100 as above.
  34. JeremyRubin force-pushed on Feb 16, 2017
  35. JeremyRubin commented at 6:42 am on February 16, 2017: contributor

    Fixed the issues that @kallewoof raised, and squashed.

    Unsquashed preserved here https://github.com/JeremyRubin/bitcoin/tree/checkqueue-tests-unsquashed.

  36. kallewoof commented at 5:58 pm on February 25, 2017: member

    utACK 96c7f2c

    I’m a bit concerned about non-deterministic behavior in tests as this tends to be a pain when you do run into a problem. Or is this fixed seed / PRNG so that the numbers are always the same each time? (for GetRand())

  37. JeremyRubin commented at 7:22 pm on February 25, 2017: contributor
    I could make them deterministic if that’s desirable, but realistically these tests are already non-deterministic by virtue of being multithreaded. None of the uses of GetRand are particularly dangerous here, although perhaps they area a little slower than could be.
  38. kallewoof commented at 9:31 pm on February 25, 2017: member
    I think that would be desirable, even if the multithreading makes it not 100%.
  39. kallewoof commented at 7:22 pm on February 26, 2017: member
    ACK 96c7f2c
  40. laanwj merged this on Mar 14, 2017
  41. laanwj closed this on Mar 14, 2017

  42. laanwj referenced this in commit 2c781fb920 on Mar 14, 2017
  43. luke-jr referenced this in commit f1c348e8d2 on Jun 5, 2017
  44. luke-jr referenced this in commit b71b605c6d on Jun 5, 2017
  45. luke-jr referenced this in commit 1b864c9820 on Jun 5, 2017
  46. codablock referenced this in commit 7b7924d472 on Jan 26, 2018
  47. sickpig referenced this in commit e1e28f4666 on Apr 27, 2018
  48. michelvankessel referenced this in commit 587bed7faf on Jan 5, 2019
  49. andvgal referenced this in commit abcda0e622 on Jan 6, 2019
  50. CryptoCentric referenced this in commit 557d289d0f on Feb 27, 2019
  51. DeckerSU referenced this in commit 17a6f14968 on Jul 14, 2020
  52. LarryRuane referenced this in commit 2340776789 on Feb 25, 2021
  53. LarryRuane referenced this in commit f8d9531a61 on Feb 25, 2021
  54. LarryRuane referenced this in commit 8e67b36e86 on Apr 2, 2021
  55. LarryRuane referenced this in commit a314cdba56 on Apr 2, 2021
  56. zkbot referenced this in commit 49ffee3f20 on Apr 2, 2021
  57. MarcoFalke locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-05 22:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me