[net,mempool] Call AcceptToMemoryPool() asynchronously in p2p #13413

pull skeees wants to merge 14 commits into bitcoin:master from skeees:mempool-async changing 27 files +1501 −305
  1. skeees commented at 3:22 pm on June 7, 2018: contributor

    This depends on #12934 and #13407: review those first

    As a follow up to #12934 which introduces some armature code to make message passing based communication a bit easier and moves ProcessNewBlock into its own thread, this does similar for AcceptToMemoryPool() in p2p

    If this approach seems reasonable, I think logical next steps are (in forthcoming prs) to try to reduce dependency on cs_main in p2p so that the benefits of this parallelism can be better realized

  2. skeees force-pushed on Jun 7, 2018
  3. skeees force-pushed on Jun 7, 2018
  4. DrahtBot commented at 7:33 pm on June 7, 2018: member
    • #13527 (policy: Remove promiscuousmempoolflags by MarcoFalke)
    • #13481 (doc: Rewrite some validation docs as lock annotations by MarcoFalke)
    • #13426 ([bugfix] Add u8path and u8string to fix #13103 by ken2812221)
    • #13423 ([net] Thread safety annotations in net_processing by skeees)
    • #13417 ([net] Tighten scope in net_processing by skeees)
    • #13407 ([refactor, move-only-ish] Refactor mempool accept/reject logic by skeees)
    • #13399 (rpc: Add submitblockheader by MarcoFalke)
    • #13311 (Don’t edit Chainparams after initialization by jtimon)
    • #13249 (Make objects in range declarations immutable by default. Avoid unnecessary copying of objects in range declarations. by practicalswift)
    • #13235 (Break circular dependency: init -> * -> init by extracting shutdown.h by Empact)
    • #13168 (Thread names in logs and deadlock debug tools (take 2) by jamesob)
    • #13145 (Use common getPath method to create temp directory in tests. by winder)
    • #13123 (net: Add Clang thread safety annotations for guarded variables in the networking code by practicalswift)
    • #13083 (Add compile time checking for cs_main runtime locking assertions by practicalswift)
    • #13076 (Fix ScanForWalletTransactions to return an enum indicating scan result: success / failure / user_abort by Empact)
    • #13029 (Interpret absense of prune= as prune=1 if there are pruned blocks by Sjors)
    • #12360 (Bury bip9 deployments by jnewbery)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  5. DrahtBot commented at 8:08 pm on June 15, 2018: member
  6. DrahtBot added the label Needs rebase on Jun 15, 2018
  7. Implement a thread-safe FIFO (producer/consumer style) queue 9668666b27
  8. Unit tests for ProducerConsumerQueue 52ac3b5469
  9. Add ConsumerThread: to consumer and operate on work from a ProducerConsumerQueue 6c624232a7
  10. ConsumerThread unit tests fb022e3850
  11. ValidationLayer() - interface for calls into block validation 5d40aa8e96
  12. Call ProcessNewBlock() asynchronously in a separate thread from p2p layer 8f0d086d53
  13. Replace all instances of ProcessNewBlock() with ValidationLayer.Validate() 0428d43f15
  14. Limit available scope of ProcessNewBlock to ValidationLayer (move-only) f31b454842
  15. Fix whitespace in test_bitcoin.cpp (whitespace,move-only) 65ae792d78
  16. [refactor, move-only-ish] Refactor mempool accept/reject logic
    Create separate functions to handle responses when a transaction
    received from a peer is accepted or rejected.
    
    This commit introduces one behavior change:
       Currently, if -promiscuousmempool is set (only allowed on regtest
       and testnet), it is possible for AcceptToMemoryPool() to return
       true, but for the CValidationState() to contain DoS points if the
       transaction failed validation with policy standard script flags
       but then passed with promiscuous flags. This will cause you to ban
       the peer who sent the transaction. This commit fixes this behavior
       by reseting the validation state in that one situation.
    d110d264c8
  17. [refactor] extract reusable logic in ValidationLayer into AsyncLayer 38f649a015
  18. ValidationLayer uses AsyncLayer 523d39f947
  19. Add MempoolLayer() for async invocation of AcceptToMemoryPool() bd24661d9f
  20. skeees force-pushed on Jun 19, 2018
  21. skeees force-pushed on Jun 19, 2018
  22. skeees force-pushed on Jun 19, 2018
  23. DrahtBot removed the label Needs rebase on Jun 19, 2018
  24. Call AcceptToMemoryPool() asynchronously in net_processing a71183b1b0
  25. skeees force-pushed on Jun 20, 2018
  26. DrahtBot added the label Needs rebase on Jul 7, 2018
  27. DrahtBot commented at 8:37 am on July 7, 2018: member
  28. sipa referenced this in commit 1e90862f5d on Jul 14, 2018
  29. in src/core/consumerthread.h:66 in a71183b1b0
    61+            if (!m_threads_observed.count(id)) {
    62+                m_threads_observed.insert(std::this_thread::get_id());
    63+
    64+                // resubmit it so that it gets a chance to get to the right thread
    65+                // when resubmitting, do not block and do not care about failures
    66+                // theres a potential deadlock where we try to push this to a queue thats
    


    practicalswift commented at 9:17 pm on September 1, 2018:
    Typo found by codespell: theres (should be “there is”)

    practicalswift commented at 9:18 pm on September 1, 2018:
    Also: “thats” should be “that is”
  30. in src/core/producerconsumerqueue.h:128 in a71183b1b0
    123+    {
    124+        static_assert(m_consumer_mode == WorkerMode::BLOCKING, "");
    125+
    126+        T ret;
    127+
    128+        // use a temporary so theres no side effecting code inside an assert which could be disabled
    


    practicalswift commented at 9:18 pm on September 1, 2018:
    Same here: “theres” should be “there is”
  31. DrahtBot added the label Up for grabs on Dec 3, 2018
  32. MarcoFalke commented at 4:10 pm on December 3, 2018: member
    There hasn’t been much activity lately and the patch still needs rebase, so I am closing this for now. Please let me know when you want to continue working on this, so the pull request can be re-opened.
  33. MarcoFalke closed this on Dec 3, 2018

  34. laanwj removed the label Needs rebase on Oct 24, 2019
  35. PastaPastaPasta referenced this in commit 5c6fb6e75c on Apr 15, 2020
  36. PastaPastaPasta referenced this in commit a3f0f3cf63 on Apr 16, 2020
  37. PastaPastaPasta referenced this in commit 504b696b78 on Apr 16, 2020
  38. PastaPastaPasta referenced this in commit 7e588c7594 on Apr 19, 2020
  39. PastaPastaPasta referenced this in commit 6cce7b22db on May 10, 2020
  40. PastaPastaPasta referenced this in commit abc246a698 on May 12, 2020
  41. PastaPastaPasta referenced this in commit 5b72c199ff on Jun 9, 2020
  42. PastaPastaPasta referenced this in commit 9c6d174c4d on Jun 9, 2020
  43. PastaPastaPasta referenced this in commit cfac0a9f3f on Jun 10, 2020
  44. PastaPastaPasta referenced this in commit c26f14a99c on Jun 11, 2020
  45. MarcoFalke locked this on Dec 16, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-10-04 22:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me