sync: introduce a thread-safe generic container and use it to remove a bunch of “GlobalMutex"es #25390

pull vasild wants to merge 9 commits into bitcoin:master from vasild:ThreadSafePtr changing 8 files +126 −65
  1. vasild commented at 12:46 pm on June 16, 2022: contributor

    Introduce a generic container that provides a thread-safe access to any object by using a mutex which is acquired every time the object accessed.

    For example:

     0Synced<std::unordered_map<int, int>> m{{3, 9}, {4, 16}};
     1
     2{
     3    SYNCED_LOCK(m, m_locked);
     4
     5    // m_locked represents the internal object, i.e. std::unordered_map,
     6    // while m_locked is in scope the internal mutex is locked.
     7
     8    auto it = m_locked->find(3);
     9    if (it != m_locked->end()) {
    10        std::cout << it->second << std::endl;
    11    }
    12
    13    for (auto& [k, v] : *m_locked) {
    14        std::cout << k << ", " << v << std::endl;
    15    }
    16}
    17
    18WITH_SYNCED_LOCK(m, p, p->emplace(5, 25));
    

    Remove the global mutexes g_maplocalhost_mutex, g_deadline_timers_mutex, cs_dir_locks, g_loading_wallet_mutex, g_wallet_release_mutex and use Synced<T> instead.

    Benefits

    copied from a comment below:

    The Synced<T> abstraction is similar to what is suggested in this comment but it does so in a generic way to avoid code repetition. Its benefit is:

    1. It avoids code repetition at the implementation sites. See PR#26151 for a live example. Namely this:
     0class Foo
     1{
     2public:
     3    void PushBack(x)
     4    {
     5        LOCK(m_mutex);
     6        m_data.push_back(x);
     7    }
     8
     9    size_t Size()
    10    {
    11        LOCK(m_mutex);
    12        return m_data.size();
    13    }
    14
    15    // maybe also other methods if needed...
    16
    17    auto Lock()
    18    {
    19        return DebugLock<Mutex>{m_mutex, "Foo::m_mutex", __FILE__, __LINE__};
    20    }
    21
    22private:
    23    Mutex m_mutex;
    24    std::vector<int> m_data;
    25};
    26
    27class Bar
    28{
    29public:
    30    void PushBack(x)
    31    {
    32        LOCK(m_mutex);
    33        m_data.push_back(x);
    34    }
    35
    36    size_t Size()
    37    {
    38        LOCK(m_mutex);
    39        return m_data.size();
    40    }
    41
    42    // maybe also other methods if needed...
    43
    44    auto Lock()
    45    {
    46        return DebugLock<Mutex>{m_mutex, "Bar::m_mutex", __FILE__, __LINE__};
    47    }
    48
    49private:
    50    Mutex m_mutex;
    51    std::vector<std::string> m_data;
    52};
    53
    54class Baz
    55{
    56public:
    57    void Insert(x)
    58    {
    59        LOCK(m_mutex);
    60        m_data.insert(x);
    61    }
    62
    63    size_t Size()
    64    {
    65        LOCK(m_mutex);
    66        return m_data.size();
    67    }
    68
    69    // maybe also other methods if needed...
    70
    71    auto Lock()
    72    {
    73        return DebugLock<Mutex>{m_mutex, "Baz::m_mutex", __FILE__, __LINE__};
    74    }
    75
    76private:
    77    Mutex m_mutex;
    78    std::set<std::string> m_data;
    79};
    

    becomes this:

    0Synced<std::vector<int>> Foo;
    1Synced<std::vector<std::string>> Bar;
    2Synced<std::set<std::string>> Baz;
    
    1. The mutex is properly encapsulated. With a global mutex and a global variable annotated with GUARDED_BY() it is indeed not possible to add new code that accesses the variable without protection (if using Clang and -Wthread-safety-analysis and -Werror), but it is possible to abuse the mutex and start using it to protect some more, possibly unrelated stuff (we already have this in the current code).
  2. fanquake requested review from ajtowns on Jun 16, 2022
  3. fanquake requested review from theuni on Jun 16, 2022
  4. DrahtBot added the label P2P on Jun 16, 2022
  5. DrahtBot added the label RPC/REST/ZMQ on Jun 16, 2022
  6. w0xlt commented at 1:28 pm on June 16, 2022: contributor
    Approach ACK
  7. DrahtBot commented at 3:04 pm on June 16, 2022: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage

    For detailed information about the code coverage, see the test coverage report.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Concept ACK hebasto
    Approach ACK w0xlt, ajtowns
    Stale ACK jonatack

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #25722 (refactor: Use util::Result class for wallet loading by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  8. ryanofsky commented at 5:28 pm on June 16, 2022: contributor

    This seems like a potentially useful alternative to clang thread safety annotations. The ThreadSafePtr<T> class forces code to lock a mutex when accessing data, just like TSA annotations do, except unlike TSA annotations, it doesn’t rely on a nonstandard compiler extension, or suffer from quirks that come from doing a limited static analysis.

    ThreadSafePtr<T> is obviously not a complete substitute for thread safety annotations since it only handles the simple case where a single non-recursive Mutex is used to protect access to a single variable. But the variable can have any C++ type (primitive, container, or struct), so it’s probably flexible enough for a lot of cases.

    I don’t think the ThreadSafePtr<T> is the best name because xxx_ptr<T> implies the type is some kind of lightweight reference to the T, not a container which holds the T. I would call it something like Synced<T> or LockedData<T>. (An analogy here would be std::optional<T>, which is not called std::optional_ptr<T>, even though it has * and -> members, because it’s purpose is to be a container, not a pointer. The class is named after what it’s used for not what kind of members it has.)

  9. in src/sync.h:490 in 3d77fdb312 outdated
    485+
    486+    Proxy operator->() { return Proxy{m_mutex, m_obj}; }
    487+    const Proxy operator->() const { return Proxy{m_mutex, m_obj}; }
    488+
    489+    Proxy operator*() { return Proxy{m_mutex, m_obj}; }
    490+    const Proxy operator*() const { return Proxy{m_mutex, m_obj}; }
    


    ryanofsky commented at 5:42 pm on June 16, 2022:

    In commit “sync: introduce a thread safe smart pointer” (3d77fdb3127649d560c8f39d8ab83d32c9d0dff6)

    I think it would be an improvement to just give these methods a normal name like Lock instead of making them operators. Calling code would change from:

    0auto v_locked = *v;
    1for (auto& i : v_locked) ...
    

    to

    0auto v_locked = v.Lock();
    1for (auto& i : v_locked) ...
    

    which would be clearer and simpler. Proxy class is the actual class acting like a pointer so it needs * and -> methods. But outer ThreadSafePtr class is not really a pointer at all, but a container, so there is not a concrete reason it needs to have * and -> members unless that’s an aesthetic preference.


    vasild commented at 4:46 pm on June 20, 2022:

    Renamed operator*() to Lock(). The operator->() is needed in order to cause a chained -> calls.

    Notice that the call:

    0(*g_my_net_addr)[my_net_addr_entry] = lsi;
    

    now becomes:

    0g_my_net_addr.Lock()[my_net_addr_entry] = lsi;
    
  10. in src/sync.h:487 in 3d77fdb312 outdated
    482+        const UniqueLock<Mutex> m_lock;
    483+        T* const m_raw_ptr_to_obj;
    484+    };
    485+
    486+    Proxy operator->() { return Proxy{m_mutex, m_obj}; }
    487+    const Proxy operator->() const { return Proxy{m_mutex, m_obj}; }
    


    ryanofsky commented at 5:59 pm on June 16, 2022:

    In commit “sync: introduce a thread safe smart pointer” (3d77fdb3127649d560c8f39d8ab83d32c9d0dff6)

    I’m pretty sure the const version of these -> and * methods are unusable and will always lead to compile errors if you ever tried to call them. If you want to allow accessing contents of const ThreadSafePtr objects, probably you need to introduce Proxy/ConstProxy classes similar to c++ iterator/const_interator classes. Otherwise you could drop these const methods.


    vasild commented at 4:43 pm on June 20, 2022:

    I removed them because they are not needed (currently). In a previous incarnation of this I had it working with const ThreadSafePtr, but removed the complications that were required for that.

    Can always extend it (restore these) if const objects are necessary.

  11. ajtowns commented at 2:55 am on June 17, 2022: contributor

    This seems like a non-starter to me? It’s not even able to detect obvious double locks at compile-time:

     0    ThreadSafePtr<std::map<int, int>> m;
     1    m->emplace(5, 25);
     2    {
     3        auto m_locked = *m;
     4        m_locked->emplace(6, 36);
     5        m->emplace(7, 49);  // double lock
     6
     7        auto m_locked2 = *m; // double lock
     8        m_locked->emplace(9, 81);
     9        m_locked2->emplace(10, 100);
    10    }
    

    Even if it weren’t worse at catching bugs, it doesn’t seem like an improvement over writing:

     0    Mutex mut;
     1    std::map<int, int> m GUARDED_BY(mut);
     2
     3    WITH_LOCK(mut, m.emplace(5, 25));
     4    {
     5        LOCK(mut);
     6        m.emplace(6, 36);
     7        m.emplace(7, 49);  // no double lock
     8
     9        LOCK(mut); // double lock - detected at compile time
    10        m.emplace(9, 81);
    11        m.emplace(10, 100);
    12    }
    
  12. ryanofsky commented at 4:21 am on June 17, 2022: contributor

    It’s not even able to detect obvious double locks at compile-time:

    I assume it can be annotated just like any other lock. Agree implementation should fix this, though.

    Even if it weren’t worse at catching bugs, it doesn’t seem like an improvement over writing

    Well one improvement is that it enforces locking on all compilers, unlike the clang annotations. The code itself doesn’t seem much better or worse in this case for this very simple data structure, but it’s probably is worth experimenting with for chains and chainstates and mempools, etc to be able to avoid logic bugs, distinguish different uses of cs_main for different purposes, and not just have LOCK(cs_main) everywhere what no indication about what exactly is being locked or why.

    Looking at your examples, though I would even more want to replace the *m syntax with m.Lock() as suggested previously to make usage more obvious.

  13. vasild force-pushed on Jun 20, 2022
  14. vasild commented at 4:54 pm on June 20, 2022: contributor

    e577177a33...a870b2b891: address suggestions and remove optional commit that was a kind of scope creep for this PR - net: simplify logic around reachable networks and -onlynet, it will make it in a followup. @ryanofsky very insightful review, thanks for the suggestions!

    This seems like a potentially useful alternative to clang thread safety annotations. The ThreadSafePtr<T> class forces code to lock a mutex when accessing data, just like TSA annotations do, except…

    Hmm, right, I did not think of this from that perspective. In addition - TSA do not actually “force” anything, they emit a mere warning if compiled with clang. They do nothing for gcc. And if -Werror is not used to turn the warning into an error, then they can be missed/ignored.

    I see this as a complementary to TSA.

    I don’t think the ThreadSafePtr<T> is the best name because xxx_ptr<T> implies the type is some kind of lightweight reference to the T, not a container which holds the T. I would call it something like Synced<T>

    Renamed to Synced<T>, thanks!

  15. ajtowns commented at 9:06 am on June 23, 2022: contributor

    I assume it can be annotated just like any other lock. Agree implementation should fix this, though.

    I think you’d have to mark the Synced object as being lock itself (LOCKABLE), and mark the Proxy class as being a RAII guard (SCOPED_LOCKABLE), with the constructor/destructor annotated appropriately (EXCLUSIVE_LOCK_FUNCTION(ref_to_scoped_obj) and UNLOCK_FUNCTION()), and with the functions that create the Proxy (Lock and operator->) annotated with negative constraints (EXCLUSIVE_LOCKS_REQUIRED(!this))? I’ve got pretty low confidence that that will actually work as hoped/expected though…

    Well one improvement is that it enforces locking on all compilers, unlike the clang annotations.

    We already enforce locking is correct via compiling with clang in CI; it’s certainly an improvement to get those warnings earlier for anyone who’s not using clang or doesn’t have the options enabled, but [EDIT: oops, didn’t finish the thought:] not at the cost of losing some checks entirely.

  16. maflcko commented at 9:49 am on June 23, 2022: member
    I don’t see any obvious benefit here, looking at the comparison with current code in #25390 (comment)
  17. vasild force-pushed on Jun 29, 2022
  18. vasild commented at 10:04 am on June 29, 2022: contributor

    a870b2b891...7b05e787cf: use LOCK(synced) at call sites and remove more GlobalMutexes.

    The benefit is not at the call sites - they can use different flavors of syntax sugar but all of them more or less boil down to the same thing. I changed it to use { LOCK(foo); foo->Method1(); foo->Method2(); ... } so that it is not possible to misuse like @ajtowns suggested above.

    The Synced<T> abstraction is similar to what is suggested in this comment but it does so in a generic way to avoid code repetition. Its benefit is twofold:

    1. It avoids code repetition at the implementation sites. Namely this:
     0class Foo
     1{
     2public:
     3    void PushBack(x)
     4    {
     5        LOCK(m_mutex);
     6        m_data.push_back(x);
     7    }
     8
     9    size_t Size()
    10    {
    11        LOCK(m_mutex);
    12        return m_data.size();
    13    }
    14
    15    // maybe also other methods if needed...
    16
    17    auto Lock()
    18    {
    19        return DebugLock<Mutex>{m_mutex, "Foo::m_mutex", __FILE__, __LINE__};
    20    }
    21
    22private:
    23    Mutex m_mutex;
    24    std::vector<int> m_data;
    25};
    26
    27class Bar
    28{
    29public:
    30    void PushBack(x)
    31    {
    32        LOCK(m_mutex);
    33        m_data.push_back(x);
    34    }
    35
    36    size_t Size()
    37    {
    38        LOCK(m_mutex);
    39        return m_data.size();
    40    }
    41
    42    // maybe also other methods if needed...
    43
    44    auto Lock()
    45    {
    46        return DebugLock<Mutex>{m_mutex, "Bar::m_mutex", __FILE__, __LINE__};
    47    }
    48
    49private:
    50    Mutex m_mutex;
    51    std::vector<std::string> m_data;
    52};
    53
    54class Baz
    55{
    56public:
    57    void Insert(x)
    58    {
    59        LOCK(m_mutex);
    60        m_data.insert(x);
    61    }
    62
    63    size_t Size()
    64    {
    65        LOCK(m_mutex);
    66        return m_data.size();
    67    }
    68
    69    // maybe also other methods if needed...
    70
    71    auto Lock()
    72    {
    73        return DebugLock<Mutex>{m_mutex, "Baz::m_mutex", __FILE__, __LINE__};
    74    }
    75
    76private:
    77    Mutex m_mutex;
    78    std::set<std::string> m_data;
    79};
    

    becomes this:

    0Synced<std::vector<int>> Foo;
    1Synced<std::vector<std::string>> Bar;
    2Synced<std::set<std::string>> Baz;
    
    1. The mutex is properly encapsulated. With a global mutex and a global variable annotated with GUARDED_BY() it is indeed not possible to add new code that accesses the variable without protection (if using Clang and -Wthread-safety-analysis and -Werror), but it is possible to abuse the mutex and start using it to protect some more, possibly unrelated stuff (we already have this in the current code).
  19. maflcko commented at 10:13 am on June 29, 2022: member
    Ah, looks like this is std::atomic for structs/classes then
  20. vasild commented at 11:06 am on June 29, 2022: contributor

    Ah, looks like this is std::atomic for structs/classes then

    Right, kind of. std::atomic can be used for any trivially copyable structs/classes too and if necessary it will use a mutex internally. The difference is that reading from an atomic would read it in a safe way from the memory and would return a copy of the stored object. So if we call a method of the stored object the mutex will be released before the method is called and the method will be called on the copy. atomic also does not provide a way to lock for longer time, spanning calls to multiple methods.

  21. vasild renamed this:
    sync: introduce a thread-safe smart pointer and use it to remove g_maplocalhost_mutex
    sync: introduce a thread-safe smart container and use it to remove g_maplocalhost_mutex
    on Jul 5, 2022
  22. vasild renamed this:
    sync: introduce a thread-safe smart container and use it to remove g_maplocalhost_mutex
    sync: introduce a thread-safe smart container and use it to remove a bunch of "GlobalMutex"es
    on Jul 5, 2022
  23. in src/sync.h:464 in 618809ab6b outdated
    459+            // the result of the comparison to the current thread cannot be
    460+            // changed by other, concurrently executing, threads.
    461+            : m_lock{parent.m_owner == std::this_thread::get_id() ? nullptr : &parent.m_mutex,
    462+                     mutex_name,
    463+                     file_name,
    464+                     line},
    


    ryanofsky commented at 3:44 pm on July 12, 2022:

    In commit “sync: introduce a thread safe smart container” (618809ab6b1f1c67ea3e299626f1ccb89f55d9f1)

    It seems like you could avoid this m_owner stuff by just replacing Mutex with RecursiveMutex below. Otherwise this code just seems like it is implementing RecursiveMutex on top of Mutex, which only makes it more complex and probably less efficient than using RecursiveMutex directly.

    Maybe it would be better to disallow double locking and use clang thread annotations to prevent obvious cases of double locking at compile time. But maybe this is not possible. And allowing double locking doesn’t seem like an inherently bad thing here apart from the slight performance overhead. Code fragility problems associated with recursive mutex usage don’t seem like they would be an problem here since the mutex is private and the class has a limited interface.


    vasild commented at 8:54 am on July 13, 2022:

    I agree. Simplified this by removing m_owner and using RecursiveMutex.

    This reduced the Synced implementation from 110 to 58 lines (excluding comments).

    It is possible to further reduce it by removing Synced::UniqueLock but then call sites would have to use auto lock = foo.Lock(); instead of the sweet LOCK(foo);. I think either way is fine.

    Maybe it would be better to disallow double locking and use clang thread annotations to prevent obvious cases of double locking at compile time. But maybe this is not possible.

    I will look at this again, but last time I tried I couldn’t do that - stumbled on some limitations and bizarre behavior that looked like a bug in the thread safety annotations. Notice that even now some cases are prevented at compile time, like:

    0LOCK(foo);
    1...
    2LOCK(foo);
    
  24. ryanofsky commented at 4:06 pm on July 12, 2022: contributor

    I think right now I am at “concept -0.5” on this PR at 7b05e787cfc43494f0cea6364928f184c98e3fae. If other people think this change is a good idea, it seems fine, but I think the complexity it adds to sync.h isn’t really justified by the minor code simplifications it enables elsewhere. Maybe if changes to sync.h were simplified or if a compelling use-case were explained, this would look like a better tradeoff.


    re: #25390 (comment)

    This seems like a non-starter to me? It’s not even able to detect obvious double locks at compile-time:

    This is fixed now by allowing double locking. Following test passes test_bitcoin -t sync_tests/synced_double_lock

     0diff --git a/src/test/sync_tests.cpp b/src/test/sync_tests.cpp
     1index 55c2c5108de..3c225847529 100644
     2--- a/src/test/sync_tests.cpp
     3+++ b/src/test/sync_tests.cpp
     4@@ -140,4 +140,18 @@ BOOST_AUTO_TEST_CASE(inconsistent_lock_order_detected)
     5 #endif // DEBUG_LOCKORDER
     6 }
     7 
     8+BOOST_AUTO_TEST_CASE(synced_double_lock)
     9+{
    10+    Synced<std::map<int, int>> m;
    11+    m->emplace(5, 25);
    12+    {
    13+        auto m_locked = m.Lock();
    14+        m_locked->emplace(6, 36);
    15+        m->emplace(7, 49);  // double lock
    16+        auto m_locked2 = m.Lock(); // double lock
    17+        m_locked->emplace(9, 81);
    18+        m_locked2->emplace(10, 100);
    19+    }
    20+}
    21+
    22 BOOST_AUTO_TEST_SUITE_END()
    
  25. vasild force-pushed on Jul 13, 2022
  26. vasild commented at 8:57 am on July 13, 2022: contributor
    7b05e787cf...acb21a5a24: simplify the implementation of Synced, as suggested above.
  27. DrahtBot added the label Needs rebase on Jul 13, 2022
  28. vasild force-pushed on Jul 14, 2022
  29. vasild commented at 7:18 am on July 14, 2022: contributor
    acb21a5a24...4b870a1538: rebase due to conflicts
  30. DrahtBot removed the label Needs rebase on Jul 14, 2022
  31. vasild force-pushed on Jul 21, 2022
  32. vasild commented at 10:22 am on July 21, 2022: contributor

    4b870a1538...135ca49e24: Further simplify the Synced class:

    • Inherit Synced from RecursiveMutex. This allows the removal of the member Synced::m_mutex and the type Synced::UniqueLock
    • Inherit Synced::Proxy from ::UniqueLock<RecursiveMutex>. This allows the removal of the member Synced::Proxy::m_lock.

    After the above changes there are two methods with similar names:

    • Synced::lock(), inherited from RecursiveMutex, it just locks the object
    • Synced::Lock(), it locks the object and returns a scoped lockable.

    To avoid confusion, rename the latter to operator*(). This way constructs like foo.Lock()[x] become (*foo)[x] which looks better to me.

    Now the Synced class is 47 lines (excluding comments).

  33. vasild force-pushed on Jul 21, 2022
  34. vasild commented at 11:25 am on July 21, 2022: contributor
    135ca49e24...11ac578f91: use MutexType instead of M in templates because MutexType is already used elsewhere in sync.h
  35. in src/sync.h:483 in bca6b06341 outdated
    478+     * 2. `Proxy::operator->()` is called, which returns a raw pointer to
    479+     * `m_obj` which is dereferenced in order to call the relevant method.
    480+     */
    481+    Proxy operator->()
    482+    {
    483+        return Proxy{*this, "Synced::operator->()", __FILE__, __LINE__};
    


    ajtowns commented at 5:07 am on September 23, 2022:

    After a lot of poking at this, I believe what’s necessary to make clang’s thread safety logic actually notice that this function grabs a lock is to annotate the move constructor of Proxy, ie Proxy(Proxy&&) EXCLUSIVE_LOCK_FUNCTION(mutex) = default. https://reviews.llvm.org/D41933?id=129496

    Unfortunately, I don’t think that’s sufficient to avoid clang getting confused about aliasing; at least the ways I’ve tried haven’t worked.


    vasild commented at 10:30 am on September 29, 2022:
    Yeah, me too. Clang keeps emitting bogus warnings about loops like for (auto& a : *obj) for everything I tried, so I gave up. It has its limitations after all.

    ajtowns commented at 6:48 pm on September 29, 2022:

    Oh!! How about this:

     0template<typename T>
     1struct Guarded
     2{
     3private:
     4    mutable Mutex m;
     5    T obj GUARDED_BY(m);
     6public:
     7    class SCOPED_LOCKABLE Proxy
     8    {
     9    private:
    10        Guarded& g;
    11    public:
    12        Proxy(Guarded& g) EXCLUSIVE_LOCK_FUNCTION(g.m) : g{g}
    13        {
    14            g.m.lock();
    15        }
    16        ~Proxy() UNLOCK_FUNCTION() { g.m.unlock(); }
    17
    18        T& operator*() { return g.obj; }
    19        T* operator->() { return &g.obj; }
    20    };
    21};
    22
    23#define PROXY(name, guarded) decltype(guarded)::Proxy name{guarded};
    24
    25Guarded<uint32_t> global_i;
    26Guarded<std::vector<uint32_t>> global_nums;
    27void foo()
    28{
    29    PROXY(p, global_i);
    30    *p = 5;
    31    *p += 6;
    32    // PROXY(p2, global_i); // error: acquiring mutex 'global_i.m' that is already held [-Werror,-Wthread-safety-analysis]
    33    // *p2 = 9;
    34}
    35
    36int64_t lock_already_held(Guarded<std::vector<uint32_t>>::Proxy &p)
    37{
    38    int64_t j = 0;
    39    for (auto i : *p) { j += i; }
    40    return j;
    41}
    42
    43int64_t bar()
    44{
    45    PROXY(p, global_nums);
    46    int64_t j = 0;
    47    for (auto i : *p) { j += i; }
    48    return j + lock_already_held(p);
    49}
    

    That:

    • gives compile time errors if you try accessing without a lock
    • doesn’t require recursive locks
    • gives clang compile time errors if you try double locking

    which hits everything I wanted, and doesn’t even seem too klunky. Perhaps needs some additional work to handle const.


    ryanofsky commented at 8:35 pm on September 29, 2022:

    re; #25390 (review)

    This suggestion seems very good. I’m still not sure how broadly useful this locking container would be, but I agree having recursive mutex semantics (however they are implemented) would make using it less safe, so better to avoid that if possible.

    About the Guarded/PROXY suggestion specifically:

    • I wonder if PROXY macro is necessary. Could Guarded class just have a auto Lock() { return Proxy{*this}; } method instead? It seems like that could work just as well in a for loop or auto nums = global_nums->Lock() statement.
    • Maybe rename s/PROXY/GUARDED_LOCK/ so macro name and class name match. This would also be more consistent with other locking macro names WAIT_LOCK and TRY_LOCK
    • Maybe switch PROXY(name, guarded) argument order for consistency with WAIT_LOCK and TRY_LOCK where the new variable name argument is last, and the existing mutex argument is first.

    ajtowns commented at 0:01 am on September 30, 2022:

    I think the macro is necessary – otherwise return Proxy{*this} means you have to annotate the Proxy(Proxy&&) move constructor, but the only thing you can annotate that with is locking its internal reference to g.m but that then creates an alias of the mutex which prevents clang from detecting when double locking is happening since clang can’t tell that two different invocations are locking aliases of the same lock.

    Changing the name and the argument order makes lots of sense, I just posted it as soon as I found something that worked. Guarded<T> probably isn’t a great name either. (I think it makes sense for Proxy to match whatever the macro is named)


    vasild commented at 1:46 pm on October 3, 2022:

    What about this:

     0template <typename T>
     1class Synced
     2{
     3public:
     4    template <typename... Args>
     5    Synced(Args... args) : m_obj{args...}
     6    {
     7    }
     8
     9    class SCOPED_LOCKABLE Proxy : ::UniqueLock<Mutex>
    10    {
    11    public:
    12        Proxy(Synced& parent) EXCLUSIVE_LOCK_FUNCTION(parent.m_mutex)
    13            : ::UniqueLock<Mutex>{parent.m_mutex, "...", "...", 123}, m_parent{parent}
    14        {
    15        }
    16
    17        ~Proxy() UNLOCK_FUNCTION() {}
    18
    19        T& operator*()
    20        {
    21            return m_parent.m_obj;
    22        }
    23
    24        T* operator->()
    25        {
    26            return &m_parent.m_obj;
    27        }
    28
    29    private:
    30        Synced& m_parent;
    31    };
    32
    33private:
    34    mutable Mutex m_mutex;
    35    T m_obj GUARDED_BY(m_mutex);
    36};
    37
    38#define SYNCED_LOCK(synced, name) decltype(synced)::Proxy name{synced}
    

    With example usage:

     0Synced<std::vector<int>> synced{1, 2, 3, 4};
     1
     2{
     3    SYNCED_LOCK(synced, p);
     4
     5    p->push_back(5);
     6}
     7
     8{
     9    SYNCED_LOCK(synced, p);
    10
    11    std::cout << (*p)[0] << std::endl;
    12
    13    for (auto& i : *p) {
    14        std::cout << i << std::endl;
    15    }
    16}
    

    We lose the sweet syntax:

    0Synced<std::vector<int>> synced{1, 2, 3, 4};
    1
    2synced->push_back(5);
    

    compared to the other variants, but this would still offer simplifications in higher level code and if you are ok with it, then I am fine.


    ajtowns commented at 7:37 pm on October 3, 2022:

    I’m not sure what the differences are between you’re quoting above vs what I suggested? They currently look pretty much the same to me? (I was going to check that begin/end/op[] didn’t create double locking issues, but you’ve gotten rid of them already)

    I think it’d be worth adding LIFETIMEBOUND annotations (from attributes.h) as well. Perhaps you should delete the move constructor for Synced?

    I agree synced->push_back(5) is pretty, but it seems likely to be bug prone to me: consider it = synced_map->find(key) – the lock will be released as soon as the iterator is returned, allowing it to be immediately invalidated. So I kind of think we should just count ourselves lucky that we can’t make it work and callers have to be explicit about the lock scope…


    ajtowns commented at 7:41 pm on October 3, 2022:
    It might be nice to have a ConstProxy variant too, so that if you have a const reference to an object containing a Synced<foo> you can still access have synced access to the object as const foo. On the other hand, maybe better to worry about that when an actual use comes up.

    ajtowns commented at 8:52 pm on October 3, 2022:
    Proxy : ::UniqueLock<Mutex> – isn’t it better to say private explicitly? (Guess who just assumed it defaulted to public until the compiler told him otherwise…)

    vasild commented at 7:59 am on October 4, 2022:

    I’m not sure what the differences are between you’re quoting above vs what I suggested?

    Yes, they are pretty much the same. The question was maybe more directed to @ryanofsky - if he is ok with that. I want to avoid pushing another iteration of this PR and it to be shot by somebody shortly after :) If you both agree on the above, then lets do that!

    To-polish: LIFETIMEBOUND, delete the Synced move constructor, consider const, explicit inheritance type.


    ajtowns commented at 8:43 am on October 5, 2022:

    If you made m_obj and m_mutex be protected instead of private, that would allow you to write:

    0class MyFoo : public Synced<foo>
    1{
    2public:
    3    int pretty() EXCLUSIVE_LOCKS_REQUIRED(!m_mutex) { LOCK(m_mutex); return m_obj.pretty(); }
    4};
    

    and then do i = myfoo.pretty(); – gets you the same end result as the synced->pretty() stuff, but only for things you explicitly add a wrapper for, which at least also gives you a chance to make sure it’s actually safe to be used in isolation?


    vasild commented at 9:09 am on October 5, 2022:

    Ok, I wouldn’t add features that are not used anywhere, but s/private/protected is not actually more code. Makes sense if somebody wants to inherit it and do more involved things.

    To-polish: LIFETIMEBOUND, delete the Synced move constructor, consider const, explicit inheritance type, protected m_mutex and m_obj. @ryanofsky, what do you think?


    ryanofsky commented at 3:26 pm on October 5, 2022:

    To-polish: LIFETIMEBOUND, delete the Synced move constructor, consider const, explicit inheritance type, protected m_mutex and m_obj.

    @ryanofsky, what do you think?

    All of these tweaks sound good to me.

    Generally speaking about the PR as a whole, I’m skeptical the Synced class will be broadly useful, but I think it’s a simple enough utility that can be used in a few places without many drawbacks, and that it’s worth experimenting with. I was more interested in the class previously when I thought it was a locking pointer enforcing use of an external mutex (that could be used to migrate code away from using cs_main), than a locking container that just used an internal mutex. But it seems like a good thing to use in places where it fits.


    vasild commented at 10:00 am on October 12, 2022:

    Updated, I only left the const stuff away, since it is not needed now and can be added later if needed.

    For the usefulness of the Synced class, see the OP, I updated it with some of the benefits.


    ajtowns commented at 6:21 am on October 18, 2022:

    Hmm, when used with shared locks, it seems like clang’s thread safety analysis doesn’t actually prevent you from calling non-const member functions of a GUARDED_BY object when only holding the lock in shared mode (it does prevent you from doing a = b, just not a.set(b)). So if you take a shared lock over a vector, then resize it and invalidate everyone elses iterators, clang thinks it’s all fine.

    Which I think means that if we wanted to do shared locking more broadly, then this method might be the safest way we have of it: ie have SYNCED_SHARED(foo, fooproxy) take a shared lock and have *fooproxy give a const Foo& so you can’t accidently modify foo while you’re potentially sharing it with other threads.

    EDIT: for possible future reference, just doing this looks like it works sensibly:

     0class Synced
     1{
     2...
     3    class SCOPED_LOCKABLE ConstProxy : public ::UniqueLock<Mutex>
     4    {
     5    public:
     6        ConstProxy(const Synced& parent LIFETIMEBOUND, const char* name, const char* file, int line)
     7            EXCLUSIVE_LOCK_FUNCTION(parent.m_mutex)
     8            : ::UniqueLock<Mutex>{parent.m_mutex, name, file, line}, m_parent{parent}
     9        {
    10        }
    11
    12        ~ConstProxy() UNLOCK_FUNCTION() {}
    13
    14        const T& operator*() LIFETIMEBOUND { return m_parent.m_obj; }
    15
    16        const T* operator->() LIFETIMEBOUND { return &m_parent.m_obj; }
    17
    18    private:
    19        const Synced& m_parent;
    20    };
    21...
    22};
    23
    24#define SYNCED_SHARED(synced, name) decltype(synced)::ConstProxy name{synced, #synced, __FILE__, __LINE__}
    

    Tweaking that to use a std::shared_lock instead of unique_lock and marking it as SHARED_LOCK_FUNCTION instead of exclusive would let us use shared locks pretty easily and safely, I think.

  36. in src/sync.h:432 in 11ac578f91 outdated
    427+ *     assert(size == v->size());
    428+ * }
    429+ * @endcode
    430+ */
    431+template <typename T>
    432+class Synced : public RecursiveMutex
    


    ajtowns commented at 5:03 pm on September 28, 2022:
    I don’t think we should be doing more recursive mutex stuff (they’re harder to reason about, and less efficient any time that matters). I haven’t been able to figure out a way of doing this sensibly otherwise, however – clang just gets too confused about aliasing.

    vasild commented at 10:48 am on September 29, 2022:

    It used Mutex before and mimicked a RecursiveMutex, but I totally agree with @ryanofsky’s comment and changed it to RecursiveMutex.

    Even before that there was another approach which did not need a recursive mutex, and double locking was not allowed, but you did not like that.


    ajtowns commented at 6:06 pm on September 29, 2022:
    In my opinion: we shouldn’t allow double locking (for almost all locks), and should give compile time errors when double locking is attempted. I’d love to see a variant of this that achieved both those things.

    vasild commented at 9:51 am on October 12, 2022:
    Done - in the latest incarnation no double locking is allowed.
  37. ajtowns commented at 5:07 pm on September 28, 2022: contributor
    I’m still Approach NACK on this. :(
  38. fanquake referenced this in commit 2e77dff744 on Oct 11, 2022
  39. DrahtBot added the label Needs rebase on Oct 11, 2022
  40. vasild force-pushed on Oct 12, 2022
  41. vasild commented at 9:50 am on October 12, 2022: contributor
    11ac578f91...70dac32e76: rebase due to conflicts + update as per the discussion in #25390 (review)
  42. vasild renamed this:
    sync: introduce a thread-safe smart container and use it to remove a bunch of "GlobalMutex"es
    sync: introduce a thread-safe generic container and use it to remove a bunch of "GlobalMutex"es
    on Oct 12, 2022
  43. DrahtBot removed the label Needs rebase on Oct 12, 2022
  44. in src/net.cpp:343 in 121bee1947 outdated
    342 
    343 bool IsReachable(enum Network net)
    344 {
    345-    LOCK(g_maplocalhost_mutex);
    346-    return !vfLimited[net];
    347+    return WITH_SYNCED_LOCK(g_reachable_networks, p, return p->count(net) > 0);
    


    ajtowns commented at 12:44 pm on October 18, 2022:
    Seems clearer to write that as two lines (SYNCED_LOCK(); return ..) to me.

    vasild commented at 1:08 pm on November 2, 2022:
    Done.
  45. in src/sync.h:496 in c6f1d58b04 outdated
    461+protected:
    462+    mutable Mutex m_mutex;
    463+    T m_obj GUARDED_BY(m_mutex);
    464+};
    465+
    466+#define SYNCED_LOCK(synced, name) decltype(synced)::Proxy name{synced, #synced, __FILE__, __LINE__}
    


    ajtowns commented at 12:48 pm on October 18, 2022:

    I wonder if hardcoding the proxy name would be better; something like:

    0#define SYNCED_LOCK(synced) decltype(synced)::Proxy synced##_proxy{synced, #synced, __FILE__, __LINE__}
    

    then you write SYNCED_LOCK(foo); foo_proxy->bar() and don’t have to choose a name, and don’t have to worry about remembering what p was a proxy for.


    vasild commented at 1:11 pm on November 2, 2022:

    Sometimes the hardcoded name could become too long/verbose for a too short scope: e.g. g_reachable_networks_proxy or g_reachable_networks_locked when it is just used on the line below and nowhere else.

    I added now the possibility to call SYNCED_LOCK() with 1 or 2 arguments and opted for a _locked suffix instead of _proxy for the 1-arg case. So the callers have complete flexibility on how to call it.


    vasild commented at 2:06 pm on November 2, 2022:

    That does not compile on windows: https://cirrus-ci.com/task/4559637407072256?logs=build#L814 :-/

    If you are ok with this, I will fall back to the 2-arg macro. Then the caller can pick the name depending on their preferences - p, foo_proxy, foo_locked or whatever.


    ajtowns commented at 0:00 am on November 3, 2022:
    If you want naming it to be optional, probably better to have different macros, same as LOCK(m) and WAIT_LOCK(m,g). I’m not really sure what I prefer here.

    vasild commented at 1:15 pm on November 8, 2022:
    I went back to what it was before - SYNCED_LOCK(object, name). The 1-arg macro was used in just one place and it did not compile on windows, not worth it. By selecting the name the caller can choose whether to use a short one for small scopes of a few lines, or a longer and more descriptive one for bigger scopes.
  46. in src/net.cpp:125 in 121bee1947 outdated
    116@@ -115,7 +117,10 @@ bool fDiscover = true;
    117 bool fListen = true;
    118 GlobalMutex g_maplocalhost_mutex;
    119 std::map<CNetAddr, LocalServiceInfo> mapLocalHost GUARDED_BY(g_maplocalhost_mutex);
    120-static bool vfLimited[NET_MAX] GUARDED_BY(g_maplocalhost_mutex) = {};
    121+
    122+static Synced<std::unordered_set<Network>> g_reachable_networks{
    123+    NET_UNROUTABLE, NET_IPV4, NET_IPV6, NET_ONION, NET_I2P, NET_CJDNS, NET_INTERNAL};
    


    ajtowns commented at 1:14 pm on October 18, 2022:
    Changing this from a bool array to an unordered set seems surprising, and inverting the logic means introducing an extra place that has to be modified when adding new nets. Wouldn’t Synced<std::bitset<NET_MAX>> g_limited_networks make more sense?

    vasild commented at 1:13 pm on November 2, 2022:

    My reasoning is that the most natural way to express this is to have a list of networks that are reachable (or unreachable if the logic is to be inverted) and std::set is the best way to do that. Having an array of bools is a bit hackish as it requires that all enums are sequential and start at 0 and requires the existence of NET_MAX which smells bad. My plan is to remove NET_MAX. Ideally, enum lables should be used without depending on their specific integer values.

    Anyway, if this is controveral I will ditch that change and leave it as vfLimited[NET_MAX] since it is not the main topic of this PR. Should I?


    ajtowns commented at 7:48 am on November 9, 2022:
    enums are sequential and automatically start at 0; and NET_MAX is added for precisely that purpose. That’s the way these things are designed to be used, it’s not hackish or a bad smell…

    vasild commented at 2:43 pm on November 11, 2022:
    I disagree, but that discussion is for another PR. I reverted it back to what it is in master - an array of bools.
  47. in src/net.cpp:124 in 121bee1947 outdated
    116@@ -115,7 +117,10 @@ bool fDiscover = true;
    117 bool fListen = true;
    118 GlobalMutex g_maplocalhost_mutex;
    119 std::map<CNetAddr, LocalServiceInfo> mapLocalHost GUARDED_BY(g_maplocalhost_mutex);
    120-static bool vfLimited[NET_MAX] GUARDED_BY(g_maplocalhost_mutex) = {};
    121+
    122+static Synced<std::unordered_set<Network>> g_reachable_networks{
    123+    NET_UNROUTABLE, NET_IPV4, NET_IPV6, NET_ONION, NET_I2P, NET_CJDNS, NET_INTERNAL};
    124+
    


    ajtowns commented at 1:21 pm on October 18, 2022:

    An alternative approach to splitting these two entries between separate mutexes would be to create an additional struct:

    0struct NetworkReachability
    1{
    2    std::map<CNetAddr, LocalServiceInfo> local_addrs;
    3    std::set<Network> reachable_nets;
    4};
    5Synced<NetworkReachability> g_network_reachability;
    

    and then SYNCED_LOCK(g_network_reachability, p); lets you access p->local_addrs or p->reachable_nets. Hard to see it making much practical difference here either way though.


    vasild commented at 1:13 pm on November 2, 2022:

    … separate mutexes …

    What you propose will use just one mutex? Anyway, the list of reachable networks is unrelated to the list of local addresses. I do not think they should be grouped into one struct or protected by the same mutex.


    ajtowns commented at 11:50 pm on November 2, 2022:
    Yes, it was an “alternative .. to .. separate mutexes”. Mostly intended to show using this pattern doesn’t require moving everything to a dedicated mutex if that’s not desired. In this case, like I said “hard to see it making much practical difference” though.

    vasild commented at 1:12 pm on November 8, 2022:
    Ok, I am leaving it as it is and marking this as resolved. I think it is natural to guard unrelated things with separate mutexes.
  48. ajtowns commented at 8:05 am on October 19, 2022: contributor

    Approach ACK

    Don’t think the bool-array to unordered_set makes much sense, otherwise looks good.

  49. sidhujag referenced this in commit 5984166da6 on Oct 23, 2022
  50. vasild force-pushed on Nov 2, 2022
  51. vasild commented at 1:08 pm on November 2, 2022: contributor
    70dac32e76...c3e4c34219: rebase and address suggestions
  52. vasild force-pushed on Nov 8, 2022
  53. vasild commented at 1:08 pm on November 8, 2022: contributor
    c3e4c34219...0f457d2bdd: rebase and go back to a single macro SYNCED_LOCK() with 2 arguments.
  54. vasild force-pushed on Nov 8, 2022
  55. vasild commented at 4:34 pm on November 8, 2022: contributor
    0f457d2bdd...6c0bcd0928: update comment
  56. vasild force-pushed on Nov 11, 2022
  57. vasild commented at 2:45 pm on November 11, 2022: contributor
    6c0bcd0928...f8f7e62f81: don’t change the type of the variable that holds the list of reachable networks (an array of bools), see #25390 (review)
  58. DrahtBot added the label Needs rebase on Jan 30, 2023
  59. vasild force-pushed on Feb 1, 2023
  60. vasild commented at 3:04 pm on February 1, 2023: contributor
    f8f7e62f81...3db109e794: rebase due to conflicts @ajtowns, I think I addressed all your suggestions?
  61. DrahtBot removed the label Needs rebase on Feb 1, 2023
  62. DrahtBot added the label Needs rebase on Feb 17, 2023
  63. vasild force-pushed on Feb 24, 2023
  64. vasild commented at 2:09 pm on February 24, 2023: contributor
    3db109e794...e42ce20c65: rebase due to conflicts
  65. DrahtBot removed the label Needs rebase on Feb 24, 2023
  66. DrahtBot added the label Needs rebase on Apr 3, 2023
  67. vasild force-pushed on Apr 7, 2023
  68. vasild commented at 4:21 pm on April 7, 2023: contributor
    e42ce20c65...ebfe47e594: rebase due to conflicts
  69. DrahtBot removed the label Needs rebase on Apr 7, 2023
  70. jonatack commented at 3:17 pm on April 18, 2023: contributor
    Light Approach ACK after reading the code and the excellent review discussions. It looks like this has been progressively honed and improved quite a bit. Will test and review.
  71. in src/net.cpp:120 in ebfe47e594 outdated
    121-static bool vfLimited[NET_MAX] GUARDED_BY(g_maplocalhost_mutex) = {};
    122+
    123+/**
    124+ * The local network addresses of this node.
    125+ */
    126+Synced<std::map<CNetAddr, LocalServiceInfo>> g_my_net_addr;
    


    jonatack commented at 3:46 pm on April 18, 2023:
    a49907318fff6d Naming nit, would this be better plural i.e. g_my_net_addrs, g_local_addrs or g_my_local_addrs?
  72. in src/net.cpp:313 in ebfe47e594 outdated
    322@@ -318,23 +323,20 @@ bool AddLocal(const CNetAddr &addr, int nScore)
    323 
    324 void RemoveLocal(const CService& addr)
    325 {
    326-    LOCK(g_maplocalhost_mutex);
    327     LogPrintf("RemoveLocal(%s)\n", addr.ToStringAddrPort());
    328-    mapLocalHost.erase(addr);
    329+    WITH_SYNCED_LOCK(g_my_net_addr, p, p->erase(addr));
    


    jonatack commented at 3:48 pm on April 18, 2023:

    Style nit, unsure, would it be a good idea to use named args in the WITH_SYNCED_LOCK and SYNCED_LOCK calls? More verbose but more clear, though it’s not too non-intuitive.

    0    WITH_SYNCED_LOCK(g_my_net_addr, /*name=*/p, /*code=*/p->erase(addr));
    

    vasild commented at 7:02 am on April 19, 2023:
    I think that’s a bit too verbose.
  73. in src/rpc/net.cpp:693 in ebfe47e594 outdated
    659@@ -660,13 +660,12 @@ static RPCHelpMan getnetworkinfo()
    660     }
    661     UniValue localAddresses(UniValue::VARR);
    662     {
    663-        LOCK(g_maplocalhost_mutex);
    664-        for (const std::pair<const CNetAddr, LocalServiceInfo> &item : mapLocalHost)
    665-        {
    666+        SYNCED_LOCK(g_my_net_addr, p);
    667+        for (const auto& [addr, service_info] : *p) {
    


    jonatack commented at 3:50 pm on April 18, 2023:
    Nice cleanup.
  74. in src/sync.h:496 in ebfe47e594 outdated
    461+protected:
    462+    mutable Mutex m_mutex;
    463+    T m_obj GUARDED_BY(m_mutex);
    464+};
    465+
    466+#define SYNCED_LOCK(synced, name) decltype(synced)::Proxy name{synced, #synced, __FILE__, __LINE__}
    


    jonatack commented at 3:54 pm on April 18, 2023:
    373df61bc56 Style nit, unsure, would something like local_instance or local_name read better than name if callers use named args? Feel free to ignore.

    vasild commented at 7:05 am on April 19, 2023:
    Named args at call sites look too verbose to me and here a short name is ok because the scope if very limited - only used on one line.
  75. jonatack commented at 4:09 pm on April 18, 2023: contributor

    Light initial ACK ebfe47e5942fa5e0ce21b44511258b6dfa4cf621 modulo poking around with the thread safety coverage/guarantees to break them. Compiled and ran unit tests at each commit with ARM64 clang 16 (except the scripted-diff ones).

    Feel free to ignore the comments below unless you need to repush.

  76. hebasto commented at 3:26 pm on May 22, 2023: member

    Concept ~0. While the first commit introduces new code in sync.h that needs maintaining and unit test coverage, the following commits do not demonstrate convincing reasons for it.

    but it is possible to abuse the mutex and start using it to protect some more, possibly unrelated stuff (we already have this in the current code).

    Concept ACK on improving this.

  77. DrahtBot added the label CI failed on May 30, 2023
  78. DrahtBot removed the label CI failed on May 31, 2023
  79. DrahtBot added the label Needs rebase on Jul 13, 2023
  80. vasild force-pushed on Jul 19, 2023
  81. vasild commented at 3:11 pm on July 19, 2023: contributor

    ebfe47e594...38c0d628e0: rebase due to conflicts @hebasto, if the purpose of the low level code is to make the higher level code more robust and less bug-prone, then I think Synced achieves this. The pattern:

    0Mutex m;
    1Type my_variable GUARDED_BY(m);
    

    is very common. The Synced structure encapsulates both, enforces the idiom and makes it impossible to “to abuse the mutex and start using it to protect some more, possibly unrelated stuff”.

    It also helps remove GlobalMutex usages. GlobalMutex could be confusing.

  82. DrahtBot removed the label Needs rebase on Jul 19, 2023
  83. achow101 added the label Needs Conceptual Review on Sep 20, 2023
  84. DrahtBot added the label Needs rebase on Sep 21, 2023
  85. vasild commented at 1:38 pm on October 5, 2023: contributor
    38c0d628e0...b749c4d2ca: rebase due to conflicts
  86. vasild force-pushed on Oct 5, 2023
  87. DrahtBot removed the label Needs rebase on Oct 5, 2023
  88. DrahtBot added the label Needs rebase on Oct 19, 2023
  89. vasild force-pushed on Oct 23, 2023
  90. vasild commented at 12:21 pm on October 23, 2023: contributor
    b749c4d2ca...92a444d0d7: rebase due to conflicts, drop commit 93e60b9a0c3ee44fbf1f64c91e908c3e507d1d7f net: detach vfLimited from g_maplocalhost_mutex because vfLimited + SetReachable() have already been encapsulated (see 6e308651c441cbf8763c67cc099c538c333c2872 from #27071).
  91. DrahtBot removed the label Needs rebase on Oct 23, 2023
  92. DrahtBot added the label CI failed on Oct 23, 2023
  93. DrahtBot removed the label CI failed on Oct 25, 2023
  94. DrahtBot added the label Needs rebase on Dec 13, 2023
  95. sync: introduce a thread safe generic container
    It can contain anything, encapsulates a mutex to synchronize access to
    it and enfoces access through it.
    7de603958e
  96. scripted-diff: rename mapLocalHost to g_my_net_addr
    -BEGIN VERIFY SCRIPT-
    sed -i -e 's/mapLocalHost/g_my_net_addr/g' $(git grep -l mapLocalHost)
    -END VERIFY SCRIPT-
    7b3a96f31e
  97. net: use Synced<T> for g_my_net_addr and remove g_maplocalhost_mutex
    Convert `g_my_net_addr` to use `Synced<T>` and ditch the global mutex
    `g_maplocalhost_mutex`.
    5e14e9f8ba
  98. scripted-diff: rename deadlineTimers to g_deadline_timers
    -BEGIN VERIFY SCRIPT-
    sed -i -e 's/deadlineTimers/g_deadline_timers/g' $(git grep -l deadlineTimers)
    -END VERIFY SCRIPT-
    5ac59c07f7
  99. rpc: use Synced<T> for g_deadline_timers and remove g_deadline_timers_mutex
    Convert `g_deadline_timers` to use `Synced<T>` and ditch the global mutex
    `g_deadline_timers_mutex`.
    829ecd3cfc
  100. scripted-diff: rename dir_locks to g_dir_locks
    -BEGIN VERIFY SCRIPT-
    sed -i -e 's/\<dir_locks\>/g_dir_locks/g' $(git grep -l dir_locks)
    -END VERIFY SCRIPT-
    f751888f9b
  101. rpc: use Synced<T> for g_dir_locks and remove cs_dir_locks
    Convert `g_dir_locks` to use `Synced<T>` and ditch the global mutex
    `cs_dir_locks`.
    909b600598
  102. wallet: use Synced<T> for g_loading_wallet_set and remove g_loading_wallet_mutex
    Convert `g_loading_wallet_set` to use `Synced<T>` and ditch the global mutex
    `g_loading_wallet_mutex`.
    37c5a1c5b5
  103. wallet: use Synced<T> for g_unloading_wallet_set and remove g_wallet_release_mutex
    Convert `g_unloading_wallet_set` to use `Synced<T>` and ditch the global mutex
    `g_wallet_release_mutex`.
    f3489638dc
  104. vasild force-pushed on Dec 13, 2023
  105. vasild commented at 2:35 pm on December 13, 2023: contributor
    92a444d0d7...f3489638dc: rebase due to conflicts
  106. DrahtBot removed the label Needs rebase on Dec 13, 2023
  107. jonatack commented at 8:57 pm on January 6, 2024: contributor
    Will review the updates since my initial ACK #25390#pullrequestreview-1390434676.
  108. maflcko commented at 3:17 pm on April 9, 2024: member
    Are there any examples from the last couple of years where this would have helped to prevent a bug or other issue?
  109. vasild commented at 12:57 pm on April 25, 2024: contributor

    Are there any examples from the last couple of years where this would have helped to prevent a bug or other issue?

    Here is an example where two unrelated variables were guarded by the same mutex (already fixed in the latest code):

    https://github.com/bitcoin/bitcoin/blob/c42ded3d9bda8b273780a4a81490bbf1b9e9c261/src/net.cpp#L116-L118

  110. vasild commented at 1:00 pm on April 25, 2024: contributor
    Closing this due to no interest from reviewers for a long time plus currently my hands are full with more important things. Would be happy to revisit if there is interest.
  111. vasild closed this on Apr 25, 2024

  112. maflcko commented at 1:05 pm on April 25, 2024: member

    Are there any examples from the last couple of years where this would have helped to prevent a bug or other issue?

    Here is an example where two unrelated variables were guarded by the same mutex (already fixed in the latest code):

    I don’t think this is an example of a bug or other issue.

    This was added in commit b312cd770701aa806e9b264154646f481d212c1c to fix a tsan warning. I guess the annotation was using the wrong mutex, but I doubt this will happen in newly written code. Also, the mistake in this case seems harmless, as the compiled binary of bitcoind didn’t even change: #13123 (comment)


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-01 10:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me