[Development] Mutex future directions

[Thiago Macieira]

On sábado, 19 de maio de 2012 10.48.22, Charley Bay wrote:
> <snip, QMutex future-direction-discussion>
> 
> I've followed this very technical discussion from the beginning--
> impressive array of topics.  Thanks to all digging into this, and great
> thanks to Thiago for opening the issue.
> 
> I have a question on the point of "recursive-locks":
> 
> I understand "recursive-locks" (e.g., "recursive-lock-count") were not part
> of the original QMutex, and were added later.  I understand it's merely a
> count-lock-wrapper around the *real* lock.

Actually, no. QMutex has always supported recursive mutexes.

The current implementation is such that only the non-recursive kind is 
optimised. The reason for that is that implementing them with futexes, Mach 
semaphores and Windows events was quite easy. By bypassing pthreads, we gain a 
little in performance.

Implementing the recursive mutex on top of that wasn't very difficult. Though, 
to be honest, it requires constant reviewing by other people. We had the same 
solution in libdbus-1 and it had race conditions gone unnoticed for years. It 
also took the glibc developers 2 years to get all the locking primitives right 
on pthread.

> Further, I understand that "recursive-locks" can help
> "get-one-out-of-a-corner-in-which-they-find-themselves", where the lock was
> already taken by the current thread and we don't want to deadlock when
> accidentally attempting the lock again.  This problem tends to occur in
> designs where the lock is implied for (small) transactional operations,
> *and* for (large) transactional operations that tend to trigger implicit
> execution of those (small) transactional operations *within* the large
> transactional operation.

Often, it's for the big locks that you can't completely assure that no 
recursion will happen. The small ones usually protect a very limited code path 
and it's doable to prove that it will never recurse.

For that reason, since you have lots of code to protect and you're likely not 
to be locking and unlocking that often, the small loss of performance is 
acceptable. That said, the recursive lock isn't that much worse. It's actually 
quite good.

> I agree with Oliver:
> 
>  *) I think recursive mutex don't deserve improvements on the detriment of
> >    normal ones
> >
>
> From a practical standpoint, I understand why recursive mutexes exist (see
> description above).  However, from a *logical/cleanliness* standpoint,
> every time I've used them, I've later re-factored to where I do *not* do
> mutex-recursion.  (Currently I use no recursive mutexes.)
> 
> Do people really use them intentionally (do designs legitimately require
> them), or are they as I suspect, a way to "get-yourself-out-of-trouble"
> when your design failed to account for proper transactional granularity?

There are a few cases where recursive locks are legitimate uses. For example, 
in QtDBus, one of the mutexes in QDBusConnection is recursive because we are 
calling out to the libdbus-1 library to handle some condition, which may cause 
it to raise further changes to the socket notifiers and timers via the same 
callbacks that are used by other threads. The change can be executed on this 
thread while the mutex is being held, but not so by other threads.

In other cases, you're probably right that the recursive mutex is just a 
gimmick to "I can't prove it will not recurse". But then you need to be 
careful in writing your code that operates under the lock, like I did for 
QtDBus. A recursion from the same thread can be just as bad as simultaneous 
incursion from another thread -- e.g., imagine you kept an iterator to a 
vector, but the recursed call caused a reallocation.

So in fact, the rules of thumb about locks are: 
 - protect the smallest code path that you really need to protect
 - do not call out to functions you don't control, never call back to the user

To protect the smallest code path, you often need lots of locks and lots of 
locking, which is the goal of the current implementation by Olivier and Brad 
(cheap to create, cheap to lock if not contended).

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center
     Intel Sweden AB - Registration Number: 556189-6027
     Knarrarnäsgatan 15, 164 40 Kista, Stockholm, Sweden
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.qt-project.org/pipermail/development/attachments/20120519/cd69e45d/attachment.sig>