[Development] Development Digest, Vol 72, Issue 38
Vladimir Moolle
vmoolle at ics.com
Sat Sep 30 00:01:28 CEST 2017
Hi, not sure if this would look funny (would it?), but may be referencing
the stock documentation page at
http://doc.qt.io/qt-5.9/threads-reentrancy.html could help avoid further
confusion (and unnecessarily heating the thread).
On Sat, Sep 30, 2017 at 12:50 AM, <development-request at qt-project.org>
wrote:
> Send Development mailing list submissions to
> development at qt-project.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.qt-project.org/mailman/listinfo/development
> or, via email, send a message with subject or body 'help' to
> development-request at qt-project.org
>
> You can reach the person managing the list at
> development-owner at qt-project.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Development digest..."
>
>
> Today's Topics:
>
> 1. Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)
> (Ren? J.V. Bertin)
> 2. Re: Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)
> (Konrad Rosenbaum)
> 3. Re: Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)
> (Ren? J.V. Bertin)
> 4. Re: Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)
> (Milian Wolff)
> 5. Re: Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)
> (Thiago Macieira)
> 6. Re: Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)
> (Thiago Macieira)
> 7. Re: Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)
> (Ren? J. V. Bertin)
> 8. Re: Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)
> (Thiago Macieira)
> 9. Re: Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)
> (Ren? J. V. Bertin)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 29 Sep 2017 11:35:40 +0200
> From: Ren? J.V. Bertin <rjvbertin at gmail.com>
> To: "'development at qt-project.org'" <development at qt-project.org>
> Subject: [Development] Should QFileSystemWatcher be thread-safe? (Qt
> 5.8.0)
> Message-ID: <1811414.Bhil7bdXDD at bola>
> Content-Type: text/plain; charset="us-ascii"
>
> Hi,
>
> I've been running into issues using adding and removing entries from QFSW
> in concurrent threads. This is in KDevelop while adding all directories of
> multiple projects (source trees) to a single QFSW instance per project. The
> app is somewhat complex so I think but am not entirely certain that the
> source trees are read on a single background thread; if I'm correct each
> QFSW instance is fed from a single thread only. (The QFSW instances are
> created and "slotted" on the main thread.)
>
> The issues range from more or less severe lock-ups on Mac (the hard
> lock-ups occur in the FSEvents backend) to double-free crashes on Linux. I
> can only avoid them by using a single mutex that ensures that only 1 thread
> can use the class at a time.
>
> The QFSW documentation only mentions the class is reentrant. Is QFSW
> supposed to be thread-safe (at least at the class level), on all platforms?
> Or do we have to provide our own protection mechanisms if we want to use
> the class this way?
>
> Thanks,
> R.
>
>
> ------------------------------
>
> Message: 2
> Date: Fri, 29 Sep 2017 11:49:32 +0200
> From: "Konrad Rosenbaum" <konrad at silmor.de>
> To: "'development at qt-project.org'" <development at qt-project.org>
> Subject: Re: [Development] Should QFileSystemWatcher be thread-safe?
> (Qt 5.8.0)
> Message-ID:
> <cf36d378c3d5fe6ecc72ec620771965b.squirrel at squirrel.six.silmor.de>
> Content-Type: text/plain;charset=iso-8859-1
>
> Hi,
>
> On Fri, September 29, 2017 11:35, Ren? J.V. Bertin wrote:
> > I've been running into issues using adding and removing entries from QFSW
> > in concurrent threads.
> [cut]
> > The QFSW documentation only mentions the class is reentrant. Is QFSW
> > supposed to be thread-safe (at least at the class level), on all
> > platforms? Or do we have to provide our own protection mechanisms if we
> > want to use the class this way?
>
> In the Qt documentation (and most other sane docu) "reentrant" means the
> class can be used on different threads, but each instance must be
> restricted to a single thread. You can have different instances on
> different threads.
>
> Actually documenting it as "thread safe" would imply that even instances
> can be shared across threads. This is much harder to implement for (I
> believe) obvious reasons.
>
> This is why you were able to solve the problem with a Mutex: you ensured
> that the instance was only used by one instance at a time.
>
> Another way is to explicitly use Qt::QueuedConnection to feed the QFSW or
> to untangle your code to ensure each QSFW is only visible from one thread.
>
>
> Konrad
>
>
>
> ------------------------------
>
> Message: 3
> Date: Fri, 29 Sep 2017 14:54:53 +0200
> From: Ren? J.V. Bertin <rjvbertin at gmail.com>
> To: Milian Wolff <milian.wolff at kdab.com>
> Cc: development at qt-project.org
> Subject: Re: [Development] Should QFileSystemWatcher be thread-safe?
> (Qt 5.8.0)
> Message-ID: <1637195.QtcOVzZ2nT at portia.local>
> Content-Type: text/plain; charset="us-ascii"
>
> On Friday September 29 2017 14:27:37 Milian Wolff wrote:
>
> > behavior you are supposedly seeing.
>
> So you still haven't bothered trying to reproduce this or prove me wrong,
> aside implying I'm making this all up? In that case I'm not sure why *you*
> come trolling on a post that intends to find out if QFSW is being used in a
> way it's not supposed to be used - in which case there is no point in
> wasting time with upstream demonstrators and patches.
>
>
>
>
> ------------------------------
>
> Message: 4
> Date: Fri, 29 Sep 2017 14:27:37 +0200
> From: Milian Wolff <milian.wolff at kdab.com>
> To: development at qt-project.org
> Cc: Ren? J.V. Bertin <rjvbertin at gmail.com>
> Subject: Re: [Development] Should QFileSystemWatcher be thread-safe?
> (Qt 5.8.0)
> Message-ID: <5125747.d2nQ8lOia2 at agathebauer>
> Content-Type: text/plain; charset="iso-8859-1"
>
> On Freitag, 29. September 2017 11:35:40 CEST Ren? J.V. Bertin wrote:
> > Hi,
> >
> > I've been running into issues using adding and removing entries from
> QFSW in
> > concurrent threads. This is in KDevelop while adding all directories of
> > multiple projects (source trees) to a single QFSW instance per project.
> The
> > app is somewhat complex so I think but am not entirely certain that the
> > source trees are read on a single background thread; if I'm correct each
> > QFSW instance is fed from a single thread only. (The QFSW instances are
> > created and "slotted" on the main thread.)
> >
> > The issues range from more or less severe lock-ups on Mac (the hard
> lock-ups
> > occur in the FSEvents backend) to double-free crashes on Linux. I can
> only
> > avoid them by using a single mutex that ensures that only 1 thread can
> use
> > the class at a time.
> >
> > The QFSW documentation only mentions the class is reentrant. Is QFSW
> > supposed to be thread-safe (at least at the class level), on all
> platforms?
> > Or do we have to provide our own protection mechanisms if we want to use
> > the class this way?
>
> Please Rene, before you waste more time by getting yet more people
> involved.
> Build a MWE, i.e. a trivial single-file Qt application that exhibits the
> behavior you are supposedly seeing. Then when you think you are sure what
> is
> going on we can talk. Don't put up open hypotheses on random channels -
> it's
> really draining everyones energy for no good gain. And don't come again
> with
> the "I don't have time" - it's less time to create a MWE than to write
> dozens
> of mails that then have no outcome.
>
> --
> Milian Wolff | milian.wolff at kdab.com | Senior Software Engineer
> KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
> Tel: +49-30-521325470
> KDAB - The Qt Experts
>
>
>
>
> ------------------------------
>
> Message: 5
> Date: Fri, 29 Sep 2017 08:51:54 -0700
> From: Thiago Macieira <thiago.macieira at intel.com>
> To: <development at qt-project.org>
> Subject: Re: [Development] Should QFileSystemWatcher be thread-safe?
> (Qt 5.8.0)
> Message-ID: <2623428.N9ttvi6u71 at tjmaciei-mobl1>
> Content-Type: text/plain; charset="us-ascii"
>
> On sexta-feira, 29 de setembro de 2017 02:49:32 PDT Konrad Rosenbaum wrote:
> > This is why you were able to solve the problem with a Mutex: you ensured
> > that the instance was only used by one instance at a time.
>
> That is not a full solution. You may be able to serialise all *your* access
> with a mutex, but since QFSW is doing some form of I/O, it may still be
> woken
> up by another thread and do work there. You need to block that thread's
> event
> loop (and you need to know which thread that is yourself) if you want to
> ensure full serialisation.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> ------------------------------
>
> Message: 6
> Date: Fri, 29 Sep 2017 08:52:45 -0700
> From: Thiago Macieira <thiago.macieira at intel.com>
> To: <development at qt-project.org>
> Subject: Re: [Development] Should QFileSystemWatcher be thread-safe?
> (Qt 5.8.0)
> Message-ID: <7177843.JS2enr4j1R at tjmaciei-mobl1>
> Content-Type: text/plain; charset="iso-8859-1"
>
> On sexta-feira, 29 de setembro de 2017 05:54:53 PDT Ren? J.V. Bertin wrote:
> > On Friday September 29 2017 14:27:37 Milian Wolff wrote:
> > > behavior you are supposedly seeing.
> >
> > So you still haven't bothered trying to reproduce this or prove me wrong,
> > aside implying I'm making this all up? In that case I'm not sure why
> *you*
> > come trolling on a post that intends to find out if QFSW is being used
> in a
> > way it's not supposed to be used - in which case there is no point in
> > wasting time with upstream demonstrators and patches.
>
> We're not claiming that you have no problem. We're saying the problem is
> probably in your own code.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> ------------------------------
>
> Message: 7
> Date: Fri, 29 Sep 2017 21:50:07 +0200
> From: Ren? J. V. Bertin <rjvbertin at gmail.com>
> To: development at qt-project.org
> Subject: Re: [Development] Should QFileSystemWatcher be thread-safe?
> (Qt 5.8.0)
> Message-ID: <6416392.OqD2sMKAdo at patux.local>
> Content-Type: text/plain; charset="ISO-8859-1"
>
> Thiago Macieira wrote:
>
>
> > We're not claiming that you have no problem. We're saying the problem is
> > probably in your own code.
>
> It's a kind of chicken-or-egg question. I'm indeed doing something that
> triggers
> a problem which is clearly deep inside Qt code. What I want to know is
> whether
> it's acceptable/foreseeable that that problem gets triggered or if it
> shouldn't
> be triggered.
>
> For now I'm going to take away the conclusion that the mutex indeed has to
> be in
> my code.
>
> > In the Qt documentation (and most other sane docu) "reentrant" means the
> > class can be used on different threads, but each instance must be
> > restricted to a single thread.
>
> So I didn't entirely mix up reentrant and thread-safe after all.
> I may end up trying to write a demonstrator after all to see if I can
> reproduce
> the lock-ups on Mac with a minimal example.
>
> > you ensured that the instance was only used by one instance at a time.
>
> Probably even more than that: only a single QFSW instance used at any given
> moment.
>
> How does QFSW handle a situation in which an already watched directory is
> changed while another directory is being added? Would something else
> happen when
> adding directories happens on a background thread while change signals are
> connected to slots on the main thread?
>
> > Another way is to explicitly use Qt::QueuedConnection to feed the QFSW
>
> A signal from the background dir reader thread connected to a slot on the
> main
> thread? That could be an interesting alternative to investigate indeed.
>
> R.
>
>
>
> ------------------------------
>
> Message: 8
> Date: Fri, 29 Sep 2017 13:09:41 -0700
> From: Thiago Macieira <thiago.macieira at intel.com>
> To: <development at qt-project.org>
> Subject: Re: [Development] Should QFileSystemWatcher be thread-safe?
> (Qt 5.8.0)
> Message-ID: <3989100.DsVr9K6CMQ at tjmaciei-mobl1>
> Content-Type: text/plain; charset="iso-8859-1"
>
> On sexta-feira, 29 de setembro de 2017 12:50:07 PDT Ren? J. V. Bertin
> wrote:
> > Thiago Macieira wrote:
> > > We're not claiming that you have no problem. We're saying the problem
> is
> > > probably in your own code.
> >
> > It's a kind of chicken-or-egg question. I'm indeed doing something that
> > triggers a problem which is clearly deep inside Qt code. What I want to
> > know is whether it's acceptable/foreseeable that that problem gets
> > triggered or if it shouldn't be triggered.
> >
> > For now I'm going to take away the conclusion that the mutex indeed has
> to
> > be in my code.
>
> Even then we won't support you. QFSW, like almost all QObjects, must only
> be
> touched by one thread. Anything else you do is at your own risk.
>
> > How does QFSW handle a situation in which an already watched directory is
> > changed while another directory is being added? Would something else
> happen
> > when adding directories happens on a background thread while change
> signals
> > are connected to slots on the main thread?
>
> The Darwin implementation has an internal mutex so that the dispatcher
> thread
> (I'm assuming) is able to get to the current state.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
> Software Architect - Intel Open Source Technology Center
>
>
>
> ------------------------------
>
> Message: 9
> Date: Fri, 29 Sep 2017 23:47:29 +0200
> From: Ren? J. V. Bertin <rjvbertin at gmail.com>
> To: development at qt-project.org
> Subject: Re: [Development] Should QFileSystemWatcher be thread-safe?
> (Qt 5.8.0)
> Message-ID: <1775543.FOxzUkLYz6 at portia.local>
> Content-Type: text/plain; charset="ISO-8859-1"
>
> Thiago Macieira wrote:
>
> >> How does QFSW handle a situation in which an already watched directory
> is
> >> changed while another directory is being added? Would something else
> happen
> >> when adding directories happens on a background thread while change
> signals
> >> are connected to slots on the main thread?
> >
> > The Darwin implementation has an internal mutex so that the dispatcher
> thread
> > (I'm assuming) is able to get to the current state.
>
> Can I assume that the other platforms have a similar feature that blocks
> access
> to the current state while entries are being added or removed (and also
> defers
> adding/removing entries while sending out change notifications)?
>
> R
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Development mailing list
> Development at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>
>
> ------------------------------
>
> End of Development Digest, Vol 72, Issue 38
> *******************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20170930/db200667/attachment.html>
More information about the Development
mailing list