[Qt-interest] QtScript signal/slot thread issues
Russell Ryan
rryan at MIT.EDU
Wed Jan 28 21:46:45 CET 2009
Hi Kent,
I'm RJ Ryan, another Mixxx developer working with Sean on the QtScript
bugs we're having.
First to summarize:
- We've eliminated some bugs by following your suggestions to avoid
multiple threads using the QScriptEngine at once
- We still have a crash bug that seems to be related to the
QScriptEngine. It takes more effort to produce, but it amounts to
generating a lot of MIDI signals (rapidly moving the sliders/knobs on
the device up and down).
MidiScriptEngine, our class that wraps QScriptEngine, is now a QThread,
which in its run() method, creates the QScriptEngine, then calls exec(),
so that the event loop starts running. Our goal is that all of our
interactions with the QScriptEngine go through this thread.
The former 'execute()' method on MidiScriptEngine (which previously
executed QScriptEngine::call() directly) now generates a custom QEvent,
which is captured by the MidiScriptEngine::event() method, which
branches to safeExecute, a private method which does the
QScriptEngine::call method. We have verified that the Thread ID of
MidiScriptEngine::event() calls is the same as that of the
MidiScriptEngine::run() method.
When MidiScriptEngine is created via the main application thread, we
immediately call moveToThread() on itself, so that its thread context is
switched to its own thread. I was unsure if this was necessary.
Now there are 3 main ways that something can happen to the engine:
1) a MIDI device produces a signal, which is received by a MidiObject
QThread, and results in an MidiScriptEngine::execute() call, which is
marshalled into the MidiScriptEngine thread via the machinery I
described above.
2) A ControlObject emits a valueChanged() signal. If the control
object's valueChanged() signal is qScriptConnect()'d to a script
function, then the QScriptEngine calls the function once the slot arrives.
3) All scripts are provided a reference to the MidiScriptEngine object,
and they are able to call MidiScriptEngine::connectControl(), which
results in a qScriptConnect or Disconnect.
In your previous emails you said that qScriptConnect uses
AutoConnections for the connections. In order to maintain our mutual
exclusion of engine use, we would like the slot side of the
qScriptConnect to execute in the MidiScriptEngine thread. People in #qt
told me that QueuedConnections slots or AutoConnection slots across
threads would show up in an event() method as a QEvent::MetaCall event.
We have tested this, and it does not seem to be the case. Signals that
are emitted are arriving at the script slots, but not showing up in the
MidiScriptEngine::event() method. (We simply print every event, handle
our custom events, and return the default implementation otherwise).
Question 1
=====
We have verified that all ControlObjects are created in the main
application thread. We have also verified that the QScriptEngine is
created in the MidiScriptEngine thread.
Is it true that under these circumstances, an AutoConnection should
queue all of its slots in our event loop, instead of being directly
fired? Is it possible to test this?
Question 2
=====
Is it safe for a QtScript to invoke a function that would affect the
engine? In our case, a QtScript can make a call to the
MidiScriptEngine::connectControl, which results in a qScriptConnect on
the engine. Would this re-enter the QScriptEngine ? The callstack would
be something like QScriptEngine::call() -> (script) ->
MidiScriptEngine::connectControl() -> qScriptConnect(). What if that
ended up call()'ing something instead of qScriptConnect()?
Characteristics of our crash
=====
If you disable qScriptConnects (i.e. valueChanged signals won't reach
the script) then the crash goes away.
If you disable execute() calls from the MIDI signals coming from the
device, then the crash goes away.
Some other notes
=====
- execute() calls can never occur concurrently because only one other
thread delivers them
All work on this issues is now going on in this branch:
https://mixxx.svn.sourceforge.net/svnroot/mixxx/branches/Features_MIDIScriptAutoReaction/
If you'd like to build it, we have a special MIDI xml/script
crashTest.xml/js, which should allow you to easily crash it if you have
a MIDI controller.
Here are a number of backtraces. In some cases the MidiObject thread is
listed as crashed, while in others the MidiScriptEngine thread is listed
as crashed. When the MidiObject thread is crashed, it appears as if the
MidiScriptEngine thread has been corrupted.
http://pastebin.com/d49184d0
In this backtrace, the MidiScriptEngine thread has crashed.
http://pastebin.com/d723da46c
In this one, the MidiObjectAlsaSeq thread (LWP 3594) has crashed, and
the thread directly below it (the MidiScriptEngine thread LWP 3593) is
listed as corrupted.
http://pastebin.com/d17db795f
Here MidiObjectAlsaSeq has crashed, but the MidiScriptEngine thread
seems fine.
We'd appreciate any insight you can provide, since at this point we're
stuck again.
Thanks very much,
RJ Ryan
Sean M. Pappalardo wrote:
> Kent's responses attached
>
> <<--------------------------------------------------------------------------------->>
> This E-Mail message has been scanned for viruses
> and cleared by >>SmartMail<< from Smarter Technology, Inc.
> <<--------------------------------------------------------------------------------->>
>
>
> ------------------------------------------------------------------------
>
> Subject:
> Re: [Qt-interest] QtScript segfault on C++ signal (dis)connection/firing
> From:
> Kent Hansen <khansen at trolltech.com>
> Date:
> Wed, 14 Jan 2009 17:44:42 +0100
> To:
> qt-interest at trolltech.com
>
> To:
> qt-interest at trolltech.com
>
>
> Hi Sean,
> Any chance that the same script engine is being accessed from multiple
> threads? QtScript is not thread-safe, so be sure that there's only a
> single thread evaluating code (unless you've explicitly added a
> synchronization mechanism), and make sure that all connections are
> queued. If QtScript intercepts a signal and starts to handle it, and
> then another thread starts evaluating a script while this is going on,
> one likely result is that the QtScript call stack will get corrupted,
> which the backtrace seems to indicate.
>
> Regards,
> Kent
>
>
> Sean M. Pappalardo wrote:
>
>> Hello again.
>>
>> Just wondering if you've been able to find anything out? This issue is
>> quite literally a show-stopper, considering this is a DJ app. :)
>>
>> After a bunch more testing, the crashing doesn't happen if I don't
>> connect any of the C++ signals. I can call MidiScriptEngine::setValue()
>> all day from the script with no problems. It's crashing when signals are
>> connected and setValue() is called many times in a row, affecting
>> objects that emit signals. And it's worse on faster systems (more
>> setValue()s per second, I assume.) The signals only fire once per
>> latency period though. I did also find that increasing the latency alot
>> (400ms or more) makes it less likely to happen, depending on processor
>> speed.
>>
>> I ran valgrind with Mixxx compiled against Qt 4.5 which seems to give
>> more detail in the QtScript area, incase it helps:
>>
>> =20125== Process terminating with default action of signal 11 (SIGSEGV)
>> ==20125== Access not within mapped region at address 0x8
>> ==20125== at 0x530F078:
>> QScriptContextPrivate::execute(QScript::Code*) (qscriptcontext_p.cpp:2020)
>> ==20125== by 0x5320B15:
>> QScript::ScriptFunction::execute(QScriptContextPrivate*)
>> (qscriptcontext_p.cpp:313)
>> ==20125== by 0x5335B7A: QScriptEnginePrivate::call(QScriptValueImpl
>> const&, QScriptValueImpl const&, QList<QScriptValueImpl> const&, bool)
>> (qscriptengine_p.cpp:1252)
>> ==20125== by 0x5371496: QScriptValue::call(QScriptValue const&,
>> QList<QScriptValue> const&) (qscriptvalueimpl_p.h:719)
>> ==20125== by 0x80FA4C1: MidiObject::receive(MidiCategory, char, char,
>> char, QString) (midiobject.cpp:273)
>> ==20125== by 0x81BA89F: MidiObjectALSASeq::run()
>> (midiobjectalsaseq.cpp:286)
>> ==20125== by 0x502861D: QThreadPrivate::start(void*)
>> (qthread_unix.cpp:184)
>> ==20125== by 0x42CB4BF: start_thread (in
>> /lib/i686/cmov/libpthread-2.7.so)
>> ==20125== by 0x55B461D: clone (in /lib/i686/cmov/libc-2.7.so)
>>
>> Thanks again for your time. Let me know your thoughts and if there's
>> anything else I can do to help.
>>
>> Sincerely,
>> Sean M. Pappalardo
>>
>> <<--------------------------------------------------------------------------------->>
>> This E-Mail message has been scanned for viruses
>> and cleared by >>SmartMail<< from Smarter Technology, Inc.
>> <<--------------------------------------------------------------------------------->>
>>
>>
>
> _______________________________________________
> Qt-interest mailing list
> Qt-interest at trolltech.com
> http://lists.trolltech.com/mailman/listinfo/qt-interest
>
> <<--------------------------------------------------------------------------------->>
> This E-Mail message has been scanned for viruses
> and cleared by >>SmartMail<< from Smarter Technology, Inc.
> <<--------------------------------------------------------------------------------->>
>
>
> ------------------------------------------------------------------------
>
> Subject:
> Re: [Qt-interest] QtScript segfault on C++ signal (dis)connection/firing
> From:
> Kent Hansen <khansen at trolltech.com>
> Date:
> Thu, 15 Jan 2009 18:13:43 +0100
> To:
> qt-interest at trolltech.com
>
> To:
> qt-interest at trolltech.com
>
>
> Hi Sean,
>
> Sean M. Pappalardo wrote:
>
>> Hello again.
>>
>> Kent Hansen wrote:
>>
>>
>>> Hi Sean,
>>> Any chance that the same script engine is being accessed from multiple
>>> threads?
>>>
>>>
>> That's actually almost certain. I'm pretty sure the emitting object is
>> in a different thread than where the engine was instantiated.
>>
>>
>>
>>> QtScript is not thread-safe, so be sure that there's only a
>>> single thread evaluating code (unless you've explicitly added a
>>> synchronization mechanism),
>>>
>>>
>> A single thread ever or one thread at a time?
>>
>>
>
> One thread at a time per QScriptEngine. This also applies to operations
> on QScriptValues (e.g. QScriptValue::call()), since those values are
> bound to a particular engine.
>
>
>>
>>
>>> and make sure that all connections are
>>> queued.
>>>
>>>
>> Signal connections? Can you point me to info on how to do that? (I'm
>> relatively new to C++.)
>>
>> Thank you again very much for your time! This information will help alot!
>>
>>
>
> Connections created with QtScript (e.g. qScriptConnect()) are
> auto-connections; see
> http://doc.trolltech.com/4.4/qt.html#ConnectionType-enum.
> If multiple threads are emitting signals to be caught by a single script
> engine, you should ensure that the objects emitting the signals don't
> live in the script engine thread, and ensure that calls to
> QScriptEngine::evaluate() (or any other operation that can have a
> side-effect on the engine, e.g. QScriptValue::toString()) are done from
> the script engine thread (or synchronized properly -- but I recommend
> the queuing approach). See http://doc.trolltech.com/4.4/threads.html and
> http://doc.trolltech.com/4.4/qobject.html#moveToThread.
>
> Regards,
> Kent
> _______________________________________________
> Qt-interest mailing list
> Qt-interest at trolltech.com
> http://lists.trolltech.com/mailman/listinfo/qt-interest
>
> <<--------------------------------------------------------------------------------->>
> This E-Mail message has been scanned for viruses
> and cleared by >>SmartMail<< from Smarter Technology, Inc.
> <<--------------------------------------------------------------------------------->>
>
>
> ------------------------------------------------------------------------
>
> Subject:
> Re: [Qt-interest] QtScript conceptual questions
> From:
> Kent Hansen <khansen at trolltech.com>
> Date:
> Thu, 15 Jan 2009 18:40:13 +0100
> To:
> qt-interest at trolltech.com
>
> To:
> qt-interest at trolltech.com
>
>
> Hi Sean,
>
> Sean M. Pappalardo wrote:
>
>> Hello again.
>>
>> I have a few high-level questions about QtScript since it seems my
>> understanding is seriously flawed:
>>
>> 1) Does one need to evaluate() an entire script file before calling
>> individual functions contained within? (If not, would it hurt to do so?
>> The point being to report syntax errors at the start of the program
>> instead of at call time.)
>>
>>
>
> evaluate() returns a SyntaxError object if there is a syntax error in
> the script; you can get a string representation by calling toString() on
> it, or access the properties of the object individually (e.g.
> lineNumber, message).
> If there _is_ a syntax error in the script, then no part of the script
> is actually going to get evaluated (the only exception to this rule are
> statements like "0 = 0" and "++0", which will throw a syntax error when
> (if) that statement is actually reached during evaluation).
>
>
>> 2) When one does QScriptValue slot = execute(functionname); what's
>> actually happening? Are we making a self-contained object that is the
>> function? Or simply referencing its instance in the ScriptEngine?
>>
>>
>
> What does execute(functionname) do?
> If you mean something like
>
> QScriptValue fun = engine.evaluate("function foo() { return 123; }; foo");
>
> Then yes, the C++ variable "fun" will now hold a reference to the script
> function "foo".
> "fun.call()" in C++ will achieve the same as doing "foo()" in a script.
>
>
>> 3) When one then does qScriptConnect(object, SIGNAL(signalName()),
>> QScriptValue(), slot); what's actually happening here? Are we connecting
>> the signal to a self-contained object?
>>
>>
>
> There's a magic C++ object behind the scenes that connects to the
> signal. When the signal is received, the C++ arguments are converted to
> script values and the function you passed as the last argument to
> qScriptConnect() is called with the converted arguments.
>
>
>> 4) What implications does all this have for multi-threaded code?
>> Especially considering
>> http://doc.trolltech.com/4.4/qtscript.html#controlling-qobject-ownership
>>
>>
>
> See my latest reply in the "QtScript segfault on C++ signal
> (dis)connection/firing" thread. Object ownership doesn't matter here,
> thread affinity does.
>
>
>> 5) What's the best way to protect against one thread evaluate()ing some
>> function and a connected signal in another thread evaluate()ing another?
>>
>>
>
> Again, see my previous reply. Because of the way signal handling works,
> you'd have to not only synchronize the explicit calls to QtScript
> functions, but also your signal emissions -- which would be a rather
> unworkable solution. Use queued connections, use
> QMetaObject::invokeMethod(), have threads communicate through events;
> see e.g.
> http://doc.trolltech.com/4.4/eventsandfilters.html#sending-events. In
> this way you rely on the (thread-safe!) primitives provided by Qt to
> serialize access to the script engine.
>
> Regards,
> Kent
> _______________________________________________
> Qt-interest mailing list
> Qt-interest at trolltech.com
> http://lists.trolltech.com/mailman/listinfo/qt-interest
>
> <<--------------------------------------------------------------------------------->>
> This E-Mail message has been scanned for viruses
> and cleared by >>SmartMail<< from Smarter Technology, Inc.
> <<--------------------------------------------------------------------------------->>
>
More information about the Qt-interest-old
mailing list