[Development] Property bindings in Qt 6

Wed Oct 2 03:54:04 CEST 2019

Hi Simon,

That sort of coalescing solves one problem, but not the other -
specifically, that if the on-change handlers are sufficiently complex,
their evaluation (and/or the evaluation of their side effects) is
likely to take longer than the time available, causing
incompletely-updated state to be used when rendering the next frame.
More generally, it's a problem with any imperative expressions
(including on-change handlers but also, importantly, bindings) which
form a dirty-expression graph with delayed (pull, i.e. evaluated when
required) evaluation semantics.  The larger the graph, the more likely
you are to run out of time evaluating the expressions when needed
prior to rendering; similarly, the more complex the expressions, the
more likely you are to run into the same issue.  (Technically, it's an
issue with immediate-mode evaluation also, especially since transient
state changes can ripple through the graph causing re-evaluation of
significant numbers of expressions; but it doesn't suffer from the
large-dirty-graph problem, or the potentially reduced timeslice for
evaluation that the delayed evaluation mechanism implies - unless I am
misunderstanding what is being proposed?)

It is this problem which I'm interested in hearing solutions for... I
guess there are two partial solutions built into what you are
suggesting: 1) the coalescing avoids repeated on-change and binding
expression evaluations, reducing the amount of work done per-frame, 2)
the expressions are themselves far more optimised, and the property
mechanism avoids dynamic function call boundaries which prevent
inlining.  Both of these are of course steps in the right direction,
but don't actually solve the issue (they merely reduce the likelihood
of hitting it in usual cases).  I'm not certain that there is a
solution (solving the halting problem ;-), but I had thought of
something like the following:

a) over repeated runs of a given dirty-expression graph, store
statistics for how long evaluation takes for each sub-graph
b) keep counts of which expressions are marked dirty multiple times
per frame (i.e. which ones the coalescing saves evaluation time) and
which are not (i.e. which ones would be better off evaluated in
immediate mode, assuming they don't pull other dirty sections of the
graph)
c) define some heuristics for which sections of graphs are likely to
be "related" or "needed" for a single frame of rendering

Then evaluation of sub-graphs can be metered out over this frame and
the next frame, depending on their predicted evaluation time, and
culled if they will not be needed at all (e.g. text property binding
in a non-visible element, where that property is a leaf-node (thus
changes don't result in side effects) in the graph) which could
prevent large amounts of unneeded work (text layout comes to mind).
The culled sub-graphs can be metered out over subsequent frames during
idle time, to prevent the dirty graph from growing too large in case
the state changes in such a way that those are now all required (e.g.
some element has visible property set to true, meaning that all of its
children are now "needed").

Maybe that's all too complex, and the overhead of adding that dynamic
run-time analysis might largely outweigh the advantages you get from
it (specifically, from avoiding unnecessary work altogether in the
culled bindings cases, while still avoiding "massive graph of dirty
bindings which need to be synchronously evaluated" in the "page
becomes visible" case).  I'd be interested to know what other
solutions are being considered.  Some automatic thread pool for
evaluation might be an interesting research topic, but the dependency
graph and the cost of marshalling and context switches would probably
invalidate the gains, I guess..

Best regards,
Chris.

On Sat, Sep 28, 2019 at 4:49 PM Simon Hausmann <Simon.Hausmann at qt.io> wrote:
>
> Hi,
>
> You’re spot on with your observation. It’s interesting to see how you ran into the same issues back then :)
>
> Samuel (or was it Olivier?) found a solution to that that I find very elegant and would like to consider for QtQuick in the future: Instead of an idle timer, all imperative triggered onFooChange handlers are queued up and called right before the next frame. That avoids repeated change handler calls and worked out beautifully.
>
> Generally, narrowing the scope down to frame based rendering feels very natural.
>
> I understand though that not everybody agrees on that, so perhaps this needs to be an opt-in mechanism that we can enable for QtQuick.
>
> Simon
>
> > On 27. Sep 2019, at 03:51, Chris Adams <chris.adams at qinetic.com.au> wrote:
> >
> > Hi Simon,
> >
> > Nice work!  This was one of the things which Aaron mentioned as being
> > a goal he had for QML way back in the day, and I'm glad to see that
> > someone is doing the work to make it a reality.
> > In theory, one of the issues with pull-style bindings is that large
> > dependency chains can be built up, resulting in large evaluation-debt
> > which must get paid within a single short timeslice to avoid skipping
> > frames (see https://lists.qt-project.org/pipermail/development/2012-March/002390.html
> > for some discussion of this problem).
> > How do you intend to avoid this issue?  Are you implementing some form
> > of evaluation metering (i.e. using idle time to evaluate segments of
> > evaluation-debt and mark those as clean) to mitigate the problem?
> >
> > Best regards,
> > Chris.
> >
> >> On Fri, Sep 27, 2019 at 1:06 AM Simon Hausmann <Simon.Hausmann at qt.io> wrote:
> >>
> >> Hi,
> >>
> >> Earlier this year, Olivier, Samuel, Auri and I worked on a project to re-evaluate how we could bring the declarative Qt Quick approach of doing user interfaces closer to C++, in order to allow building and running user interfaces in very memory and processor-power constrained environments. There were many different outcomes of this. One of them was that we figured out a way to compile QML binding expressions down to full C++, without any run-time interpretation. This required building a new way of defining properties and their relationships, a new property binding system. The results were so convincing that the plan was born to productize this for Qt 6 in multiple layers and steps. I'd like to initiate a first step in that direction by proposing API and functionality for Qt 6 and briefly outline how we see the building blocks apply to QML and Qt Quick:
> >>
> >> In QML, today, properties consist of a type, a setter function and a getter function, and the functions are implemented by the developer. There is also a change signal that needs to be emitted when the value changes.
> >>
> >> Binding expressions declared in .qml files are created behind the scenes and the QML engine makes sure to call the getter functions during the evaluation and the setter function to write the result. Through a connection to the change signal, bindings are automatically re-evaluated when properties change and the new values are passed to the setter functions. It's pretty magic and it works, but it requires a fair amount of indirection and side-loading of data structures.
> >>
> >> I would like to propose an API that replaces the setter and getter functions on objects with a new property template class that encapsulates the property value instead, and the ability to tie binding expressions to these properties for automatic updates. In short, it looks like this:
> >>
> >>    QProperty<QString> surname("John");
> >>    QProperty<QString> lastname("Smith");
> >>
> >>    QProperty<QString> fullname;
> >>    fullname.setBinding([&]() { return surname() + " " + lastname(); });
> >>
> >>    qDebug() << fullname(); // Prints "John Smith"
> >>
> >>    surname = "Emma"; // Marks binding expression as dirty
> >>
> >>    qDebug() << fullname(); // Re-evaluates the binding expression and prints "Emma Smith"
> >>
> >>
> >> You can see a work-in-progress patch for this in Gerrit at
> >>
> >>    https://codereview.qt-project.org/c/qt/qtbase/+/275352
> >>
> >>
> >> The basic data structure behind this is the property value itself as well as doubly linked lists to track dependencies between properties and binding expressions. Due to the encapsulation of the data itself in a class, it is possible to do a lazy evaluation of bindings. (Credit goes in particular to Olivier for the idea and first implementation in our project)
> >>
> >>
> >> Once this class and its documentation is complete, the next step is to build a bridge to the QML engine and the moc, so that it's possible to associate binding expressions in .qml files with properties declared this way. Similarly, it needs to be possible to access such properties through the meta-call, if they are placed inside Q_OBJECT classes.
> >>
> >> The next step is to begin applying this to the implementation of Qt Quick. Some of which may require shims for the public Qt Quick API (to keep it Q_PROPERTY based), and for the private Qt Quick types the idea would be to start using QProperty.
> >>
> >> Finally, once all the pieces are in place, we hope to extend the qml tooling to compile the binding expressions in .qml files to C++ that uses this more light-weight property system whenever possible. Ulf has been working towards this from the QML engine direction (see the recent email about moc and meta-type extraction) and Fabian has been working on the QML linter as a starting point towards a compilation model for QML.
> >>
> >>
> >> This is our rough plan of how we'd like to address one aspect of QML and Qt Quick today. We are looking forward to any feedback and questions to help us review and refine this design.
> >>
> >>
> >> Simon
> >> _______________________________________________
> >> Development mailing list
> >> Development at qt-project.org
> >> https://lists.qt-project.org/listinfo/development